Fixed the FlexLM on the zone

Posted: October 4, 2011 in Uncategorized
Tags: , , ,

I know this sounds crazy, but we actually got some support from Oracle on the latest issue. I opened a ticket regarding the SI_HW_Provider issue I blogged about last week, not expecting any help from official channels (just like the zone inode issue a few months back). But they are actually acknowledging there is a bug there, or at least the potential of a bug. And guess what? I also got an alternative fix from the kernel group using ‘mdb’ instead of dtrace. This fix however will fix the entire global zone and all non-global zones.

Not bad! I actually feel a little better about Oracle buying and swallowing Sun. It seems like there may be hope and that support is getting better. There have been some definite improvements over the last five months, the inode issue being the lone stand out. But Brendan Gregg stepped in on that one, just outside the official Oracle channels.

So when I last left this issue I had a little dtrace fix for the SI_HW_Provider combined with the inode dtrace fix so that the license manager would start the license binary, there were issues shown via truss that it could not establish the lock file. A coworker found some info that pointed me in the right direction.

It appears that the licensing service can’t handle what I now consider a reasonable “open file limit”. The ulimit for open files must be no more than 1,024. I created a Solaris project user.flexlm user and configured it with “process.max-file-descriptor=(priv,1024,deny)” and boom, the service starts right up and does his job.

The next step is to start the old X window application. It starts up, can display information about the licensing that it discovers from the license file itself but it never connects to the license manager. In fact it never even tries. I ran truss-like debugging on it and I saw it died with opening the socket. After a couple of hours looking at it last week, yesterday, and today, I dropped my max-file-descriptors down to 1024 (ulimit -n 1024) and problem is solved.

So the final solution? System project now has a max-file-descriptor of 1,024 open files per user and my choice of methods to change the SystemInfo Hardware Provider, plus the existing inode dtrace edit. With any luck and limited future meddling by those Oracle kids we should be able to keep this very ancient, tired X window application compiled for SPARC under Solaris 2.5 running and limited to as small a zone as it needs but on as fresh of hardware as we feel inclined to use.

I think I deserve a cup of coffee.

mokka pot espresso or AeroPress ? That is the question.

  1. Bob says:

    Would love to know the solution you received from Oracle. Trying to convince a co-worker to implement your dtrace hack and would like to know what Oracle recommended.

    • coffeeortech says:

      It is actually covered over the course of several blogs…
      So three things – at least for my really, really, really old license and flexlm binaries. You may have a different experience if it is newer.
      1. the no. of open files ulimit has to be 1024 or less for all users, the flexlm user and the user who will be checking out a license
      2. Need the dtrace script that runs as the flexlm user that fixes the inodes in memory – detailed in this post
      3. Need to edit the kernel using mdb to change it from “Oracle Corporation” to “Sun_Microsystems ” as detailed here –
      That’s the limit of the support from Oracle regarding this.

      And, a follow on comment, daniel babault suggests running it on reboot just in case a kernel patch has been applied.

      And for the record, Oracle support for hardware and Solaris? Still sucks.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s