I know this sounds crazy, but we actually got some support from Oracle on the latest issue. I opened a ticket regarding the SI_HW_Provider issue I blogged about last week, not expecting any help from official channels (just like the zone inode issue a few months back). But they are actually acknowledging there is a bug there, or at least the potential of a bug. And guess what? I also got an alternative fix from the kernel group using ‘mdb’ instead of dtrace. This fix however will fix the entire global zone and all non-global zones.
Not bad! I actually feel a little better about Oracle buying and swallowing Sun. It seems like there may be hope and that support is getting better. There have been some definite improvements over the last five months, the inode issue being the lone stand out. But Brendan Gregg stepped in on that one, just outside the official Oracle channels.
So when I last left this issue I had a little dtrace fix for the SI_HW_Provider combined with the inode dtrace fix so that the license manager would start the license binary, there were issues shown via truss that it could not establish the lock file. A coworker found some info that pointed me in the right direction.
It appears that the licensing service can’t handle what I now consider a reasonable “open file limit”. The ulimit for open files must be no more than 1,024. I created a Solaris project user.flexlm user and configured it with “process.max-file-descriptor=(priv,1024,deny)” and boom, the service starts right up and does his job.
The next step is to start the old X window application. It starts up, can display information about the licensing that it discovers from the license file itself but it never connects to the license manager. In fact it never even tries. I ran truss-like debugging on it and I saw it died with opening the socket. After a couple of hours looking at it last week, yesterday, and today, I dropped my max-file-descriptors down to 1024 (ulimit -n 1024) and problem is solved.
So the final solution? System project now has a max-file-descriptor of 1,024 open files per user and my choice of methods to change the SystemInfo Hardware Provider, plus the existing inode dtrace edit. With any luck and limited future meddling by those Oracle kids we should be able to keep this very ancient, tired X window application compiled for SPARC under Solaris 2.5 running and limited to as small a zone as it needs but on as fresh of hardware as we feel inclined to use.
I think I deserve a cup of coffee.