Solaris Binaries on a 64-Bit System

I was poking around my filesystem the other day, and I discovered that there was not only a /bin/sparcv9 directory, which I knew about, but also a /bin/sparcv7 directory, which I was vaguely aware of, and also that the binaries in those directories didn't match each other at all.

I mean, I expected the sparcv9 stuff not to match - after all, it's 64-bit, instead of the default 32-bit, so of course, the binary would be different. Each of them are replacements for the binary in /bin, which stands to reason. The odd thing is that the binary in /bin actually has the same inode every time, and in fact, there are 34 hard links to the replaced binaries, and most of these "binaries" don't have sparcv7/9 equivalents. The oft-linked file is 5540 bytes long, and it issues a sysinfo() call, and then tries to find an ISA equivalent and launch it instead.

(When looking at the table, the file sizes are shown for comparison purposes to the 5540 byte size of the multi-dispatch binary, not for comparison to each other - it is understandable that the 64-bit versions are larger. Also, note that w and uptime are hard links to the same binary in each machine class, which is not atypical. Finally, there are two more binaries in /bin/sparcv9 which are also in /bin, although not in /bin/sparcv7, pvs and ls. These are not hard links to the multi-dispatch binary, as all of the others are, but are their own files in /bin.)

/bin Size of /bin/sparcv7 version Size of /bin/sparcv9 version
adb129672192488
gcore1488419640
ipcs (not suid)11672 (suid)16136 (suid)
kgmon2059227296
plimit1148414584
prex6672881536
ps (not suid)28080 (suid)37088 (suid)
savecore1267616368
setuname862011744
tnfxtract881611840
truss130608178472
uptime (not suid)11528 (suid)15584 (suid)
w (not suid)
pvs (13144)missing17976
ls (18120)missing18120
/usr/ucb Size of /usr/ucb/sparcv7 version Size of /usr/ucb/sparcv9 version
ps (not suid)23508 (suid)32248 (suid)
/usr/sbin Size of /usr/sbin/sparcv7 version Size of /usr/sbin/sparcv9 version
crash138616177648
lockstat2666431840
prtconf (not suid)19540 (suid)24496 (suid)
swap (not sgid)9848 (sgid)12856 (sgid)
sysdef (not suid)24728 (suid)33448 (suid)
whodo (not suid)12940 (suid)17416 (suid)
/usr/proc/bin Size of /usr/proc/bin/sparcv7 version Size of /usr/proc/bin/sparcv9 version
pcred57087936
pfiles1018013968
pflags1047614064
pldd59088128
pmap1010813392
prun50927072
psig737210208
pstack926413496
pstop50687040
ptime66968968
ptree815211256
pwait67249392
pwdx835611832

The other program which has the same inode as the multi-dispatch binary is /usr/lib/isaexec, which does not have a platform specific alternative. It also does not have a manpage, although the library call of the same name does.

Of all of these programs, the only ones which seem to inspect their name to see how to operate (a common thing for a multi-call binary to do) are w and uptime, which behave like w unless their name is uptime. The rest of them operate as their "true" name if symlinked. However, an interesting thing occurs if one makes a hard link in /var/tmp, for example, which is a world-writable sticky directory on the same partition as /usr/bin on my system. If one issues the command ln /bin/ps /var/tmp/w and then tries to run this bogus w, the system responds with cannot find/execute "w" in ISA subdirectories. Clearly, the multi-call binary dereferences symlinks, but when it finds a final filename, /some/path/to/a/file, it tries to execute /some/path/to/a/ISA/file. truss confirms this.

The last oddity is in /usr/ccs/bin. There are a number of utilities there which have 64-bit variants, specifically because they seem to deal with 64-bit object files, and they need to be reasonably intelligent about it. However, instead of using the multi-dispatch ISA-finding binary that is used elsewhere on the system, Sun merely provided /usr/ccs/bin and /usr/ccs/bin/sparcv9. I imagine that if a /usr/ccs/bin/sparcv7 directory were created and all of the programs in /usr/ccs/bin which have equivalents in /usr/ccs/bin/sparcv9 were placed there, and then the multi-call binary were hard linked in /usr/ccs/bin, the Right Thing would happen. I have not tested this, however, because Sun releases patches to programs in /usr/ccs/bin on occasion, and I do not want my copy of ps to stop working because it thinks that it is now ld, for example. (I suspect that this is packaging error on Sun's part.) Clearly, /bin/pvs and /bin/ls fall into this category as well.

The long and the short of it is that Sun generally releases a binary which does its darndest to launch the most appropriate binary for your system, failing back to backwards compatible versions if it must. As a result, when on a sparcv9 processor, in general, you shouldn't need the platform-specific variants on your path. The counterexamples are /bin (or /usr/bin, to which /bin is a symlink), since there are two programs there which aren't properly linked, and /usr/ccs/bin, since none of it is properly linked. Apparently, isaexec is the canonical name of this multi-dispatch program, and it is there solely to execute programs for a given ISA. (This also explains why Sun likes keeping /bin a symlink to /usr/bin - it lets the hard links work, even if /usr is not on /.)

Update: I have since discovered that Sun documents some of this oddness, in their Solaris 64-bit Developer's Guide, in as much as they describe isaexec and explain how they use it.

Back to the regularly scheduled drivel.