So, I figured (Open-) Solaris and VirtualBox are both owned by Sun now, they should go together splendidly.
I opted for the Sun-offering rather than Nexenta etc. since I intend to test some things in a "Sun environment", that is, make sure things work with the on-board facilities. Otherwise "Sun-kernel and GNU userland" would likely have been my preference.
Now, there are two different offerings, mind. The freely available download of OpenSolaris "Indiana", for a start. At 686 MB, this comes without a compiler, any compiler, it seems, and that means, no Sun compiler and no GNU compiler. It seems nice enough otherwise. However if I wanted to install the Virtual Box additions so I could move the mouse-pointer in and out of the VirtualBox window (or even integrate the Solaris instance's windows with my linux host's), the keyboard went berserk (in the guest only, and in X only), so I had to click Install … to mount the image, and then kill X to cd /media/V*; pkgadd -d *.pkg really quickly. On the upside, it came with some sort of graphical package-manager (which possibly allows you to pull the GNU compiler collection).
Addendum: It seems like the keyboard situation can be avoided by connecting using RDP, see below.
Anyway, I needed the Sun compilers, so I had to opt for the Developer Express something offering (what's with these names?), which in theory requires you to register before download. The good news is, if you're already on the Sun payroll, your normal log-in credentials will do the trick. This image weighs 3.7 GB; the minimum disk requirements are in the region of 8 GB, with 20 GB recommended. It includes the compilers (and auto-tools) of course, but it's not just the other image, plus devel. GNOME looks subtly different, I had no trouble installing the VirtualBox additions, and if there is a graphical package manager I haven't found it so far. But then I haven't looked very hard since so far all I needed was to go to sunfreeware.com (or blastwave, whichever you prefer — blastwave and pkg-get actually handle dependencies, so that's probably what you want, more on that later) to install screen "so the pain will stop." Seriously, that should be their tagline. "Screen. So the pain will stop." But I digress.
Addendum: Since I was only changing config-files at first, I only used vim in the first hour. Yes, you know what's coming. Once you try to do real work, you'll find there is no emacs. No, no xemacs either. So I guess I want to install pkg-get after all. But seriously. A "developer-edition" with no emacs? WTH?? So pkg-get (or pkgadm, but PBA that there is another pkgadm in $PATH!) it is! pkg-get -i xemacs, add /opt/csw/xemacs/bin to PATH, then let the configuring begin!
It's been a while since I worked full-time on Solaris, but some things never change, it seems. If you become root, you'll still get a /bin/sh. If you call up vi, that's what you get, even though there's a vim included. Yes, the defaults are set to hurt.
And some of them are a tad unexpected.
So, net didn't work out of the box. Or so it seemed, given that browsing didn't work (Control-T, new tab in Firefox 2 rather reliably breaks the session for me, taking 100 % CPU on the host with the guest and VirtualBox both going unresponsive), and ping also didn't seem to work. Apparently, this is not the best method to verify your net — that would be something like trying to ssh to a numeric address —, and this is why: first, ping is noted in the VirtualBox docs as only semi-supported with VirtualBox's default networking-setup ("NAT"). And then second, the Solaris' default nsswitch.confdisables DNS requests. Oh, there's a plethora of fine example files, and one of them, nsswitch.dns, does just what you need, namely directly host-queries the right way, but the default once more is set to hurt.
I still need to use Control-N for new window instead of tabs, though …
Now, VirtualBox. In theory and if you use the Open Source Edition, there will be packages for required kernel-modules in your distribution. Since I somehow ended up with a kernel and no matching modules (and the VirtualBox's compile the modules for the current kernel refuses on the grounds that you should use the packages — probably just an extra echo and an exit somewhere in the script, but I didn't bother to look), and since the modules tend to compile with no hassle at all, I opted for Sun's RPMs. (Which is more than I can say for vmplayer player with current kernels; I had to patch with anyany and then add selected lines from two other patches to get VMWare's modules to compile with a recent compiler, for a recent kernel. I've adored VMWare for years, but that whole "the modules almost never build on recent kernels" bit is beginning to irk me something fierce. Don't take my word for it, you have reason to expect for me to be biased, so google it.) So I got the VirtualBox 1.6 packages, figuring that if Sun put in any extra work for Solaris guests after acquiring innotek, the safest bet would be the very recent versions. And sure enough things work splendidly for me so far, although according to the docs shared folder is supported for Windows and Linux guests only, so if you wish to exchange files between the host and a Solaris guest, you're stuck with using an NFS server (which you may or may not have running already) or possibly SMB (which you really really want if you also have Windows guests — compiling in emulated Windows on an emulated disk takes roundabout forever, while having the win-guest use the hosts file-system via Samba is surprisingly snappy).
If your NFS-mount fails and the syslogs shows messages of the form
mountd: NFS mount of /home attempted from 184.108.40.206
mountd: NFS request from your.machine.com originated on insecure port, psychoanalysis suggested
mountd: Blocked attempt of 220.127.116.11 to mount /home
Solaris tried mounting from a non-privileged port. One way to heal that is to allow that sort of thing in the Linux host's /etc/exports by adding the insecure option to that filesystem:
Also, if you have trouble with your locking (as evidenced by apps, well, locking up), you may wish to to make sure that you are using the kernel-space NFS server rather than the user-space one on the Linux host. (If you change over now and the mount on the solaris guest fails afterwards, try passing -overs=3 to downgrade to protocol version 3.)
On a semi-related note, if you wish to run VirtualBox on a server, an analog of this incantation may be for you: VBoxHeadless -startvm "Solaris Developer" — then use rdesktop-vrdp with the server's name or address to pick up the session, possibly also passing -k fr or some such to the latter to get the keyboard right. (In theory, this should also support seamless mode when passing -A to the client, but that won't work for me, while seamless works when not using RDP. Also, GNOME offers me 1024x768 as maximum resolution for the Solaris even with installed VBoxAdditions (if not on RDP I get it too, but it can easily be fixed by just resizing the window to anything I like). No answer on #VirtualBox yet …)
Mind that the default build on Solaris 10+ x86 uses gcc, not Sun's fortecc. To build with forte all the same, use BUILD/compile-solaris-amd64-forte-debug (depending on your environment, you may have to put a # in front of EXTRA_64_BIT="-xarch=amd64"). Otherwise, pkg-get -i gcc3 (or later, but gcc3 is what our reference build uses).
As far as I remember from my futile attempts at getting the developer edition to run under qemu-kvm the installer acually asks you if you want to use DNS. The default is, of course, not to use it. New fangled nonsense.
Migrating a Windows (XP in my case) image from VMWare to VirtualBox (because of the aforementioned reasons against VMWare) is annoying and non-trivial, but possible by the way; as long as you still have a working instance of VMWare, you don't even need your XP DVD (and thank G-d for that, if I could find the smegging thing, I might have installed over by now; I can find the licence alright, lot of good that does me, but not the bleeding disc):