SmartOS and the global zone
One of the most common issues new users of SmartOS face is understanding the role and design of the global zone. Often they will download SmartOS and try to start using it as they would any other Unix operating system, but quickly run into basic problems such as installing packages or adding users. However, SmartOS is not your usual operating system, and it is imperative that you understand two key principles:
SmartOS is specifically designed as an OS for running Virtual Machines, not as a general purpose OS.
The global zone is effectively a read-only hypervisor, and should only be used for creating and managing Virtual Machines. Everything else should be performed inside Virtual Machines.
I should be clear at this point that I am specifically addressing the issue of downloading and running your own SmartOS installation. If you provision a SmartMachine from Joyent, then you are running inside a Virtual Machine, and this post does not apply - you are free to start serving up awesome applications!
Let’s look at a few aspects of the global zone which make it a great fit for its intended purpose, and help explain why it doesn’t work as you might expect.
The global zone is a ramdisk
This is the key reason why things don’t work as you might expect. SmartOS does not install to disk like other operating systems, instead it boots directly from USB/CD/PXE into a mostly read-only environment.
Why is it done this way? There are a number of good reasons:
- Upgrades are trivial. No more patching, just reboot into a new image!
- Increased disk space. No wasted space on disk to hold the OS, all the space is dedicated to VMs and user data.
- Increased disk performance. It’s common with other systems to have your OS installed to a pair of mirrored disks and then pool the remaining disks for data. With SmartOS you can have all your disks in the same RAIDZ pool, increasing performance.
- Additional security. Most of the system files are read-only, and
/etcis re-created on each boot, making it much harder to exploit.
- Increased stability. Ever had your root disks start to fail and system commands no longer run? This doesn’t happen on SmartOS.
- Much simpler to install and provision, especially when you have a large number of machines.
So, what does this look like, and what are the implications?
# zonename global # df -h / /usr Filesystem size used avail capacity Mounted on /devices/ramdisk:a 264M 219M 45M 83% / /devices/pseudo/lofi@0:1 376M 354M 22M 95% /usr # lofiadm Block Device File Options /dev/lofi/1 /usr.lgz Compressed(gzip)
The root file system is a ramdisk, and
/usr is a read-only loopback mount of
a single compressed file held on the ramdisk.
Apart from a few specific files and directories listed below, this means that you cannot change the global zone. To emphasise this point further:
- You cannot add users.
- You cannot write anywhere under /usr.
- You cannot permanently store or change files under
- Changes to SMF services will be reset each reboot.
There are plenty of other things you cannot do, but hopefully this gives you some idea of the restricted nature of the global zone environment.
If you are looking for a general purpose OS to serve as a NAS etc, then you can do worse than give OmniOS a spin - it has most of the same features as SmartOS, but in a normal install-to-disk configuration.
Having said all that, let’s look at some ways you can manage the global zone, if you have settled on running SmartOS.
So what can I do?
Firstly, let’s look at the key writeable areas:
Filesystem size used avail capacity Mounted on zones 2.0T 143M 1.8T 1% /zones zones/var 2.0T 4.0G 1.8T 1% /var zones/opt 2.0T 55K 1.8T 1% /opt zones/usbkey 2.0T 154K 1.8T 1% /usbkey /usbkey/shadow 1.8T 154K 1.8T 1% /etc/shadow /usbkey/ssh 1.8T 154K 1.8T 1% /etc/ssh
zones is the big zpool which is spread across all your disks, and SmartOS
will create a number of file systems on it for use in the global zone:
This is where your Virtual Machines are stored, the datasets used to create them, and some other key files related to zones. Don’t mess around in here unless you know what you are doing.
We all like logs, so those are retained in
/var as normal, as well as various
state files such as the current list of
imgadm datasets and the SSH host keys.
/opt is your main hook into more advanced setup of the global zone. SmartOS
will import any SMF manifests it finds in
/opt/custom/smf, which allows you
rc.local functionality as demonstrated in this
/opt is writeable you can also install packages as per this wiki
page, however you need
to bear in mind that, as explained earlier, things such as adding users are not
possible, so you may see various errors.
When you first boot SmartOS and go through the rudimentary installer the
details you provide are stored in
/usbkey/config, which is used during boot
to configure the machine. If you need to change any of those variables, this
is the file you should edit.
There are also some variables which aren’t set up by default, so if you want to
- Install an
authorized_keysfile for the root user.
- Set a keyboard map.
then have a read of this post I wrote a while ago to find out how to configure those.
/etc/shadow and /etc/ssh
In order to support some very basic configuration, a few
/etc files are their
own mount points onto /usbkey so that changes to them are saved.
/etc/shadowso that you can change the root password.
/etc/sshis initially where the SSH host keys were stored, however they are now stored under /var/ssh so don’t be surprised if this mount point disappears at some point.
While we’re on the subject, let’s complete this post with a look at how the global zone is implemented, for interested readers.
There is a good wiki guide which describes how to mount the USB key. Once you have done that you can take a look at the GRUB configuration:
: Assumes your current working directory is the USB key mountpoint. # less boot/grub/menu.lst ... title Live 64-bit (text) kernel /platform/i86pc/kernel/amd64/unix -B console=text,root_shadow='<crypt>',smartos=true module /platform/i86pc/amd64/boot_archive title Live 64-bit (noinstall) kernel /platform/i86pc/kernel/amd64/unix -B console=text,root_shadow='<crypt>',standalone=true,noimport=true module /platform/i86pc/amd64/boot_archive ...
There are a few boot configuration variables which alter how the system is
started, and you can see how they are used by grepping for them in the SMF init
This configures the default root password prior to /etc/shadow being mounted,
and changes for each release. If you need to know what it is, then browse the
download site and look at the
SINGLE_USER_ROOT_PASSWORD.release.txt which correlates with your
output, or if you wish you could generate your own password and paste the new
crypted string in here.
Alternatively as a convenience you can find the default root password inside
# cat platform/root.password seitee3oome4aiPh
This variable defines whether to perform the normal SmartOS global zone
initialisation, such as mounting
/usbkey and configuring the system from the
/usbkey/config file (see
If left unset (like in the ‘Live 64-bit (noinstall)’ boot option), these will
not be performed.
standalone=true should be set in those cases to avoid
trying to be a Joyent compute node.
Setting this will skip any configuration and mounting of zpools, and is useful for if you have issues during the first installation and need to check disks, etc.
It’s likely that you do not need to change any of these variables, and if you
hit problems simply use the
noinstall boot option and figure things out from
The only other files on the USB key are:
# find platform -type f platform/i86pc/kernel/amd64/unix platform/i86pc/amd64/boot_archive platform/i86pc/amd64/boot_archive.gitstatus platform/i86pc/amd64/boot_archive.manifest
These are mostly self-explanatory:
unixis the SmartOS kernel.
boot_archiveis the ramdisk image containing the entire OS.
boot_archive.gitstatuscontains the tip revisions of the github repositories used to build that particular image.
boot_archive.manifestcontains MD5 checksums of the OS files.
Given we have just explained about the
platform directory, it is also worth
pointing out that you do not have to reflash your USB key every time, which is
great if your server is inaccessible. Instead, assuming your USB key is big
enough, you can simply download the newest
file, move the existing
platform directory out of the way, and unpack
platform-latest.tgz there instead. Doing this as an atomic operation is
preferred, for example:
: As of writing this is 20121115T191935Z # gtar zxf ~/platform-latest.tgz # mv platform platform-$(uname -v) && mv platform-20121115T191935Z platform
This avoids having no
platform directory, just in case you have a power cut
at that exact point in time!
If you want to set up a file server or similar, then SmartOS is probably not for you. However, if you are interested in running Virtual Machines and are able to do all of your work inside them, then SmartOS is perfect for that purpose, and has a number of advantages over other operating systems.
Hopefully this has been useful, even if it is to deter you from using SmartOS!
- 16 Jul 2015 » Reducing RAM usage in pkgin
- 03 Mar 2015 » pkgsrc-2014Q4: LTS, signed packages, and more
- 06 Oct 2014 » Building packages at scale
- 04 Dec 2013 » A node.js-powered 8-bit CPU - part four
- 03 Dec 2013 » A node.js-powered 8-bit CPU - part three
- 02 Dec 2013 » A node.js-powered 8-bit CPU - part two
- 01 Dec 2013 » A node.js-powered 8-bit CPU - part one
- 21 Nov 2013 » MDB support for Go
- 30 Jul 2013 » What's new in pkgsrc-2013Q2
- 24 Jul 2013 » Distributed chrooted pkgsrc bulk builds
- 07 Jun 2013 » pkgsrc on SmartOS - creating new packages
- 15 Apr 2013 » What's new in pkgsrc-2013Q1
- 19 Mar 2013 » Installing SVR4 packages on SmartOS
- 27 Feb 2013 » SmartOS is Not GNU/Linux
- 18 Feb 2013 » SmartOS development preview dataset
- 17 Jan 2013 » pkgsrc on SmartOS - fixing broken builds
- 15 Jan 2013 » pkgsrc on SmartOS - zone creation and basic builds
- 10 Jan 2013 » Multi-architecture package support in SmartOS
- 09 Jan 2013 » Solaris portability - cfmakeraw()
- 08 Jan 2013 » Solaris portability - flock()
- 06 Jan 2013 » pkgsrc-2012Q4 illumos packages now available
- 23 Nov 2012 » SmartOS and the global zone
- 24 Oct 2012 » Setting up Samba on SmartOS
- 10 Oct 2012 » pkgsrc-2012Q3 packages for illumos
- 23 Aug 2012 » Creating local SmartOS packages
- 10 Jul 2012 » 7,000 binary packages for OSX Lion
- 09 Jul 2012 » 9,000 packages for SmartOS and illumos
- 07 May 2012 » Goodbye Oracle, Hello Joyent!
- 13 Apr 2012 » SmartOS global zone tweaks
- 12 Apr 2012 » Automated VirtualBox SmartOS installs
- 30 Mar 2012 » iptables script for Debian / Ubuntu
- 20 Feb 2012 » New site design
- 11 Jan 2012 » Set up anonymous FTP upload on Oracle Linux
- 09 Jan 2012 » Kickstart Oracle Linux in VirtualBox
- 09 Jan 2012 » Kickstart Oracle Linux from Ubuntu
- 22 Dec 2011 » Last day at MySQL
- 15 Dec 2011 » Installing OpenBSD with softraid
- 21 Sep 2011 » Create VirtualBox VM from the command line
- 14 Sep 2011 » Creating chroots for fun and MySQL testing
- 30 Jun 2011 » Graphing memory usage during an MTR run
- 29 Jun 2011 » Fix input box keybindings in Firefox
- 24 Jun 2011 » How to lose weight
- 23 Jun 2011 » How to fix stdio buffering
- 13 Jun 2011 » Serving multiple DNS search domains in IOS DHCP
- 13 Jun 2011 » Fix Firefox URL double click behaviour
- 20 Apr 2011 » SSH via HTTP proxy in OSX
- 09 Nov 2010 » How to build MySQL releases
- 29 Apr 2010 » 'apt-get' and 5,000 packages for Solaris10/x86
- 16 Sep 2009 » ZFS and NFS vs OSX
- 12 Sep 2009 » pkgsrc on Solaris
- 09 Dec 2008 » Jumpstart from OSX
- 31 Dec 2007 » Set up local caching DNS server on OSX 10.4