Openwall Linux 3.0: Linux for the security-conscious
Openwall GNU/*/Linux (or, in short, Owl) is a security-enhanced Linux distribution, intended as a server platform. Almost five years after the (then Linux 2.4-based) 2.0 release and more than 10 years after the start of the project, the developers have now released a major update in version 3.0. It is based on a RHEL 5.5-like Linux 2.6 kernel along with optional OpenVZ container-based virtualization.
The first question most people will have is: what is so "security-enhanced" about Owl? Aren't major Linux distributions such as Red Hat Enterprise Linux, Ubuntu, openSUSE, and so on secure? Of course, they continuously patch known security vulnerabilities and some of them (Red Hat in particular) implement security features to decrease the impact of vulnerabilities, but none of them really are focused on preventing vulnerable software from getting into the distribution in the first place.
For the Owl developers, software design and code quality are the first priorities. Before they include a package in their distribution, they check whether it typically runs with elevated privileges (such as SUID/SGID programs) or whether it uses data obtained over the network as its input (such as network services). Both cases are possible attack vectors, so the Owl developers audit the source code and implement safer default configurations. They also modify the software to introduce privilege separation and to apply the least privilege principle. The developers describe their philosophy in the CONCEPTS page on the project's web site.
Since Owl 2.0 (which got security fixes and a couple of bug fixes over four years), there have been a lot of changes in the distribution. The 2.4 kernel has been swapped for OpenVZ's latest from their "RHEL5 testing" branch (currently 2.6.18-194.26.1.el5.028stab079.1), with some additional security-related patches. There is now also Ext4 filesystem support and the installer offers Ext4 by default. Xz compression support has also been added throughout the whole system: not only with commands like xz, xzcat, xzdiff, xzgrep, xzless, and so on, but also with xz support in tar, rpm, less and other tools.
Installation
Compressed ISO images for x86 and x86_64 can be downloaded from the project's home page or one of the mirrors. Alternatively, users who want to donate money to Openwall can purchase a CD. Both the ISO images and the CD contain a live system and an installer program, as well as the full source code and a build environment. There are also OpenVZ templates that can be used to run in OpenVZ containers (operating system-level virtualization). This could be useful to isolate various virtual servers, each in their own Owl OpenVZ container, on top of another Linux distribution or Owl itself, which has the necessary kernel and tools to run OpenVZ containers.
If you choose "normal" in the boot menu, the Owl CD boots into single user
mode. Once you see the shell prompt, you can choose to configure the
system (such as localization, timezone, network, ...) with setup
and exit the shell to let the system boot into the multi-user live mode, or
you can start the installer with settle. The installer has an
ncurses-based bare-bones interface and doesn't hold the user's hand, but it
does the job.
The focus on security is particularly clear in the window
where the user has to choose a root password: the text describes a handful
of requirements for a secure password and shows a possible secure but easy
to remember password like "While3frail8buggy". If the user fails to enter a
secure password, the installer refuses the password and asks for a new
one. Also, Owl 3.0 is one of the few Linux distributions that still uses
LILO instead of the more complex (and hence, potentially less secure) GRUB.
No SUID programs
The announcement of Owl 3.0 specifically mentions the effort the developers have done to eliminate SUID programs:
This assertion can be easily verified by a find / -perm -4000 command, which doesn't return any file with the SUID bit set in a fresh Owl install. To make this possible, the Owl developers rewrote the ping command, for example, to run as an unprivileged user. Another interesting rewrite is the passwd command. Traditionally, password hashes and password policy information of all users are stored in a single file, /etc/shadow. This forces passwd to be SUID root, which means that when a user runs the program, it has the privilege to alter all entries in the shadow file, not only the one of this user. As a result, if passwd is compromised because of an exploit, the attacker is able to change all passwords.
The Owl developers have invented an alternative mechanism for password management, which they call tcb. Each user is assigned a separate shadow file, owned by this user, e.g. /etc/tcb/root/shadow owned by the "root" user and the "auth" group, and /etc/tcb/joe/shadow owned by the user "joe" and the "auth" group. This "auth" group may be used to grant a process read access to all password hashes, but the passwd and chage commands are made SGID shadow, so both commands have only access to the user's own password hash and password policy. This move to tcb is transparent for existing applications, because they don't read the shadow file directly but rely on interfaces like PAM (in Owl handled by the PAM module pam_tcb) and NSS (handled by libnss_tcb).
It's interesting to see that other Linux distributions are also trying to remove SUID applications. For example, Red Hat's Dan Walsh wants to replace SUID in most applications by using file capabilities in Fedora 15. On the Ubuntu side, Canonical's Kees Cook is also working on using file capabilities. The Owl developers, though, have another approach, and the lead developer Alexander Peslyak describes some issues with the file capabilities approach. Also of note is what LWN.net guest author Neil Brown had to say about SUID in his article "Ghosts of Unix past, part 4: High-maintenance designs".
Security tools
Owl provides the control command to enable, disable, or configure some security-related facilities. Using the command without options lists all available facilities, their current setting, and any available settings. You can also get the current setting of a specific facility: for example, if you want to know if passwd is using the tcb or the traditional mechanism, just run control passwd. And if you want to see all available choices for the passwd setting, just run control passwd list. You can also change a setting, e.g. control passwd traditional to change the tcb default of passwd to the traditional shadow-based approach. The use of su is by default restricted to root because it has no SUID bit, but if the root user runs control su public, /bin/su gets the SUID bit so every user can run su to assume another user's identity. Under the hood, the control command uses shell scripts in /etc/control.d/facilities/ for the various services.
The Owl developers also ported several programs from OpenBSD, which is also a security-focused operating system: mtree, Vixie cron, telnet (with modifications to introduce privilege separation), netcat, and mailx. In general, software that is imported in Owl gets an average of four patch files to improve security: half of the patches from the Owl developers, the other half imported from various distributions or the BSDs.
The Owl developers have also created some useful security programs that can be used on other distributions. The famous password cracker John the Ripper, for example, is made by them, as is the password strength checker passwdqc, the port scan detection tool scanlogd, and the secure POP3 daemon popa3d.
By default, the system has a fair amount of tools installed, like vim, mutt, netcat, screen, nmap, openntpd, the OpenSSH server and client, postfix, procmail, vsftpd, lftp, and, of course, the already mentioned tools developed by the Openwall team. Owl uses the rpm package manager, but it has no repositories; instead, the user can rebuild the entire system from source with a make buildworld command and install the packages with make installworld. This is also the way to update an Owl system. If you want additional packages, the developers promise that in most cases it is possible to install packages for Red Hat Enterprise Linux, CentOS, or Fedora on Owl.
The Openwall developers have a community wiki with a lot of information about the Owl distribution and their other projects. In addition, the owl-users mailing list is the perfect place for questions about the use of Owl. The development team is rather small but dedicated, and they welcome patches, for which they publish a howto. All in all, Owl is really suited for the security-conscious Linux server administrator, but it also shows that there are alternatives to the security approaches taken by the mainstream Linux distributions.
| Index entries for this article | |
|---|---|
| GuestArticles | Vervloesem, Koen |
Posted Jan 7, 2011 0:58 UTC (Fri)
by Baylink (guest, #755)
[Link] (4 responses)
And I've never really liked it, either... The more standard facilities you replace with your own BFIs, the fewer people have audited those implementations.
Posted Jan 13, 2011 16:20 UTC (Thu)
by solardiz (guest, #35993)
[Link] (3 responses)
Now let's get to the real technical stuff: unlike those older systems, Owl and our "tcb" package is the very first implementation to make use of the shadow files separation to reduce the privileges of the "passwd" command. No other system did it before we did it in Owl. (Now our "tcb" implementation is also reused by ALT Linux's distributions and by Mandriva, which is great. I hope more distributions will follow.)
Yes, replacing standard facilities involves risk of introducing new vulnerabilities. However, with our "tcb", we're reducing privileges of used-to-be-SUID-root programs at the same time - so certain classes of potential vulnerabilities turn into almost non-issues for that reason.
I think that there are more people who have fully audited our "tcb" suite (that's at least its three developers, all of whom are "into security") than people who have audited, say, Linux-PAM (relevant since our "tcb" replaces Linux-PAM's pam_unix with its own pam_tcb). Ridiculously naive security holes in three Linux-PAM modules (upstream, not distribution-specific) were found last year: the modules were accessing users' files as root. These holes would not be in there if just one "security expert" had carefully audited the entire Linux-PAM tree before last year. (None of the affected modules were in use on Owl by default, which is why they had been excluded from our own mandatory audit... although we did build and package them, so we had to release a security update for those who might have made use of the modules. And guess who finally patched the holes in upstream's Linux-PAM? One of the "tcb" developers did.)
Posted Jan 15, 2011 21:37 UTC (Sat)
by gvy (guest, #11981)
[Link] (2 responses)
Thanks!
Posted Jan 16, 2011 19:11 UTC (Sun)
by solardiz (guest, #35993)
[Link] (1 responses)
Yeah, quite some stuff from Owl is also in ALT Linux's distributions, and vice versa (patches to third-party software, enhancements to "Owl things" that we got back from ALT Linux). It's a good example of Open Source working right.
> Weird thing that a 4k shell library hadn't found its way into "mainstream" ditros to date.
Frankly, I didn't think it would ever get into any other distribution - which is why I called it owl-control initially (with the "owl-" prefix, which you can also see on Owl-specific components such as owl-etc, owl-hier, owl-startup, and more). I actually under-estimated Open Source: owl-control (renamed) also got into ALT Linux's distributions, and it's been ported to FreeBSD by Matthias Schmidt (I'm afraid that very few people are aware of this, though...) Maybe we should make owl-control available separately from the Owl source tree as well (like we do for several other components from Owl, including "tcb"), which will make more people aware of it. Maybe "owl-control for export" should get its own name - suggestions are welcome.
Similarly, maybe we should start exporting our ports-from-OpenBSD for reuse on other Linux systems/distros (and maybe even non-Linux) - I'd start with mtree(8), which few non-BSD folks are aware of.
Posted May 12, 2014 9:25 UTC (Mon)
by gvy (guest, #11981)
[Link]
Posted Jan 14, 2011 5:42 UTC (Fri)
by solardiz (guest, #35993)
[Link] (10 responses)
http://openwall.info/wiki/internal/reboot - pre- and post-reboot actions/checklist for remote Linux servers (assuming LILO)
Posted Jan 18, 2011 13:33 UTC (Tue)
by nix (subscriber, #2304)
[Link] (9 responses)
Posted Jan 18, 2011 14:01 UTC (Tue)
by foom (subscriber, #14868)
[Link] (8 responses)
Posted Jan 18, 2011 15:25 UTC (Tue)
by nix (subscriber, #2304)
[Link]
Posted Jan 20, 2011 2:52 UTC (Thu)
by solardiz (guest, #35993)
[Link] (6 responses)
I was not aware of it. Apparently, it did not exist the last time I considered GRUB for Owl. This LWN discussion from 2004 suggests that a patch (just a patch) implementing this feature appeared around that time. Also, now that I asked, it turns out that some others on our team were in fact aware of GRUB being capable of this now.
Yes, we're having a bootloaders discussion for Owl-current now. Short term, we're simply updating from LILO 22.8 (which we have in Owl 3.0; this was the last version released by John Coffman) to 23.1+ with some extra patches in Owl-current (we're working with Joachim Wiedorn on this - he's the new upstream maintainer). Longer term, we're likely to switch to Syslinux (specifically, ISOLINUX and EXTLINUX), which appears to have the desired features as well, except for pre-reboot validation of the configuration, which we might do something about.
Posted Jan 22, 2011 4:06 UTC (Sat)
by ccurtis (guest, #49713)
[Link] (5 responses)
Posted Jan 22, 2011 20:52 UTC (Sat)
by solardiz (guest, #35993)
[Link] (3 responses)
Yes, LILO lacks support for GPT tables. This does not prevent its use on boot devices of exactly 2 TB with MBR partition tables, though (we currently have many Owl installs with LILO on 2 TB devices, although we obviously tend to allocate just a small partition for the boot/root filesystem). My current understanding is that things should continue to work with MBR partitions for boot device size of up to 4 TB, but with individual partitions no larger than 2 TB each. (I have not tested this, though.) With larger RAID arrays (and eventually larger individual disks as well), the current workaround (while staying with LILO) would be to allocate a smaller device for booting from (e.g., put the first two disks into a RAID-1 array, put the rest into a RAID array bigger than 2 TB or 4 TB). Yes, this is a nasty limitation and a reason for us to move to a bootloader that is still being enhanced (or at least has been recently) rather than is merely maintained (like LILO mostly is, with occasional changes to its existing functionality but no new functionality being added).
Posted Jan 22, 2011 22:36 UTC (Sat)
by ccurtis (guest, #49713)
[Link] (2 responses)
But it is nasty as you said. I think you understand the problem, but just in case: if you do not use GPT tables, you cannot physically reach any storage beyond that 2 (or 4?) TB limit. It doesn't matter how you partition the drive (or array) - that extra space can simply not be accessed.
There may be some dirty hacks around now, but at the time it was quite an unpleasant surprise to me. Carving out a smaller virtual device for booting wasn't something I wanted to do at the time; on the upside, I learned to appreciate GRUB.
Posted Jan 23, 2011 2:45 UTC (Sun)
by solardiz (guest, #35993)
[Link] (1 responses)
Just to explain where I got the 4 TB figure from: my current understanding is that it is possible to have an almost-2 TB partition start at almost-2 TB into the device, thus it will end just below 4 TB (perhaps two sectors below 4 TiB, although I did not bother doing the math). That's with an MBR partition table. So on a 4 TB device, the first 2 TB may be partitioned arbitrarily (e.g., a few tiny partitions for the OS install, then an almost-2 TB partition for the machine's actual purpose), and the second 2 TB will have to be in one partition. Yes, this is a somewhat weird special case.
Posted Jan 24, 2011 2:34 UTC (Mon)
by ccurtis (guest, #49713)
[Link]
But even that wouldn't have worked for my application (I don't think...), so GPT it was.
Posted Jan 24, 2011 7:37 UTC (Mon)
by solardiz (guest, #35993)
[Link]
http://www.mail-archive.com/bug-parted@gnu.org/msg00993.html
Initially, I thought that it'd fail while being installed, but maybe not - or not always.
Posted Jan 14, 2011 5:52 UTC (Fri)
by solardiz (guest, #35993)
[Link]
Openwall Linux 3.0: Linux for the security-conscious
The concept of trusted computing base is definitely very old. However, it is not limited to password "storage"/changing/authentication; that is just part of the TCB. Thus, our "tcb" package on Owl is sort of misnamed (it's just part of the TCB, not the entire thing), but it's named after the /etc/tcb directory also seen on some older Unix systems (sometimes called the "trusted" flavors by the vendors). I presume that you refer to the latter.
Openwall Linux 3.0: Linux for the security-conscious
Openwall Linux 3.0: Linux for the security-conscious
...as well as control(8) framework. Weird thing that a 4k shell library hadn't found its way into "mainstream" ditros to date.
> ...as well as control(8) framework.
Openwall Linux 3.0: Linux for the security-conscious
For the record, here's ALT Linux control.git tree and here's the original Openwall implementation. I've also written ALT en-wiki page on this topic since.
Openwall Linux 3.0: Linux for the security-conscious; control(8)
LILO
http://openwall.info/wiki/internal/grub-lilo - how to replace GRUB with LILO on a remote Linux server (with automatic fallback to GRUB on some failures)
http://openwall.info/wiki/internal/netconsole - Linux kernel netconsole setup with LILO and netcat (assumes netconsole support compiled into the kernel image, which it is with Owl's default kernels)
LILO
LILO
LILO
> Is grub-reboot not what you want?
LILO
LILO
LILO
LILO
LILO
LILO
LILO
http://www.justlinux.com/forum/showpost.php?s=a35ea34fa9a...
http://www.rodsbooks.com/gdisk/booting.html
updating a system