Defending mounted filesystems from the root user

* Open any file regardless of permissions.
* Impersonate any user with setuid(2) (or some equivalent).
* Send any signal to any process, and make other adjustments to the process's state (such as renicing it).
* Mount and unmount filesystems.
* And probably a few other highly standardized actions (i.e. *not* Linux-specific things) I've forgotten about.

Problem is, traditionally, you couldn't actually design an OS where root could only do things like the above. You also needed an interface for doing more complicated stuff, and especially for doing things in kernelspace (loading modules, debugging, enabling realtime scheduling, etc.). There are a few ways around this, at least that I can think of:

* We could try to partition off the kernelspace-modifying actions into a separate user, as you suggest, or at least into a separate set of capabilities(7) or the like. The difficulty is that you'd probably have to break up CAP_SYS_ADMIN for this to work, so it would be a lot of code churn. Ultimately, I think the existing capabilities would have to be fundamentally redesigned for this to make sense. It is not enough to split off a permission here and a permission there - we have to think logically about the transitive closure of everything that a process with capability X can ever do, directly or indirectly, and the current design does not even attempt to do that. And then we have to think about all possible combinations of capabilities, or at least all combinations that can plausibly interact with each other to escalate privileges.
* We could say "if you want to modify your kernel, either don't enable secureboot, or reimage your kernel with the appropriate changes pre-configured." The effect would be to disable the kernelspace-modifying actions altogether, and maybe even patch out their codepaths entirely so that they can't be used as ROP gadgets, but only in secureboot-enabled kernels (so that people who "just want a normal kernel" and don't want to put up with this sort of thing can ignore it). The main difficulty here is that, to my understanding, much of the existing "pre-configure your system" tooling currently lives in userspace (e.g. systemd). You'd probably need to provide a rich set of kernelspace configuration options that can be set before the system is first booted, and I'm not sure how feasible that is.
* We could partition off all of the "dangerous" permissions into a series of daemons like systemd and polkit, and administer the system by asking those daemons nicely to do it for us. That would extend secureboot trust to a much wider array of system services, which is probably undesirable (now your systemd has to be secureboot-signed?). OTOH, it's not like Microsoft maintains a strong segregation between the Windows NT kernel and the modern Windows userspace. From their perspective, I imagine the trusted component is a pretty large subset of the operating system, and I doubt they draw the line exactly at ring 0.

I can easily imagine a small and cheap device with a USB host and a USB device connector, which sits between the computer and a USB memory stick, introducing (not so) random corruptions to data read from the memory stick to attack the host.

It's fair to say that in a scenario where you're computing in malicious environments that you must be able to trust some of your hardware — if you can't trust the CPU itself, you're doomed, sure. But with a trusted computing core and IOMMU, you can (in principle) mitigate malicious I/O if you write the drivers defensively.

And it's often only possible to tell if a given on-disk metedata structure is "conformant" after loading *every other* bit of metadata into memory and effectively doing a full consistency/fsck pass. Of course you're still vulnerable to stuff being written to disk behind your back, so the only way to handle that is to always keep the full metadata in memory, and never re-read anything from disk.

To me, filesystems are in many ways an exceptionally nicely contained thing. They largely follow a well defined, narrow API with well defined semantics. Exceptions to it are probably fairly easy to express. Regardless of the filesystem, you can say things like "if I write a file, then read it back without other writes to the same file, I should get the same data (or an error) back".

That is, they seem exceptionally amenable to formal specification and analysis, and from that perspective, how they are designed today seems quite ad hoc. It shouldn't be as hard as with many other systems to actually formally define the operations (up to what gets written to the disk where) and verify that the requires properties hold, as well as do a lot of analysis on performance etc., play with different design ideas without needing to convert and boot kernels, etc. You could treat tolerance to bogus data in the same way, allowing a conscious decision on exactly how you are allowed to fail in different situations.

Now I'm not saying that should necessarily be the same as the code that gets executed (or even generated from it), but parts of it could well be if desired. Verifying the design should give quite a bit of confidence, and effort could be directed at the performance critical parts.

https://github.com/lkl/linux is a fork of the Linux kernel that turns all the interesting routines into a library, with a couple of neat tech demos of what you can do with it - including a FUSE wrapper for the filesystems in the kernel. So any filesystem that's already been implemented once, in the kernel, now has a userspace version.

There's also other ways to do it, such as UML or hardware-assisted virtualization.

Yes, you will lose some performance. I think the triangle of security, performance, and nicheness is a "pick two" situation - if you want both security and performance, you will need to attract enough interest and enthusiasm to pick up the work, possibly defining newer and easier-to-handle on-disk formats as the work happens. Otherwise, you can use an old implementation that made sense in the '90s at full performance with the security of the '90s, or you can use it at the performance of the '90s (which should be enough, honestly!).

(I'd also be very curious to see what the actual performance loss is even for day-to-day filesystems, and whether there are things that can be done to address performance like reviving the zero-copy FUSE patchset. I think I actually do very few things that are ridiculously sensitive to filesystem performance per se: most of the time I'm either working with large single files like giant CSVs or git pack files or game textures, for which the filesystem is essentially a constant factor and it's the raw I/O performance that matters, or reading and writing lots of small files like source code, which can mostly stay in the VFS cache, in theory. Applications that care very much about disk performance, like databases, tend to make a large contiguously-allocated single file anyway - and they subdivide it in userspace.)

In a way, this is the case. Think of the on disk partition table as a configuration "file" that tells the kernel how to configure its internal partition table. The API allows to re-read this configuration after userspace has changed it on disk. And it will fail, it the kernel thinks this is not safe to do. Back in the days, it was entirely impossible to re-read a partition table if any filesystem on the disk was mounted, always requiring a reboot if one modified the partition table on the primary disk. Nowadays it is a bit more permissive.

You just have to mentally differ between the kernel partition table (which is always in RAM) and the partition table on disk. Changing the latter one is no issue at all, as it will only be used by the kernel when explicitly told so or on the next boot. And this design makes a lot of sense. You can do modifications on disk that are only safe to apply after the next boot and then reboot. If the only ways of changing the on disk partition table where by means of an API that directly manipulates the internal partition table such changes would always require to boot another OS (rescue CD etc.).