LWN: Comments on "Accessing zoned block devices with zonefs"

Accessing zoned block devices with zonefs

lzap — Thu, 16 Jan 2020 15:13:40 +0000

Is there a good resource on how those SMR-managed drives actually work internally these days?

Accessing zoned block devices with zonefs

anton — Wed, 31 Jul 2019 17:42:10 +0000

Yes, classic log-structures file systems work with segments (i.e., zones) that are written sequentially. To reclaim space, the live data from some segments is copied to a new segment. COW file systems (e.g., BTRFS) generally use a similar organization as log-structured file systems, except for free-space management: COW file systems don't garbage-collect, but keep track of free blocks, and are therfore not so great on zoned devices.

The article refers to SMR HDDs. What I have seen in the HDD market in the last few years was that SMR drives were only a little larger than conventional drives, and looking for the largest drives (16TB) today, all offers are for conventional drives. It seems that SMR is on the way out. However, NAND flash only allows erasing big blocks, and despite announcements for years that we are going to get technologies without this restriction, NAND flash seems to be still going strong. However, this property of NAND flash is usually hidden behind a flash translation layer that makes it look like a normal block device.

Accessing zoned block devices with zonefs

flussence — Sat, 27 Jul 2019 05:33:18 +0000

Wouldn't a log-structured filesystem be a good fit for these? Mostly append-only, can use the conventional zones for bookkeeping and garbage collection workspace.

Accessing zoned block devices with zonefs

gdt — Fri, 26 Jul 2019 02:40:33 +0000

Could I suggest that filesystem extended attributes might be a good idea? Often the people who want good sequential performance from drives -- and are willing to work with odd filesystems to get it -- also work in environments where there are increasing rules around data security. As one example, government science agencies.

Accessing zoned block devices with zonefs

Sesse — Thu, 25 Jul 2019 21:04:31 +0000

SSDs are definitely zoned, but they're not exposing any of it to userspace (yet).

Usually SLC is for speed and not endurance, by the way.

Accessing zoned block devices with zonefs

Baughn — Thu, 25 Jul 2019 12:22:21 +0000

The chief problem for a CoW filesystem would be garbage collection. If you look at ZFS for example, it depends on being able to reuse blocks previously belonging to files that have been modified, and those blocks might be in the middle of a sequential zone.

We might need a background process running something like a copying GC, which can't be retrofitted without significant work.

Accessing zoned block devices with zonefs

ptman — Thu, 25 Jul 2019 10:52:41 +0000

IIRC some SSDs are already partially zoned. MLC drives can have some area used as SLC (giving greater write endurance) for FTL garbage collection and bookkeeping.

Accessing zoned block devices with zonefs

epa — Wed, 24 Jul 2019 22:36:15 +0000

It seems as though an ordinary, non-COW file system could work well enough with one of these drives provided the application itself is append-only. So you provide some hint to indicate a particular file will only be appended to (in normal use) and the file system sticks it on one or more of the append-only zones. That big append-only file could be a log file for a database or whatever. The amount of smarts needed in the file system itself is pretty small.

Accessing zoned block devices with zonefs

nivedita76 — Wed, 24 Jul 2019 14:57:34 +0000

The regular file systems don’t support efficiently using such a device (yet). This seems like a transitional thing until they do. It would seem like it shouldn’t be too heavy a lift for CoW filesystems to understand the zones while allocating new blocks, but it may never be possible for non-CoW filesystems to be efficient.

Accessing zoned block devices with zonefs

Freeaqingme — Wed, 24 Jul 2019 14:40:11 +0000

If ZoneFS were to become a thing, it's only a matter of time before features are being added to it until feature parity with XFS/ext4 is reached. If it's feasible to implement in Btrfs and XFS, I'd say to just implement it there and don't bother implementing a separate FS.

> The idea is that zonefs provides just enough of a filesystem for those developers [...]

In what way would those developers be hindered by using a regular FS?

Accessing zoned block devices with zonefs

willy — Wed, 24 Jul 2019 13:29:01 +0000

Your understanding is correct; epithumia is mistaken.

Accessing zoned block devices with zonefs

willy — Wed, 24 Jul 2019 13:28:13 +0000

Bad news, SSDs also want zones, although for very different reasons from shingled drives.

One of the larger pieces of cost for an SSD is the RAM used to run the FTL. If you can shrink the FTL by disallowing random writes, you save a lot of money. So we're going to see zoned SSDs too.

Accessing zoned block devices with zonefs

Sesse — Wed, 24 Jul 2019 10:36:19 +0000

The use case is really “large drives”. :-) SMR drives are built this way; you can't write to the middle of a zone, because the tracks sort-of overlap and must be written from the inside out (or is it outside in, I have no idea). You can either have the drive simulate a normal drive (rather slowly), or you can expose the functionality to userspace, where it fits really nicely into some kinds of storage patterns already.

In particular, anything BigTable-like (generically called LSM) will never overwrite existing data on write; they'll just write a correction record (“whatever the address for user X used to be, now it's Y”, or “delete user X”), where the last value wins. When needed, they'll have garbage collection, where they read an entire chunk (or zone, in this case), prune out all the old values, and write a new tree. It fits perfectly well with the append-only nature of these zones, so if you know how the zones are structured, you take nearly no hit from using an SMR disk instead of a conventional disk.

This does really feel like a transient thing, though. Host-managed SMR disks are not really all that much used yet, and as they ramp up, it's into a future where flash keeps getting cheaper and cheaper, and eventually looks like it will be overtaking rotating media in terms of cost per byte.

Accessing zoned block devices with zonefs

james — Wed, 24 Jul 2019 10:24:58 +0000

Another use case is video surveillance, where you're continually recording video and want to keep as much as possible before you need to over-write.

Commercial devices based on a zoned drive could have an embedded OS in the conventional zones to keep the bill of materials down.

Accessing zoned block devices with zonefs

smurf — Wed, 24 Jul 2019 04:48:45 +0000

Do you really need to write a whole zone at once? I thought these drives have per-zone append (until the zone is full of course).

Accessing zoned block devices with zonefs

epithumia — Wed, 24 Jul 2019 01:04:18 +0000

The use case for these things is basically write-rarely data. The devices are larger than the available non-shingled drives but can't really be used in the same way (or at least not efficiently).

And yes, you must write an entire zone at once. But the devices generally have a region which accepts random writes (or are paired with another device which serves that function), so you can potentially accumulate a zone's worth of data over time and then move that to a zone all at once. There is plenty of room for smart filesystems here.

Accessing zoned block devices with zonefs

JohnVonNeumann — Tue, 23 Jul 2019 23:44:26 +0000

For the FS/Storage noobs in here, what would be the ideal use case for zoned block devices or zonefs?

And how does atomicity work with this? The article states that there is no random access for writes to the zones, does this mean that an entire zone (like a block) has to be allocated for a single write, and further writes to the zone would require a zeroing of the zone, before write? Or can a zone be used for multiple operations, provided that the previous blocks aren't touched?