Ext4 snapshot concerns

By Jonathan Corbet
June 8, 2011

The next3 filesystem patch, which added snapshots to the ext3 filesystem, appeared just over one year ago; LWN's discussion of the patch at the time concluded that it needed to move forward to ext4 before it could possibly be merged. That change has been made, and recent ext4 snapshot patches are starting to look close to being ready for merging into the mainline. That has inspired the airing of new concerns which may slow the process somewhat.

One complaint came from Josef Bacik:

I probably should have brought this up before, but why put all this effort into shoehorning in such a big an invasive feature to ext4 when btrfs does this all already? Why not put your efforts into helping btrfs become stable and ready and then use that, instead of having to come up with a bunch of hacks to get around the myriad of weird feature combinations you can get with ext4?

Snapshot developer Amir Goldstein's response is that his employer (CTERA Networks) wanted the feature in ext4. The feature is shipping in products now, and btrfs is still not seen as stable enough to use in that environment.

There are general concerns about merging another big feature into a filesystem which is supposed to be stable and ready for production use. Nobody wants to see the addition of serious bugs to ext4 at this time. Beyond that, the snapshot feature does not currently work with all variants of the ext4 on-disk format. There are a number of ext4 features which do not currently play well together, leading Eric Sandeen to worry about where the filesystem is going:

If ext4 matches the lifespan of ext3, in 10 years I fear that it will look more like a collection of various individuals' pet projects, rather than any kind of well-designed, cohesive project. How long can we really keep adding features which are semi- or wholly- incompatible with other features?

Consider this a cry in the wilderness for less rushed feature introduction, and a more holistic approach to ext4 design...

Ext4 maintainer Ted Ts'o has responded with a rare (for the kernel community) admission that technical concerns are not the sole driver of feature-merging decisions:

It's something I do worry about; and I do share your concern. At the same time, the reality is that we are a little like the Old Dutch Masters, who had take into account the preference of their patrons (i.e., in our case, those who pay our paychecks :-).

In this case, he thinks that there are a lot of people who are interested in the snapshot feature. He worried that companies like CTERA could move away from ext4 if it can't be made to meet their needs. So his plan is to merge snapshots once (1) the patches are good enough and (2) it looks like there is a plan to address the remaining issues.

Index entries for this article
Kernel	Filesystems/ext4

Ext4 snapshot concerns

Posted Jun 9, 2011 6:56 UTC (Thu) by MisterIO (guest, #36192) [Link] (1 responses)

The real problem with btrfs is not its lack of stability(though that is still somewhat true), but the fact that it still doesn't have a working fsck! It was supposed to be almost ready by february, but, since then, no updates on its status have appeared. It would be useful to have more news about the status of that project or (maybe even better) to have an actually open development of it(currently code seems to be committed only when it's already considered ready for use).

Ext4 snapshot concerns

Posted Jun 9, 2011 7:09 UTC (Thu) by sce (subscriber, #65433) [Link]

Hear, hear. I wanted to try out btrfs on an external drive I bought some time ago, but I just couldn't make myself potentially end up in a situation where I would have a partially corrupt filesystem with no way to fix it.

Ext4 snapshot concerns

Posted Jun 10, 2011 9:05 UTC (Fri) by ebirdie (guest, #512) [Link] (1 responses)

Interesting problem to solve.

From functionality point of view I find it a bit odd that the number of technologies for file system snapshoting grows. Maybe it is just my limits in thinking: there should always be only one universal way to do a thing.

If the feature gets merged, there will be LVM, btrfs and ext4 for snapshotting, what I can tell. In general it is great that there is more than one way to do snapshotting, but is it maintainable as in kernel development and understandable as there will be several ways for a user to do snapshotting. Will LVM snapshotting get bitrot as it is old and boring, there will be btrfs and one can already use ext4 snapshots, while waiting. Is this bad technology fragmentation?

The user perspective brought me to Ted Ts'o response as it was written so that it isn't very appealing argument at first pass. Then I came to think about the user perspective as a company, whether developing technology for own service or for an end user product, can be seen as a user like it has been touted many times that "a merge needs real users". A company's use/need should not make a big difference from a casual desktop user, thus isn't necessary to argument that a feature is merged by paycheck liability, what, as argument, will bring a house of cards.

Secondly the recent talks about forking and high fences in merging:

Android, forking, and control [LWN.net] https://lwn.net/Articles/446297/
Forking the ARM kernel? [LWN.net] https://lwn.net/Articles/445417/

get more flesh and bones with this feature.

Ext4 snapshot concerns

Posted Jun 16, 2011 8:43 UTC (Thu) by mangoo (guest, #32602) [Link]

LVM snapshots and filesystem (ext4, btrfs) snapshots are two different worlds.

LVM snapshots and block devce snapshots and need extra space outside of the filesystem you're snapshotting. Try doing that on your laptop - I bet it's not very common to leave ~50% of unallocated space on a device which is not a storage server (or similar).

Filesystem snapshots use space within the filesystem itself, so one does not have to worry about any extra filesystem layout planning on a given server/PC/device.