Case-insensitive ext4

Posted Mar 27, 2019 21:17 UTC (Wed) by rweikusat2 (subscriber, #117920)
In reply to: Case-insensitive ext4 by Cyberax
Parent article: Case-insensitive ext4

There is no such thing as "a fileserver use case". Samba exists and has existed (and been used) for a while, hence, there are obviously "file server use cases" where the existing mechanisms perform well enough.

Case-insensitive ext4

Posted Mar 27, 2019 21:26 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

Samba exists, but it does case insensitivity in a very expensive way. There are other use-cases as well, like mounting FAT filesystems.

I've seen this firsthand - I'm using a Linux server for TimeMachine backups for Mac OS X. TimeMachine is braindead - it creates hundreds of thousands files in the same directory. With the default settings Samba slowed down to a crawl.

Fortunately, TimeMachine doesn't care about file name cases. So by following steps from here: https://wiki.samba.org/index.php/Performance_Tuning I was able to speed up backups by something like 10x. This is not insignificant and it would be nice for Linux to handle similar use-cases natively.

Case-insensitive ext4

Posted Mar 28, 2019 0:50 UTC (Thu) by rahulsundaram (subscriber, #21946) [Link] (6 responses)

> there are obviously "file server use cases" where the existing mechanisms perform well enough.

Have you talked to Samba developers and asked them if they are happy with the current performance or would like to see better support from the kernel? If you haven't I would encourage you to do that or talk to enterprises supporting Samba or even large customers. I think you will find that perspectives useful to add to your opinions.

Case-insensitive ext4

Posted Mar 28, 2019 16:10 UTC (Thu) by rweikusat2 (subscriber, #117920) [Link] (5 responses)

Someone claimed that Samba opening a file residing on a case-sensitive filesystem would require a pre-open, linear directory traversal. As I pointed out, this isn't true, at least not on Linux: It would be possible to use an incrementally maintained, userspace translation cache instead, however, unless I again have to use Samba for something in a resource-constrained environment, I'm not going to implement that and in the unlikely case that this would happen, I'd certainly not go through the rather pointless hassle of trying to contribute a non-trivial change to an open sausage project, as I have neither the time to do this nor the social skills and pedigree to do so successfully.

Case-insensitive ext4

Posted Mar 28, 2019 18:36 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link] (3 responses)

> As I pointed out, this isn't true, at least not on Linux: It would be possible to use an incrementally maintained
Nope. There is no way to maintain this cache with any sort of consistency guarantees. Linux filesystem change notifications are not up to it.

Case-insensitive ext4

Posted Mar 29, 2019 3:03 UTC (Fri) by pabs (subscriber, #43278) [Link] (2 responses)

What are they missing now that recent Linux versions offer rename notifications and other directory change notifications?

Case-insensitive ext4

Posted Mar 29, 2019 4:21 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link] (1 responses)

inotify is not recursive. It's also best-effort and its notifications are asynchronous.

fanotify is better, but it also can drop events from time to time under high load.

Case-insensitive ext4

Posted Oct 4, 2023 18:51 UTC (Wed) by calumapplepie (guest, #143655) [Link]

You get an event when the queue overflows. If you clear the cache on receiving such an event, you can provide consistency guarantees, at the cost of bad performance while the cache is rebuilt. Since the queue is pretty big, overflows shouldn't happen too much.

Case-insensitive ext4

Posted Mar 29, 2019 22:55 UTC (Fri) by jra (subscriber, #55261) [Link]

We already have an incrementally maintained, userspace translation cache in Samba. It catches the simple cases where we've seen a filename before - we cache it.

Unfortunately it isn't enough. Cache misses are the problem. If the SMB client sends a filename "foo" and it isn't in the directory, we don't know if it doesn't exist, or exists under another case (e.g. as "Foo"). In that case we need to scan the directory. This gets really expensive, really quickly.

We don't negatively cache as we're often used to export filesystems that local processes are also modifying.

I've been wanting a case-insensitive filesystem lookup option in Linux for a long time (I think ZFS and XFS already have it, however flawed).