Linux, meet the Registry

Posted Mar 12, 2009 23:17 UTC (Thu) by rahulsundaram (subscriber, #21946)
In reply to: Linux, meet the Registry by eru
Parent article: ext4 and data loss

Ted mentioned in the FOSDEM talk this year (video available online), that many developers ( GNOME and KDE included) tends to use hundreds of small files with very small amounts of data and that isn't really a bright idea compared to a centralized registry. Something to consider.

Linux, meet the Registry

Posted Mar 13, 2009 0:02 UTC (Fri) by bojan (subscriber, #14302) [Link] (7 responses)

When this central registry gets stuffed, the whole lot is stuffed (see: Windows). Small files that are merged together into a tree are not _that_ stupid. And, you can remove directories that contain them in order to remove parts of the tree - simple.

Maybe the real solution is to not write them out unless absolutely necessary.

Linux, meet the Registry

Posted Mar 13, 2009 2:00 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link] (6 responses)

Centralized doesn't mean binary. Even assuming it is, Firefox is using sqlite databases instead of inventing their own binary format. Binary is not necessarily evil as people seem to think. I don't see how your solution would work. When you don't write them out, you stand a higher chance of losing that data which is exactly the problem

Linux, meet the Registry

Posted Mar 13, 2009 2:46 UTC (Fri) by quotemstr (subscriber, #45331) [Link]

The filesystem is already a hierarchical database. Putting a hierarchical database on top of the filesystem is redundant. Fix bugs instead of working around them!

Linux, meet the Registry

Posted Mar 13, 2009 3:37 UTC (Fri) by bojan (subscriber, #14302) [Link] (4 responses)

> Centralized doesn't mean binary.

Say you want to fix a corrupt gconf XML file that is 20 lines long. The easy fix is to delete it and recreate the settings using preferences or gconf editor.

Say you want to fix a corrupt gconf XML file that is 200,000 lines long. Well, good luck not mucking it up in vi so it still parses.

> Even assuming it is, Firefox is using sqlite databases instead of inventing their own binary format.

Which, as we've seen, comes with its own set of problems on ext3. And, once again, if the DB file gets screwed, you are completely out of luck - _all_ your settings are gone. Eggs in one basket and all that.

> Binary is not necessarily evil as people seem to think.

Yeah, tell that to people with corrupt Windows registry.

> I don't see how your solution would work. When you don't write them out, you stand a higher chance of losing that data which is exactly the problem.

Nobody said anything about not writing them out. The problem is that it appears they are being written out even when _not_ required and in large numbers.

When users make changes to configuration, these are localised changes. Users don't normally change 200 settings at once. So, this will touch a very limited number of files that need to be persisted to disk using fsync. The problem is the currently hundreds of files are being persisted to disk often.

Linux, meet the Registry

Posted Mar 13, 2009 5:32 UTC (Fri) by eru (subscriber, #2753) [Link] (3 responses)

> Binary is not necessarily evil as people seem to think.

Yeah, tell that to people with corrupt Windows registry.

The binary/text distinction is rather illusory. Text is simply a binary file that uses a subset of byte values to represent data, and certain values as delimiters. What really matters is how a file format is structured. A binary file can be organized so that recovering data from it is possible (what does fsck(8) really do? Fix corruption in a complex binary file, with the constraint that the operation must be done in-place).

Linux, meet the Registry

Posted Mar 13, 2009 5:56 UTC (Fri) by bojan (subscriber, #14302) [Link]

> The binary/text distinction is rather illusory.

Yeah. I edit SQLite files in vi all the time ;-)

Illusory?

Posted Mar 13, 2009 13:14 UTC (Fri) by man_ls (guest, #15091) [Link]

The binary/text distinction is not illusory; it is a cognitive issue. Limiting file contents to printable characters (not just byte values since you can use multi-byte characters) makes people be able to edit them. Text files do not usually contain just random characters; they contain readable words that can be understood and documented rather easily.

The machine doesn't care, true, but to people there is a big difference between a sequence of random byte values and a sequence of written words. Just as, to me personally, there is a big difference between a text in Spanish and a set of cyrillic Russian words.

Linux, meet the Registry

Posted Mar 19, 2009 9:31 UTC (Thu) by renox (guest, #23785) [Link]

[[ The binary/text distinction is rather illusory. ]]

For computers yes, for human this is very different, that's the point!

If you have a corrupted binary, it's very, very difficult for an human to fix it (unless there's a tool which fix it "auto-magically"), whereas for a text there is still the possibility for the human to fix it.

A FS is a database, fsck is the tool to fix it (up to a point), if you add other databases in a FS this add the possibility of additional errors fixable only by the tools, with structured text files (JSON is nice: easy to read and to parse) you have the best of both worlds.

Linux, meet the Registry

Posted Sep 9, 2009 22:02 UTC (Wed) by BrucePerens (guest, #2510) [Link]

The Namesys paradigm of file-per-atom is a good idea, it's just that filesystems aren't up to the task. Too bad its creator went on to worse ideas.