Linus Torvalds on the kernel development community

The Linux Kernel Maintainers Summit is all about the development process, so it is natural to spend some time on how that process is working at the top of the maintainer hierarchy. The "is Linus happy?" session during the 2019 summit revealed that things are working fairly well at that level, but that, as always, there are a few things that could be improved.

Torvalds initially turned the question around, saying that it should be about whether everybody else is happy about his work. But then he turned it back to his current pet peeve: developers who do not put changes into linux-next before pushing them into the mainline. There was one specific tree that he was unhappy about in the 5.3 merge window; others have been problematic in the past but have improved somewhat. But, it seems, there is always somebody. In general, about 10% of the patches that show up during the merge window did not first show up in linux-next.

Dan Williams asked whether there should be a rule requiring any changes pushed upstream to be in linux-next for at least 24 hours. Torvalds responded that he doesn't even check for presence in linux-next early in the merge window; he is happy enough to get an early pull request that nothing more is required. As the merge window approaches its end, though, he does start checking, and absence from linux-next (earlier in the merge window) can result in pull requests not being acted upon.

Sasha Levin asked about whether the same sort of checking happens after -rc1 comes out; the answer was "generally not". Code entering the mainline after the merge window is supposed to be limited to important fixes, and linux-next is less useful for those. As far as Torvalds is concerned, fixes that do not appear in linux-next are not an issue at all. Levin protested that fixes are often broken; putting them in linux-next at least gives continuous-integration systems a chance to find the problems. Linux-next maintainer Stephen Rothwell noted that he keeps "fixes" trees separately now, and they can be tested independently and more frequently.

Torvalds pointed out, though, that developers tend not to realize that the creation of linux-next is a manual process; there is no automatic testing that happens just as a result of pushing something in that direction. It can take a few days before patches in linux-next are tested; instead, patches that go into the mainline are tested immediately. Beyond that, the rule is that mainline patches should not be picked for a stable release until one week after they have been merged. What actually happens is that they have to be there for at least one -rc release, but could be selected for stable before a full week has passed.

Ted Ts'o said that he used to be able to push changes into his ext4 tree and get results from the 0day tester within hours. That hasn't been the case for some time, though; he would like to have a URL where he can see if a particular tree has been tested or not. Thomas Gleixner said that it is possible to get a success email from the 0day system now, but those are not particularly reliable either.

Overall, Torvalds summarized, the process is working smoothly. There are still too many bugs, though, "no question about that". The rate of change has not increased much for a couple of years, which he described as a good thing. It's because there really isn't more work to do, rather than being a sign of a bottleneck in the system somewhere. It was quickly pointed out that the rate of change is a local phenomenon; the BPF subsystem is growing quickly, while the Arm architecture code is changing more slowly than it was a few years ago.

Greg Kroah-Hartman said that, of all the patches submitted to the mailing list, only 2% are not receiving a response, which is an improvement over past results. Christoph Hellwig jumped in to say that more patches should be ignored. Torvalds is too happy now, Hellwig said; he needs to be angrier and say "no" more often. Peter Zijlstra complained that he misses the "old Linus".

Dave Airlie asked about the review backlog in general, and whether there was any overall picture of how far behind reviewers are? Kroah-Hartman replied that some subsystems are known to be bad; nobody seems to have a full picture of the state of review across the tree, though.

Torvalds said that he varies quite a bit in how closely he looks at reviews for incoming patches. It depends a lot on the subsystem maintainer; some trees he feels he can just pull without needing to look inside at all, while others require an examination of every patch. He noted that he dislikes getting trees where patches contain only reviews from developers working for the same company as the author; he suggested that, while internal review is an important part of the process, those reviewers should not put Reviewed-by tags in the patches.

There was some discussion about the value of internal reviews, which are seen as being helpful at best and harmless at worst. Gleixner complained about getting low-quality patches with five Reviewed-by tags, though. Torvalds said that patches with only internal reviews show up mostly in the driver subsystems. Often, those patches are so hardware-specific that nobody else will be able to look at them anyway. In some cases — he mentioned the Intel graphics drivers — the subsystem as a whole is solid and internal reviewers deserve a lot of the credit.

Olof Johansson said that the best way for reviewers to build trust is to do their reviews publicly, on the mailing lists, and be seen to point out real problems. As the discussion closed, it was suggested that subsystem maintainers should be informed discreetly if company management is putting pressure on internal reviewers to let substandard code through so that those patches can receive more scrutiny once they hit the lists.

[Your editor thanks the Linux Foundation, LWN's travel sponsor, for supporting travel to this event.]

Index entries for this article
Conference	Kernel Maintainers Summit/2019

---

to post comments