Lessons from the linux-distros mailing list

The oss-security mailing list is specifically set up for reports and discussion of security flaws in open-source software after their embargo, if any, has expired. But the response to a recent report of the fix for a security flaw in the Linux kernel went in a different direction than usual. The report did not break the two-week embargo period, instead it was "late", which has highlighted some problems in the management of flaws of this nature.

The report from Lin Ma was for a use-after-free vulnerability in the near-field communication (NFC) protocol stack in the kernel. It had been found by fuzzing and was duly reported to the closed security@kernel.org and linux-distros mailing lists on September 1; it was assigned CVE-2021-3760 on the same day. Ma gave a detailed report of the problem to oss-security on October 26—nearly two months later. The flaw itself is difficult to trigger; it may require a compromised NFC device to send the malicious packet sequence.

Alexander Peslyak (or "Solar Designer"), who administers oss-security and linux-distros, replied, noting the large gap in time before the public disclosure, which Ma had apologized for in the report. Peslyak said that there were multiple problems in the handling of the report to linux-distros. "Let's use this opportunity to learn from the mishandling of this issue and avoid that for other issues."

To start with, Ma's original message to linux-distros asked for a 14-day embargo, which is reasonable, but no specific date was attached to that. Since the time period was "OK'ish", it was easy for those looking at the report to accept it, Peslyak said, but the guidelines for making reports do ask for a date. Using a date rather than a number of days has some advantages:

When it's a specific date/time, it's easier for everyone to notice it approaching - not only for people specifically tasked with that. That's just a psychological detail that I guess nevertheless statistically affects the outcomes.

So I think that the distros tasked with reviewing initial notifications should insist on the actual date/time being present in there, or add it on their own in an immediate follow-up.

The linux-distros list has representatives from multiple Linux distributions, as might be guessed; problems affecting more than just Linux should be reported to the separate distros list, which adds in representatives from FreeBSD, NetBSD, and Solaris. As described in the policies and instructions for members section of the linux-distros information page, individual distributions are listed as having a primary role in handling specific management tasks for bug reports; there is also a distribution listed as the backup for most tasks. This is part of the "contributing back" commitment that distributions make when they are added to the list. In this case, neither of the two distributions tasked with looking at initial reports assigned an actual end date for the embargo.

In addition, while the patch that Ma proposed did get into the mainline kernel, linux-distros was not apprised of the status of that effort along the way. Whatever work Ma did with upstream was not reported to the list, nor did the distribution representatives stay on top of its progress. Beyond that, the relevant disclosure channels (e.g. kernel mailing lists) were not monitored for mention of the bug, nor was Ma prompted to make the required oss-security post in a timely manner. Several different distributions dropped the ball on various parts of that. Peslyak described the activity as follows:

The only "contributing back" activity on this issue consisted of 3 postings to linux-distros: prompt CVE ID assignment by Red Hat, a reminder about 14 days having passed by SUSE on September 17 (that is, already 3 days past the embargo period end), and another reminder by (a different engineer from) SUSE on October 25 (this one worked).

Lastly, the oss-security report lacked a reference to the upstream commit that fixed the problem and a date for when it became public. He asked Ma to fill that information in; he also pointed out that this was a relatively low-impact bug but it could provide lessons for the future:

This issue itself is not that important, which is part of why it almost slipped through the cracks, but it's our reminder and opportunity to fix things before anything more important is mishandled.

Ma agreed with Peslyak's suggestions, but was unaware of the need to keep linux-distros in the loop on the progress of the fix. Ma pointed to the commit fixing the problem and thanked Peslyak for his help. As Peslyak said, though, the fix had been posted to the netdev mailing list on October 7, effectively breaking the embargo at that point; it was supposed to have ended far earlier, so no harm was done. He also said that someone could have made the required post to oss-security in Ma's stead once the embargo period was over, noting that there were a number of points in time where that could have happened.

So far, at least, only one of the distributions that are assigned to the dropped tasks that Peslyak highlighted has responded. Anthony Liguori at Amazon acknowledged that the company had missed staying on top of the progress of the bug and fix as the backup, but indicated that it wanted to keep working on the "contributing back" tasks going forward. If the linux-distros mailing list is going to be able to continue functioning, it clearly requires a cooperative effort among those who are participating. That seems to have broken down in this case, so Peslyak is trying to nip the problem in the bud.

Another part of the task is slipping through the cracks, Peslyak said in his first message: statistics gathering. There is no one currently assigned to do that and he would like to see a volunteer or two to step up. The data he is looking for is shown on this page and spelled out in the guidelines:

Keep track of per-report and per-issue handling and disclosure timelines (at least times of notification of the private list and of actual public disclosure), at regular intervals produce and share statistics (most notably, the average embargo duration) as well as the raw data (except on issues that are still under embargo) by posting to oss-security.

There are good reasons to collect that kind of information in order to monitor the health of the linux-distros community and processes, but it also may have helped directly with this bug:

[...] an important desirable side-effect of keeping the statistics up-to-date is that this would catch issues that were not reported to oss-security in time or at all. For example, if someone were updating statistics for September on October 15 (by which point nothing from September is supposed to still be embargoed), they'd catch this issue 10 days earlier.

The mailing list can provide some benefits for its members, early disclosure of important bugs, coordinated releases so that some distributions do not get left behind, and so on. But that only works if the process functions well, which requires a level of commitment from each participating organization.

Linux-distros came about after its predecessor, the vendor-sec list, was compromised and disbanded in 2011. There were a number of problems with vendor-sec, not least the size of the closed list (80-100 people), so linux-distros set out to tighten things up and to codify what is expected of participants. Peslyak would clearly like to see things get back on track, so one hopes that the linux-distros community heeds his little wakeup call.