2038: only 21 years away
That work, he said, is proceeding on three separate fronts, the first of which is the kernel itself. He has been working for the last five years to try to prepare the kernel for 2038. Much of that work involves converting 32-bit timestamps to 64-bit values, even on 32-bit systems. Some 32-bit timestamps also show up in the user-space API, which complicates the issue considerably. There is a plan for the enhancement of the user-space API with 2038-clean versions of the problematic system calls, but it has not yet gotten upstream. One recent exception is the statx() system call, which was merged for 4.11; statx() will serve as the year-2038-capable version of the stat() family of calls. There are quite a few other system calls still needing 2038-clean replacements, though.
There is one other person, Deepa Dinamani, working on the kernel side of
things; she started as an Outreachy intern and has
continued to work on the
problem after the internship ended. Dinamani has a set of virtual
filesystem layer patches in hand, which address one of the hardest
problems, and she has plans for some other system calls as well. One of
the trickier ones might be setsockopt(), which isn't easily
fixed or emulated at the glibc level. There are device-mapper and input
subsystem patches in an advanced state. Bergmann had a patch for the
video4linux subsystem, but that was rejected and needs a new approach; a
similar situation exists for the audio subsystem. Other areas needing work
in the kernel are key management and realtime clocks.
For some system calls, there will be no replacement, since the best approach appears to be emulation in the C libraries — the second focus for the year-2038 effort. There has been a lot of work done in the glibc community in particular, he said; the plan is to be fully backward compatible at that level. That means that it will be possible to build a program with either 32-bit or 64-bit timestamps, and to use the larger timestamps even on older kernels. In other words, the glibc developers are trying to make things work everywhere, with a minimum of disruption. (See this draft design document for lots of details on the glibc plan.)
The third focus is on distribution builds, which can only really proceed once the first two pieces are in place. Most distributors, Bergmann said, are unlikely to even bother with 32-bit systems in 2038, so they won't have much to worry about. One big exception may be Debian, which seems interested in maintaining support, even though it looks like it will be a painful task. It may require a full rebuild at some point, which isn't much fun for anybody involved, but it is at least a process that is known to work. Compatibility is key in such a system; there is code being deployed now that may not be 2038-clean, but people want it to keep working if at all possible.
One big area of concern is automobiles. A lot of devices, such as handsets, will have long since failed for any of a number of other reasons by then, so there is little point in ensuring that they can survive 2038. But people keep cars going for a long time. There may still be cameras in use by then, and there is highly likely to be a lot of deeply embedded systems such as building infrastructure. Some of these systems are going to fail in 2038. That is why it is important to get the problem fixed as soon as possible.
There are some things that are going to be difficult to fix, though, even when the kernel, C libraries, and distributions are mostly taken care of; many of these are the result of the use of 32-bit time_t values in file formats. Thus, for example, cpio will fail, which is problematic because it is used by the RPM package format. The NFSv3, ext3, and XFS filesystems all have problems resulting from their use of 32-bit timestamps. The first two are likely to have gone out of use long before the problem hits, and plans for the repair of XFS are in the works. Then, of course, there is a whole list of applications that nobody has yet noticed are broken, and lots of in-house applications that cannot be fixed by the community.
When asked which tools he is using for this work, Bergmann replied that his core technique involves building kernels with the 32-bit time types removed completely. That will quickly point out the places that still need to be fixed. Beyond that, he said, it's mostly a manual process. It was suggested that sparse or GCC plugins could maybe help with this task.
As things wound down, John Stultz asked how much the work in the BSD camp, which has (in some variants) already solved its year-2038 problems, might help with Linux. The answer would appear to be "not much". BSD-based distributions have the advantage of being able to rebuild everything from scratch, so they do not need to maintain user-space ABI compatibility in the same way. There is some value in in the work that has been done to fix applications with year-2038 problems, but it's not clear how much that will help the Linux community.
[Thanks to Linaro and the Linux Foundation for funding your editor's travel
to Connect.]
| Index entries for this article | |
|---|---|
| Kernel | Year 2038 problem |
| Conference | Linaro Connect/2017 |
Posted Mar 15, 2017 18:14 UTC (Wed)
by amacater (subscriber, #790)
[Link] (8 responses)
Is there a scope for making a cpio64 or similar, declaring a flag day and switching to it five years from now?
Posted Mar 15, 2017 18:20 UTC (Wed)
by andrewsh (subscriber, #71043)
[Link]
Posted Mar 15, 2017 19:34 UTC (Wed)
by dmoulding (subscriber, #95171)
[Link] (4 responses)
Posted Mar 15, 2017 19:50 UTC (Wed)
by arnd (subscriber, #8866)
[Link] (3 responses)
We can either change the kernel to completely ignore the times in initramfs and pretend every file has the same timestamp, to ensure that nothing breaks in 2038 that isn't already broken, or we can apply some windowing and interpret negative timestamps as post-2038 times.
Posted Mar 15, 2017 22:23 UTC (Wed)
by SEJeff (guest, #51588)
[Link] (2 responses)
Posted Mar 17, 2017 4:26 UTC (Fri)
by jwarnica (subscriber, #27492)
[Link] (1 responses)
This will work for most of reality.
Posted Mar 23, 2017 15:03 UTC (Thu)
by Wol (subscriber, #4433)
[Link]
I think that's a very dangerous assumption, though not likely to be a problem for the OS. It'll break user-space badly, though.
Cheers,
Posted Mar 16, 2017 0:07 UTC (Thu)
by walters (subscriber, #7396)
[Link]
And for rpm-ostree, we do obviously import RPMs - but since we're committing them into an OSTree repo, the timestamp data is hence discarded. This currently matters the most in the package layering case - we today rerun all triggers/posts on upgrade to avoid hysterisis, but it also helps avoids depending on timestamps for %post scripts. The cost is pretty low.
Basically, I think no one should really care when the individual files in a package were built. OSTree *does* have a per-commit timestamp to help prevent accidental or MITM-induced downgrades, but that's 64 bit.
Hmm, I just noticed git's commit objects actually use ASCII seconds-since-epoch; in the git source as of `v2.12.0-306-g4a9b9b32d47212e4ea43cd1fa3c11afef6d4c491`:
```
A little surprising this isn't already a uint64_t. But at least the ASCII encoding means it's fixable. Ah, and in libgit2 it's already `typedef __time64_t git_time_t;`.
Posted Mar 17, 2017 16:14 UTC (Fri)
by fw (subscriber, #26023)
[Link]
Posted Mar 16, 2017 4:22 UTC (Thu)
by lyallp (guest, #74741)
[Link]
Everything is built from source with the exception of proprietary software.
Ok, my XFS filesystems in their current form will fail but since I do daily updates, sometime in the next 21 years I would expect a migration to occur, similar to IOS 10.3 changing from HPFS+ to APFS.
Posted Mar 16, 2017 7:25 UTC (Thu)
by eru (subscriber, #2753)
[Link] (5 responses)
I suspect NFSv3 will still be around. The problem is NFSv4 is so different it is hard to migrate to smoothly.
Posted Mar 16, 2017 8:29 UTC (Thu)
by dd9jn (✭ supporter ✭, #4459)
[Link] (2 responses)
OpenPGP also uses 32 bit timestamps and we won't change that any time soon. There is plenty of time to change the Epoch before 2106 sets and end to the current Epoch.
Posted Mar 16, 2017 13:38 UTC (Thu)
by smcv (subscriber, #53363)
[Link] (1 responses)
So there is one second during 2038 that some APIs cannot represent, but the second before it and the second after it are fine? That seems an odd design...
Posted Mar 16, 2017 13:41 UTC (Thu)
by smcv (subscriber, #53363)
[Link]
Posted Mar 17, 2017 0:16 UTC (Fri)
by flussence (guest, #85566)
[Link] (1 responses)
It won't surprise me if there's a separate NFSv4 flag day between 2038 and now... I wonder how many NASes are being sold with only v3 support today?
Posted Mar 23, 2017 16:25 UTC (Thu)
by arnd (subscriber, #8866)
[Link]
The problem is nfsv3 implementations that treat the timestamp as signed, and that includes today's 32-bit linux kernels, and probably half of the other implementations (file system implementors elsewhere have been relatively random in their interpretation of time stamps being either signed or unsigned).
Posted Mar 16, 2017 8:23 UTC (Thu)
by mjthayer (guest, #39183)
[Link] (9 responses)
Posted Mar 16, 2017 12:07 UTC (Thu)
by epa (subscriber, #39769)
[Link] (2 responses)
Posted Mar 16, 2017 13:11 UTC (Thu)
by lkundrak (subscriber, #43452)
[Link] (1 responses)
Posted Mar 18, 2017 2:32 UTC (Sat)
by CChittleborough (subscriber, #60775)
[Link]
Posted Mar 16, 2017 12:43 UTC (Thu)
by sorokin (guest, #88478)
[Link] (5 responses)
If they were opaque structs with overloaded C++ operators <, >, = that would be more feasible. But if they were opaque structs we would have no problem in the first place.
Posted Mar 16, 2017 13:46 UTC (Thu)
by mjthayer (guest, #39183)
[Link] (4 responses)
Ahem, yes. And time_t is signed, which means that it cannot just be let to overflow and continue, which was what I had in mind.
Posted Mar 16, 2017 15:00 UTC (Thu)
by dd9jn (✭ supporter ✭, #4459)
[Link] (3 responses)
clock_t and time_t
thus it is undefined whether it is signed or unsigned. Traditionally the return value of time() was a long and thus many implementations used to typedef time_t as long. But there is no need for this and it would work just fine with an unsigned integer, if there would not be code which uses
if (time(NULL) < 0) oops();
or uses (before - after) terms. Portable programs won't have a problem with an unsigned time_t. But nevertheless making time_t unsigned is an ABI change and thus it is easier to do it like BSD and directly switch to a 64 bit type.
FWIW, around 2003 I met with the glibc maintainer to ask him to start migration to a 64 bit time_t. Back then I was in a need for post 2038 timestamps due to some root certificates having an insane long term. Uli rejected that with the claim that time_t was never intended to represent arbitrary times (i.e. future timestamps) :-(
Posted Mar 16, 2017 15:06 UTC (Thu)
by ianmcc (subscriber, #88379)
[Link]
Posted Mar 17, 2017 1:35 UTC (Fri)
by gdt (subscriber, #6284)
[Link] (1 responses)
undefined whether it is signed or unsigned C is even more undefined than you suggest. An implementation must use an "arithmetic type", but need not supply an arithmetic value within that type. The ANSI C Rationale says No arithmetic properties of these types are defined by the Standard … The representation need not be a
count of some basic unit; an implementation might conceivably represent different components
of a temporal value as subfields of an integer type. So you can see that ANSI C gives a programmer wanting to write portable time_t handling code very little. Only the ability to compare a time against (time_t)(-1) to check for errors. You could argue that ANSI C allows a time_t to be compared for equality, but you friend could counter-argue that the Standard doesn't forbid multiple representations for the same time in a time_t. ANSI C does require that a time in a struct tm returned by a library has only one representation. The ANSI C committee created difftime() to give the language more time_t handling ability; however that returns a double, which is problematic for some programming tasks. Thankfully POSIX/SUS applies additional requirements for time_t on UNIX-like systems. "The time() function shall return the value of time in seconds since the Epoch". As far as I can tell, POSIX/SUS makes no statement on the signedness or unsignedness of time_t, thus neither can be relied upon by a programmer. Implementation practice in AT&T and BSD UNIXen prior to POSIX was that time() was signed. Of course, such practice shouldn't be relied upon by a programmer.
Posted Mar 23, 2017 15:09 UTC (Thu)
by Wol (subscriber, #4433)
[Link]
Okay, I'm used to a database that represents dates as "days since epoch" (1 Jan 1967 iirc), but negative dates are a matter of course ... :-)
Cheers,
Posted Mar 16, 2017 9:13 UTC (Thu)
by marcH (subscriber, #57642)
[Link] (8 responses)
Interestingly enough timestamps are used not just for past and present but for *future* dates as well; for some unknown reason this aspect seems consistently dropped from discussions. Scheduling, planning, projecting, predicting, etc. For instance a mortgage plan can run past 2038 already *today*. Granted a kernel probably never looks that far ahead (doesn't it?) and no one sets alarms that many years in advance. However good to keep in mind that the concept of a future timestamp is very real otherwise/elsewhere.
Posted Mar 16, 2017 13:24 UTC (Thu)
by bandrami (guest, #94229)
[Link]
Posted Mar 16, 2017 14:41 UTC (Thu)
by gdt (subscriber, #6284)
[Link] (4 responses)
Has a calendar, scheduling or similar application ever been able to use the system time_t type as its representation for dates? Just as there is a 2038 problem I expect there might also be a 1910 problem. Even at the start of the UNIX epoch in 1970 there would have been birthdates outside the 1910-2038 range. Given that, I suspect that the UNIX system designers had not intended for the system time representation to be suitable for calendaring, scheduling, etc.
Posted Mar 16, 2017 18:45 UTC (Thu)
by rgmoore (✭ supporter ✭, #75)
[Link]
That intent doesn't matter. What matters is how the real-world programs people depend on deal with time, and plenty of them have gone ahead and used UNIX system time representation for all kinds of stuff. It's too late to berate programmers for using UNIX system time in domains it wasn't intended for; now we have to deal with the problems doing so has created.
Posted Mar 17, 2017 23:18 UTC (Fri)
by plugwash (subscriber, #29694)
[Link] (2 responses)
Most dates people use in everyday life are relatively close to the present day. Few things are scheduled more than a few years in advance.
Birthdates could be a potential issue but even in 1970 most people born in 1901 or earlier would have been retired.
Posted Mar 17, 2017 23:27 UTC (Fri)
by marcH (subscriber, #57642)
[Link] (1 responses)
Software that "mostly works" is indeed plenty good enough to sell and make money. So why would anyone go the extra mile? Any customer who can't tolerate infrequent crashes, security breaches and unsupported corner cases will either need to spend a lot more money or be left high and dry. Sometimes both.
Posted Mar 18, 2017 13:15 UTC (Sat)
by roblucid (guest, #48964)
[Link]
Posted Mar 17, 2017 9:14 UTC (Fri)
by pr1268 (subscriber, #24648)
[Link] (1 responses)
I suspect that most banks / mortgage lenders have already accommodated the Y2038 problem in their software with e.g. custom data structures for dates and times. I base my claim on remembering in the news from around 1998 or so that mentioned banks were already Y2K compliant because they anticipated that change as early as 1975 or so (25 year mortgages...). The U.S. Social Security Administration also received a grade of "A" for its preparedness (the only federal organization to receive such a grade). I also suspect (hope?) that most prepared organizations have written their own software to completely abandon all dependency on the kernel's (and C library's) representation of "epoch" seconds, irrespective of 32-bit or 64-bit, to ensure nothing breaks.
Posted Mar 17, 2017 11:24 UTC (Fri)
by NAR (subscriber, #1313)
[Link]
Posted Mar 16, 2017 12:59 UTC (Thu)
by giggls (subscriber, #48434)
[Link]
As I was able to set file dates to way beyond 2038 on my 64 Bit Linux system I would assume the latter, right?
Sven
Posted Mar 16, 2017 21:41 UTC (Thu)
by petkan (subscriber, #54713)
[Link] (8 responses)
Posted Mar 17, 2017 0:30 UTC (Fri)
by gdt (subscriber, #6284)
[Link]
The ABS' Motor Vehicle Census 2016 data cube Table 9 has 20% of all vehicles in Australia with a manufacturing date prior to 2000. That rises to 38% for non-passenger, non-freight vehicles (ie: real-life "Tonka" plant lasts as long as the toy). I glanced at two previous MVCs to check that the proportions were not radically different across the past decade. If many vehicles manufactured now do have a Y2038 issue then those statistics suggest that the effect in 2038 would be too substantial to be handwaved away. That in turn means there are national policy implications. Motor vehicles are many people's largest or second-largest asset. Alternatively, if vehicles are disposable, then having ~20% of the all vehicles retired immediately prior to 2038 has implications for disposal arrangements.
Posted Mar 21, 2017 5:13 UTC (Tue)
by louie (guest, #3285)
[Link] (6 responses)
Posted Mar 21, 2017 16:07 UTC (Tue)
by Jonno (subscriber, #49613)
[Link] (1 responses)
Honestly, the average age is a pretty meaningless statistic, because a small number of really old collector's cars can bring the number up significantly. Much more relevant for this discussion is _median_ (50-percentile) age and the 90-precentile age. I haven't been able to find those figures for the US, but for Sweden, they are 10 years and 19 years, the same at both the end of 2016 and the end of 2002 (oldest data available online [1]).
So if we want to instantly replace less than 10% of all cars in 2038, we need to fix this in time for auto-makers to include the fix in the 2019 model year cars, which means it has to be in the *next* longterm kernel release (circa 4.14) and second next glibc version (2.27). To get that done, we need to write, test, review and merge all relevant code *this year*!
[1]: http://www.scb.se/hitta-statistik/statistik-efter-amne/tr... (in Swedish).
Posted Mar 23, 2017 16:28 UTC (Thu)
by arnd (subscriber, #8866)
[Link]
Posted Mar 21, 2017 22:16 UTC (Tue)
by flussence (guest, #85566)
[Link]
(I imagine equivalent numbers for phone ownership would show the opposite trend...)
Posted Mar 23, 2017 12:10 UTC (Thu)
by temun (guest, #114741)
[Link] (2 responses)
The only part of my car which has any reason to care about standardised time is the radio, which sets its time from RDS. The RDS time (https://en.wikipedia.org/wiki/Radio_Data_System#Group_Typ...) uses a 17-bit field for the "Modified Julian Day Code" so it presumably stops working in Sep 2217, though I suppose there's a risk that it will fail in Apr 2038 if someone accidentally treated it as 16-bit and forgot to test. So my car may have a Y2038 problem, but it's a different one from the subject of this article. There are at least two dangerous dates in 2038: one in Jan (signed 32-bit seconds from 1970) and one in Apr (unsigned 16-bit days from 17 Nov 1858).
Posted Mar 23, 2017 22:48 UTC (Thu)
by bronson (subscriber, #4806)
[Link] (1 responses)
Posted Mar 23, 2017 22:59 UTC (Thu)
by bronson (subscriber, #4806)
[Link]
Not sure either of these will still be broadcast in 2038... we'll see.
Posted Mar 17, 2017 19:09 UTC (Fri)
by dpreed (subscriber, #21184)
[Link] (3 responses)
That seems like it would make file system internal storage, etc. not have the risk of changing alignment, etc.
Associated with a particular file system or ABI or locale, you could have a 64-bit offset that gets added to the 32-bit internal representation, and add that offset to present the full 64-bit number in a clean way.
Posted Mar 21, 2017 22:57 UTC (Tue)
by dvdeug (guest, #10998)
[Link]
Posted Mar 27, 2017 8:46 UTC (Mon)
by temun (guest, #114741)
[Link] (1 responses)
Some of the stand-alone GPS devices for use in cars must be getting on for 20 years old now. Do any of them have problems with the date? As far as I can recall, the elderly device which I occasionally use displays the time, but not the date, so perhaps it could continue working, in theory, for many centuries. The underlying OS is Linux, presumably 32-bit, but there's no compelling reason for the time displayed by the application software to be naively synchronised with the system time. The kernel might think it's 1 Jan 1970 every time it's rebooted. Does anyone know about this? Has anyone here ever broken into one of these Gamin/TomTom/... devices?
Posted Mar 27, 2017 15:55 UTC (Mon)
by mathstuf (subscriber, #69389)
[Link]
Posted Mar 22, 2017 22:57 UTC (Wed)
by PaulWay (guest, #45600)
[Link]
Have fun,
Paul
Posted Mar 23, 2017 19:14 UTC (Thu)
by ahinds (guest, #114751)
[Link]
Posted Mar 26, 2017 19:03 UTC (Sun)
by fest3er (guest, #60379)
[Link] (1 responses)
Posted Mar 26, 2017 19:42 UTC (Sun)
by liw (subscriber, #6379)
[Link]
And this is still a simplistic model, since it doesn't take into account retconning for narrative purposes at all. But adding the retcon compressor is left as an exercise for the reader.
After that, we can finally have some peace of mind as far as time representation goes, and tackle the three fundamnetal unsolved problems in computer science, namely cache validation, naming of things, and counting beyond two.
2038: only 21 years away
2038: only 21 years away
deb format doesn’t use cpio.
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
Wol
2038: only 21 years away
struct commit {
...
unsigned long date;
};
```
2038: only 21 years away
2038: only 21 years away
The NFSv3, ext3, and XFS filesystems all have problems resulting from their use of 32-bit timestamps. The first two are likely to have gone out of use long before the problem hits,
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
which are arithmetic types capable of representing times;
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
Wol
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
Given that, I suspect that the UNIX system designers had not intended for the system time representation to be suitable for calendaring, scheduling, etc.
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
xfs?
2038: only 21 years away
2038: only 21 years away
Cars today last longer than they ever have. In the US, the average age of a car on the road last year was 11.6 years, two full years longer than in 2002 (first year Institute for Highway Safety tracked the number). (cite) The US DOT's data is similar; 11.4 years in 2014, 9.8 years in 2002, and 5.1 years in 1969. (cite)2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away
2038: only 21 years away