Replies: 3 comments 6 replies
-
|
I've moved this to a discussion. If you've read the docs, SCons thus far hasn't required any other packages besides what's in the python distribution to run. Feel free to develop this in the scons-contrib repo if you'd like to see if you can get this working with SCons. |
Beta Was this translation helpful? Give feedback.
-
|
I get that SCons shouldn't impose such an exotic library as a dependency on its users, but what about it being a soft dependency, i.e. only be used when present? It would correspond to a suggested package in Debian parlance. In the code I saw, there was a certain level of coupling to the hashlib which I didn't immediately grasped, so I didn't pursue it, in favor of another topics (which are done now). I'll see if I can find some time to bring it on. |
Beta Was this translation helpful? Give feedback.
-
|
We'd need to get more clever... what if several devs share a build tree or a cachedir, and one of them installs xxhash in their environment. Now everything she writes will be using xxhash signatures, and the other devs will have their builds break because their run of scons won't recognize those. At the least, we'd want to switch to a multihash scheme so scons could recognize the kind of hash and emit a sane message if it's one it can't deal with at that time. |
Beta Was this translation helpful? Give feedback.
-
|
Wouldn't such issues arise today when state is shared between FIPS-compliant (MD5) and FIPS-incompliant (SHA*) environments? Or even when a group member explicily chooses the hash algorithm (e.g. because they want to use hardware-accelerated SHA on their new machine)? |
Beta Was this translation helpful? Give feedback.
-
|
A rough sketch can be found here https://github.com/rico-chet/scons/commits/add-xxhash-support. |
Beta Was this translation helpful? Give feedback.
-
|
This might start with providing boilerplate code which users can integrate into their E.g. this should work (but doesn't): https://scons.org/doc/latest/HTML/scons-api/SCons.Util#SCons.Util.hashes.set_hash_format |
Beta Was this translation helpful? Give feedback.
-
|
you have to get it into |
Beta Was this translation helpful? Give feedback.
-
|
Adding a SCons expects all hash libraries to add an argument when run under Python 3.9+: Line 57 in ebf4678 |
Beta Was this translation helpful? Give feedback.
-
|
That story was kind of weird - we got some complaints that if you ran SCons in a FIPS-compliant environment (essentially, RHEL in a US-Govt setting), it failed because md5 was disabled in Python by policy. SCons doesn't use the hash for any cryptographic purpose, only for hashing. After adding other hash options than md5, a contributor figured out that Python as of 3.9 had added the option to declare that, with the new |
Beta Was this translation helpful? Give feedback.
-
Currently, SCons only supports hash algorithms which are provided by the
hashlib, but those are rather slow (in the ballpark of 1-2 GB/s).There are much faster alternatives available, e.g.
xxHashwhich runs at the speed of RAM (an order of magnitude faster than MD5, up to ~30 times, according toxxHashdevelopers).xxHashis readily available as a C and Python3 library via Debian repositories, so e.g. Ubuntu/Debian users can install it with ease. The hash is not cryptographic, but it's acceptable according to the comments I found in SCons code.Beta Was this translation helpful? Give feedback.
All reactions