Open
Description
In #189 we are investigating how MGARD performs on some very noisy data. When this data is compressed with a tight error tolerance and then decompressed, we get an error in huffman_decoding. It seems that an integer overflow is involved.
To reproduce, compile the MGARD CLI with debugging flags. Here's how I configured the build:
$ cmake -S . -B "build" \
-D CMAKE_BUILD_TYPE="Debug" \
-D BUILD_SHARED_LIBS=OFF \
-D BUILD_TESTING=OFF \
-D MGARD_ENABLE_OPENMP=ON \
-D MGARD_ENABLE_SERIAL=OFF \
-D MGARD_ENABLE_CLI=ONBuild and install, and then run make in the attached directory. Here's what I get (linebreaks added):
$ make
g++ -c generate.cpp -o generate.o
g++ generate.o -o generate
./generate 256 logistic.dat
mgard compress --datatype double --shape 256x256x256 --smoothness 0 --tolerance 1.0e-10 \
--input logistic.dat --output logistic_s_0_tol_1.0e-10.mgard
input size (bytes): 134217728
output size (bytes): 67116219
compression ratio: 1.99978
mgard decompress --input logistic_s_0_tol_1.0e-10.mgard --output logistic_s_0_tol_1.0e-10.dat
${HOME}/projects/MGARD/src/compressors.cpp:246:22: runtime error: signed integer overflow: \
-2147429659 - 65536 cannot be represented in type 'int'
mgard: ${HOME}/projects/MGARD/src/compressors.cpp:258: void mgard::huffman_decoding(long int*, \
std::size_t, unsigned char*, size_t, unsigned char*, size_t, unsigned char*, size_t): \
Assertion `start_bit == out_data_hit_size' failed.
makefile:37: recipe for target 'logistic_s_0_tol_1.0e-10.dat' failed
make: *** [logistic_s_0_tol_1.0e-10.dat] Aborted@qliu21, I'm assigning you since you wrote the function, but I can start looking into this and we can talk about it next week.