Low bits per key

As I understand the CHD Algorithm it is suppose to be space-efficient for the MPHF replacement size, however the DisplacementMap seems to always be n bytes (4n bits) where n is the number of keys, shouldn't it be closer to 2 bits per key?

For example 65536 (2^16) keys become 2^16/4 unsigned integers which is 65536 bytes. How should the DisplacementMap be saved?