The 2012 Kernel Summit

The biggest difference between armhf and armel is that what an armhf compiler would have compiled into a single floating point instruction, an armel compiler will instead make into a function call.

On a floating point capable system that function will only be three instructions long (copy the arguments from integer registers to floating point registers, the actual floating point instruction, copy the result from a floating point register to an integer register), while on other systems the function will contain a complex emulation instead.

Thus, even armel will make use of the floating point capabilities of a processor that has one, but it will do so in an inefficient manner (for every instruction, add a function call and two register copies).

If a single user function contains many floating point operations, it is possible to minimize this cost by compiling two versions of it for armel, one that internally uses floating point instructions directly, and one that internally uses the standard emulation functions. That way you only have to pay the extra cost when crossing user-function-borders, rather than for every instruction. This can either be done on a function-by-function basis by the application developer, or on a library-by-library basis by the distributor (which is what Ben refers to). Adding such an alternative library for every fp-using library in Debian would have netted almost as large a benefit as that of Raspbian, without the need of forking the entire distribution, but I wouldn't want the job of trying to convince ~100 package maintainers and the release team that it was a worthwhile feature to add to wheezy in the middle of the pre-release freeze...