kzalloc()

[Posted August 10, 2005 by corbet]

The kernel code base is full of functions which allocate memory with kmalloc(), then zero it with memset(). Recently, Pekka Enberg concluded that much of this code could be cleaned up by using kcalloc() instead. kcalloc() has this prototype:

    void *kcalloc(size_t n, size_t size, unsigned int __nocast gfp_flags);

This function will allocate an array of n items, and will zero the entire array before returning it to the caller. Pekka's patch converted a number of kmalloc()/memset() pairs over to kcalloc(), but that patch drew a complaint from Andrew Morton:

Notice how every conversion you did passes in `1' in the first argument? And that's going to happen again and again and again. Each callsite needlessly passing that silly third argument, adding more kernel text.

Very few callers actually need to allocate an array of items, so the extra argument is unneeded in most cases. Each instance of that argument adds a bit to the size of the kernel, and, over time, that space adds up. The solution was to create yet another allocation function:

    void *kzalloc(size_t size, unsigned int __nocast gfp_flags);

This function returns a single, zeroed item. It has been added to -mm, with its appearance in the mainline likely to happen for 2.6.14.

Index entries for this article
Kernel	kzalloc()

calloc idiocy

Posted Aug 11, 2005 13:53 UTC (Thu) by bronson (subscriber, #4806) [Link] (5 responses)

I have always thought that calloc/fwrite/etc's use of "nmemb" parameters was idiotic. Which is easier to read?
calloc(4, sizeof(t))
or
calloc(4 * sizeof(t))
For me, the latter. Especially with fwrite. And it's smaller and more general too.

Abolish nmemb!!

calloc idiocy

Posted Aug 12, 2005 6:21 UTC (Fri) by chad.netzer (subscriber, #4257) [Link] (4 responses)

You, of course, meant to say "malloc(4 * sizeof(t))", but we get your point.

However, what happens when n * sizeof(t) is greater than size_t? Then the request can succeed, due to overflow making a small number out of a very large number. You get a pointer to the memory you requested, but you got a much smaller amount than you expected. calloc() detects these errors and returns NULL.

Of course, you may expect that these cases are rare, and modern OSes will simply memory fault when accessing unallocated memory. However, on embedded systems, or kernel code, such memory protection may not exist, and this error can be much more problematic. If a user can somehow trick the system into generating this allocation (ie. without checking every array memory request that a user might be able to influence for overflow), you have a potential security bug. calloc() does exactly that, so it has its uses.

See this lkml post for more info:

http://marc.theaimsgroup.com/?l=linux-kernel&m=112324...

calloc idiocy

Posted Aug 12, 2005 21:33 UTC (Fri) by giraffedata (guest, #1954) [Link] (3 responses)

You, of course, meant to say "malloc(4 * sizeof(t))", but we get your point.

I'm pretty sure he meant what he wrote; it's what I would have written, anyway: an apples to apples comparison of two hypothetical designs for calloc(), one more readable than the other.

I find calloc(4, sizeof(t)) easier to read. It says rather explicitly that you're allocating space for 4 array elements, whereas calloc(4 * sizeof(t)) requires an extra mental step to go backwards through the arithmetic and say, "Aha. He's calculating how much memory a 4 element array would take."

I myself do even better. I use a macro thusly:


    t * arrayOfT;
    MALLOCARRAY(arrayOfT, 4);

calloc idiocy

Posted Aug 14, 2005 2:26 UTC (Sun) by chad.netzer (subscriber, #4257) [Link] (2 responses)

Huh? What would be the semantics of a "hypothetical" calloc(4 * sizeof(t)), and how would they be different from the malloc(4 * sizeof(t))? Clearly bronson meant to compare malloc() to calloc()

bronson's point was that if using malloc() and calloc() are almost equivalent, differing by just an internal multiplication, there would seem to be little need for calloc(). It is an explicit indication that you are allocating an array (as you stated), and he argued that this was less readable for him.

But:

    malloc(n * sizeof(t))

and

   calloc(n, sizeof(t))

are NOT equivalent. The calloc() is implemented as something like:

if (n && ((n*sizeof(t))/n == n))
    p = malloc(n * sizeof(t));
else
    p = NULL;

Assuming I haven't made an error, this protects against n being large enough to cause an overflow (or, for example, when n is an int, and it's a small negative number). The naive malloc() call could lead to serious complications, allocating a much smaller array than the caller expected.

Your macro example got garbled, so I won't comment on it.

calloc idiocy

Posted Aug 14, 2005 2:50 UTC (Sun) by giraffedata (guest, #1954) [Link] (1 responses)

I think you missed the fundamental distinction of calloc() from malloc(). The difference in how you specify the size is incidental. The reason calloc() exists is that it allocates memory that is set to zero ("c" is for "clear"), whereas malloc() allocates memory with arbitrary contents.

I'm not sure my macro example really did get garbled. It looks fine on my screen, but does look a little funny because I tried to be consistent with the earlier example that used a one-character type name. Let me try again:


  struct foo * arrayOfFooStructs;
  MALLOCARRAY(arrayOfFooStructs, 4);

It's supposed to be two lines. The first one declares an array variable (technically a pointer, but practically an array) and the second allocates space for a 4-element array and assigns its address to that pointer. I.e. it instantiates the array.

I have seen people use special "multiply and malloc" routines in order to deal with the arithmetic overflow while not taking the time to clear the memory as with calloc(). Also, these routines sometimes multiply more than two numbers together to get the size.

calloc idiocy

Posted Aug 14, 2005 23:03 UTC (Sun) by chad.netzer (subscriber, #4257) [Link]

Ahh, I see. I did ignore the clearing aspect in my replies; I overlooked this as the the distinction that perhaps you (and bronson) were talking about. But I hope all this does illustrate to bronson (if he or she is reading) that calloc() requires the "nmemb" argument for a reason, and that the overflow protection is easy to overlook, but an important feature of calloc().

Ok, I understand your macro example now. A compact way of doing the array allocation without the clear, and safer than malloc(n * sizeof(struct foo)). Thanks for rewriting the declaration.

Cheers.

kzalloc()

Posted Aug 11, 2005 15:01 UTC (Thu) by nathan (subscriber, #3559) [Link]

Let's not lose sight of the fact that the original two calls would still have been larger than the single kcalloc call --even with its useless parameter.

calloc()

Posted Aug 11, 2005 19:32 UTC (Thu) by brettlevin (guest, #29117) [Link] (1 responses)

Does anyone know the story behind the count parameter in libc's calloc()?

calloc()

Posted Aug 11, 2005 22:17 UTC (Thu) by brettlevin (guest, #29117) [Link]

I found a libc guru to ask.

In 4.1BSD the args to malloc and calloc were unsigned ints. There were at least some machines who had 16-bit ints, but which had more than 2^16 worth of addressable memory. In such an environment it might be useful to allocate more than 64Kb in one call.

Then POSIX changed these args to size_t, easing the size restriction.