C Memory leak, or maybe not?

S

symlink

Hello,

I'm writing a software in C. As you know, in C we don't have standard library provided hash tables, so I'm relying on a header-only library that I found on github.

While designing this software I was very careful with memory: I pre-allocated the maximum amount of session objects, and I made sure that the hash table which is bookmarking sessions is large enough to minimize the number of reallocations. The program will evict sessions in LRU fashion if we have too many, hence freeing the handler in the hash table. A little testing highlighted that, despite my efforts, the hash table implementation needs to do some re-shuffling from time to time. This happens fairly infrequently, so I think it is acceptable.

Once my software seemed baked and ready to be tested, I made it run on my FreeBSD 15 box. It is a daemon, it needs to work for long time, so memory footprint is a thing to check. I found that the process keeps piling memory, despite address sanitizers and valgrind not detecting any defect.

I could eventually isolate the issue, and the culprit seems to be to the hash table I mentioned above. The implementation will sporadically issue `free(3)` for a certain buffer, followed by a new `malloc(3)`. ...which is perfectly OK, on my book, since allocated blocks are freed on the next re-shuffle, and everything is cleaned when the process is terminated. I concluded that the memory handling is reasonable. Using a memory profiler seems to confirm it. Moreover, under Linux the very same daemon does not show any memory leaking behavior.

I start wondering if this problem could be a non-problem, and maybe just being related to how FreeBSD handles memory. E.g. I'd like to test what would happen if I capped memory to a certain maximum: will I end up out of memory?

Do you have any recommendation?
 
Do you have any unit tests? Oh wait, I misread. There's not enough info here. Are all the allocations the same size? Can you re-use a prior allocation? Are you expecting memory to drop following a re-shuffle?
 
What memory counters are increasing? Is it possible that you are touching new pages, increasing the address mapped space, rather that just using more memory?
 
You may not have a memory leak but you may have a program logic bug. Put in some counters and checking code. Also watch /proc/<daemon pid/map and see if over time it is showing some unusual behavior. Some time even just printing stuff can help (print a counter every N malloc and every N free etc.). You can also add a conditional breakpoint in gdb/lldb and manually invoke the checking code. Finally, write a separate test program just for the hashtable and simulate the kind of usage your appn makes.
 
symlink

POSIX does have standard hash tables hcreate(3). You can create multiple tables with hcreate_r(3).

Regarding your implementation, if you are looking at the memory use from the system's perspective, you may see it as leaking memory. FreeBSD uses jemalloc(3). Its behaviour may be different from you might see in other systems. It does not immediately give memory back to the OS if it expects that it may have to allocate from the OS again.
 
cracauer@ said:
If Valgrind is happy it is more likely a memory fragmentation issue.
Click to expand...
Ahh, what is a memory leak?

If allocs and frees are matched that means the program is "freeing 1 for each alloc 1" but that does not mean the OS reclaims at the same rate. This is often a problem on Java or other garbage collecting systems.

That comes back to what cracauer@ points out: program is balanced on allocs and frees, but the OS needs to reclaim which may lead to fragmentation.
 
What's the program main loop control? If it's 1 thread I/O event-driven, there must be an action to associate with the increasing resident menory that doesn't happen in all iterations. A function call related to a hashtable?
I never used any hashtables that C books describe. It's mostly my own construction... Are you sure nothing is written beyond a defined data field boundary?
 
As a quick hack, grep for every use of *alloc and free in the library and instrument it with a simple integer increment/decrement.

Obviously realloc is more difficult (and many uses of it are simply wrong, i.e realloc with size 0 is *not* the same as free. I see this so often....)
 
If it was a plain leak, as in not freeing something that was malloced or realloced, a simple bookkeeping would detect it. If Valgrind is happy it's not that.

How do you measure the size of the program, anyway?
 
cracauer@ said:
If it was a plain leak, as in not freeing something that was malloced or realloced, a simple bookkeeping would detect it. If Valgrind is happy it's not that.
Click to expand...
Yeah, to clarify, I also mean things like putting a manual inc/dec with all allocs, i.e: glGenTextures, glDeleteTextures. If this is outside of Valgrind's VM (which graphical elements sometimes are due to display servers), sometimes it can't track it.
 
First off, thank you all for your kind answers, and sorry for the delay.

A few answers:

Yes, all the allocations that the hash table implementation does have the same size (answering question from elephant).

Touching new pages could be a good explanation: I mentioned that the library seems to alternate calls to malloc() and free(). The pattern is something like:

Code: 
    malloc(128) -> x1
    ...many hash operations...
    free(x1) + malloc(128) -> x2
    ...many hash operations...
    free(x2) + malloc(128) -> x3
    ...many hash operations..
    free(x3) + malloc(128) -> x4
    ...
    ...
    program termination:
    free(xN)

So basically the library keeps allocating a new chunk and then freeing the previous. It keeps getting different buffer addresses from the system. So maybe it is not actually leaking, maybe it is just how the allocator works.

I'll try to see what happens if I manage to force the library to recycle the buffer of the previous step... :)

I'll also try to apply other suggestions here you're kindly providing.

jwillia3 - Yes, hcreate_r(3) might be an option to consider (less dependencies is better!) but I will still try to figure out what is going on!

MG - The program is single threaded. I know exactly what action ends up in hash table operations. I'm sure no write out of boundary is happening because valgrind and sanitizers would complain. Why would it be related anyway?

cracauer - The size of the program is monitored via top/htop, and that made me jump on the chair. Later on I've used Valgrind's Massif to check the memory growth, and that seems to be constant. Which seems to give credit to the memory fragmentation guesses of many in this thread.

Sorry for the very long text, and thanks again.
 
symlink said:
I mentioned that the library seems to alternate calls to malloc() and free(). The pattern is something like:

Code: 
    malloc(128) -> x1
    ...many hash operations...
    free(x1) + malloc(128) -> x2
    ...many hash operations...
    free(x2) + malloc(128) -> x3
    ...many hash operations..
    free(x3) + malloc(128) -> x4
    ...
    ...
    program termination:
    free(xN)

So basically the library keeps allocating a new chunk and then freeing the previous. It keeps getting different buffer addresses from the system. So maybe it is not actually leaking, maybe it is just how the allocator works.
Click to expand...
Can you modify the hashtable library so that it doesn't use the heap at all, and instead statically allocate a buffer for it to use and pass that to the hashtable library? And hence eliminate all of the calls to malloc and free? How does it help you to malloc then free the same sized block repeatedly? I realise you didn't write the hashtable but got it from github. It might be worth checking the hashtable code and seeing if there are any compile-time options to make it use either the heap or static data, the guy who wrote it may have already thought of this. If you can eliminate the sequence of malloc-free calls, you will eliminate the heap fragmentation.
 
symlink said:
The implementation will sporadically issue `free(3)` for a certain buffer, followed by a new `malloc(3)`
Click to expand...

To fix this issue -- use a memory pool.

Memory is allocated and freed from the "the pool", and then "reused" from the pool (aka instead of from malloc/calloc/etc and free). So your process is not allocating memory directly from the O/S but instead to and from the "pool". There are likely many open source memory pool implementations written in "C" that you can obtain and use. You can likely ask your favorite AI where you can download an open source C memory pool implementation... but I don't know because... I don't use AI :-).

If you wrote this code in C++ you can create an ALLOCATOR for your std::map (aka hash map). But it sounds like you are using straight "C". But, if you are willing to mix C and C++ you can use std::map<> with a C++ ALLOCATOR (which defines the memory pool and parameters) and get a memory pool that way.

Memory pools work GREAT on Linux, Mac, FreeBSD (or *BSDs) - so your code will remain portable to all.
 
symlink said:
cracauer - The size of the program is monitored via top/htop, and that made me jump on the chair. Later on I've used Valgrind's Massif to check the memory growth, and that seems to be constant. Which seems to give credit to the memory fragmentation guesses of many in this thread.
Click to expand...

If you really want to compare Linux and FreeBSD you should use the same malloc(3) implementation on both. An example is tcmalloc, on FreeBSD in port google-perftools, on Linux usually in its own package.
 
You must log in or register to reply here.
Back
Top