Short answer

• In current glibc (ptmalloc2), free() does not call the public malloc_trim(). It never has in the multi-arena ptmalloc2 code used by glibc.
• What free() does do “automatically” is a per-arena top-of-heap trim via an internal helper (systrim/sys_trim/heap_trim) when the top chunk becomes large enough. That is the behavior the old comments and the man page sentence allude to.
• The public malloc_trim() is only run when your program calls it (or in glibc’s tests). It is not invoked implicitly by free(), malloc(), or malloc_consolidate().

Details and history

• dlmalloc (Doug Lea’s original allocator) defined malloc_trim(size_t pad) long ago. In some dlmalloc versions the free path used an internal “trim check” that could end up calling the same underlying trimming logic. In dlmalloc, “calling malloc_trim” vs “calling sys_trim” was effectively the same operation because malloc_trim just routed to sys_trim for the current state.
• ptmalloc2 (the glibc malloc since the early 2000s) extends dlmalloc with multiple arenas (for concurrency). In ptmalloc2/glibc, the per-free “automatic trimming” is done directly by the internal systrim/sys_trim/heap_trim functions on the arena being modified. The public malloc_trim() is intentionally not called from free() because:
  • malloc_trim() is a global/expensive operation: it iterates over arenas and (since glibc 2.9) may issue MADV_DONTNEED on free pages in the middle of heaps. Doing that on every free would be far too costly and add unpredictable latency.
  • The inexpensive and useful part to do automatically is only the per-arena top-of-heap shrink via brk/sbrk (or unmapping a mmapped heap when it becomes entirely free). That’s what systrim/heap_trim handle.

Where the confusion comes from

• The glibc source comments you quoted are old and generic, reflecting dlmalloc-era wording. They say “via malloc_trim in free()” but in ptmalloc2/glibc the free path goes straight to systrim/heap_trim, not the public malloc_trim().
• The man page note “This function is automatically called by free(3) in certain circumstances” is imprecise. The automatic part is the internal trimming of the top chunk (via systrim/heap_trim), not a call to the public malloc_trim().

What each function does

• systrim/sys_trim/heap_trim (internal):

  • Operates on a single arena.
  • Returns memory only from the top of that arena’s heap if the top chunk is large (beyond M_TRIM_THRESHOLD, with M_TOP_PAD considered).
  • Uses sbrk-negative when the main arena is brk-based, or may unmap an entire mmapped sub-heap when possible. It does not scan bins or madvise interior pages.
  • Called automatically from free() after consolidation when the freed space reaches the top and exceeds the threshold.

• malloc_trim(size_t pad) (public):

  • Iterates over all arenas.
  • Calls the same top-of-heap trimming logic as above for each arena.
  • Since glibc 2.9 (2007 change), also attempts to release whole free pages in the middle of heaps via madvise(MADV_DONTNEED), something the automatic free path does not do.
  • Not called automatically by glibc; call it explicitly if you want a global cleanup pass.

Practical implication

• If your process has steady live memory but growing RSS due to fragmentation and interior free pages, you must call malloc_trim() explicitly (periodically or after known reclamation phases). The automatic trimming done on free() only shrinks from the top and won’t madvise interior free pages.