proc/pid/maps structure clarification

[sternr]

Hi,

This is probably a stupid question, but I want to be sure:

when looking at the maps file of a specific process (proc/<pid>/maps)
The heap's memory is combined from all the entries between the [heap] line and the [stack] line?

If so, from the little I remember from OS 101, isnt the heap supposed to be continuous?

Thanks!

--sternr

[Richard Kettlewell]

Post by sternr
when looking at the maps file of a specific process (proc/<pid>/maps)
The heap's memory is combined from all the entries between the [heap]
line and the [stack] line?
If so, from the little I remember from OS 101, isnt the heap supposed to be continuous?

There's no reason a heap data structure has to be contiguous. (There
are binary heaps which are completely unrelated to allocation heaps, but
if you're talking about an OS class you're hopefuly not suffering from
that particular confusion. And obviously a particular implementation
might require contiguity. But it's not a general rule.)

[heap] itself is a contiguous mapping controlled by brk/sbrk. But
malloc() is not confined to that region; for larger allocations it will
create anonymous mappings instead. (Experimentally on amd64, the limit
is malloc(131048), which is obviously 128KB minus a few header bytes.)

The mappings you see between [heap] and [stack] will include shared
library mappings and any mappings made the program as it runs, neither
of which involve malloc() or the heap.

[Jorgen Grahn]

Post by Richard Kettlewell

Post by sternr
when looking at the maps file of a specific process (proc/<pid>/maps)
The heap's memory is combined from all the entries between the [heap]
line and the [stack] line?

...

Post by Richard Kettlewell
[heap] itself is a contiguous mapping controlled by brk/sbrk. But
malloc() is not confined to that region; for larger allocations it will
create anonymous mappings instead. (Experimentally on amd64, the limit
is malloc(131048), which is obviously 128KB minus a few header bytes.)

No need to experiment -- the malloc(3) man page describes this in detail
(128K is the default, but you can change the limit at runtime).

/Jorgen

[sternr]

First, thanks!

Second, so if I understand you correctly, by looking at the different entries in maps, there's no way to differentiate what blocks were mapped for\by malloc?
If so, is there any other way?
Thanks!

[Richard Kettlewell]

Post by sternr
First, thanks!
Second, so if I understand you correctly, by looking at the different
entries in maps, there's no way to differentiate what blocks were
mapped for\by malloc?

Indeed.

Post by sternr
If so, is there any other way?

You could modify malloc() to report its mappings, or inspect its
internal data structures with a debugger.

[sternr]

Ok, last question than:

When malloc doesnt use the heap, it uses mmap to allocate the requested memory.
If I understand correctly, this type of allocation is anonymous thus will be one of the unnamed entries in the maps "file"?

So while not all unnamed entries in maps contain malloc allocations, all malloc allocations must be contained in unnamed entries?

Thanks!

[Richard Kettlewell]

Post by sternr
When malloc doesnt use the heap, it uses mmap to allocate the
requested memory. If I understand correctly, this type of allocation
is anonymous thus will be one of the unnamed entries in the maps
"file"?

Yes.

Post by sternr
So while not all unnamed entries in maps contain malloc allocations,
all malloc allocations must be contained in unnamed entries?

Apart from [heap], yes.

[sternr]

Thanks alot!

[David Schwartz]

Post by sternr
If so, from the little I remember from OS 101, isnt the heap supposed to be continuous?

It really cannot be. How can you handle multiple heap allocations from
different threads running on different CPUs and keep the heat
contiguous without really ugly locking?

And what happens if you allocate 1GB on the heap, allocate a few more
bytes, and then free the 1GB? How can the heap stay contiguous in that
case? You don't want to keep the 1GB allocated just to make the heap
contiguous. And you can't move the few bytes allocated after.

DS