Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> However, mmap has other uses too. It's commonly used to allocate large regions of memory for applications.

Slack is allocating 1132 GB of virtual memory on my laptop right now. I don't know if they are using mmap but that's 1100 GB more than the physical memory.



That is Chromium doing it, and yes, it is using mmap to create a very large, (almost certainly) contiguous range of memory. Many runtimes do this, because it's useful (on 64-bit systems) to create a ridiculously large virtually mapped address space and then only commit small parts of it over time as needed, because it makes memory allocation simpler in several ways; notably it means you don't have to worry about allocating new address spaces when simply allocating memory, and it means answering things like "Is this a heap object?" is easier.


dolphin emulator has recent example of this: https://dolphin-emu.org/blog/2023/11/25/dolphin-progress-rep...

seems its not without perils on Windows:

"In an ideal world, that would be all we have to say about the new solution. But for Windows users, there's a special quirk. On most operating systems, we can use a special flag to signal that we don't really care if the system has 32 GiB of real memory. Unfortunately, Windows has no convenient way to do this. Dolphin still works fine on Windows computers that have less than 32 GiB of RAM, but if Windows is set to automatically manage the size of the page file, which is the case by default, starting any game in Dolphin will cause the page file to balloon in size. Dolphin isn't actually writing to all this newly allocated space in the page file, so there are no concerns about performance or disk lifetime. Also, Windows won't try to grow the page file beyond the amount of available disk space, and the page file shrinks back to its previous size when you close Dolphin, so for the most part there are no real consequences... "


I’m not sure allocations mean anything practical anymore. I recall OSX allocating ridiculous amounts of virtual memory to stuff but never found OSX or the software to ever feel slow and pagey.


The way I describe mmap these days is to say it allocates address space. This can sometimes be a clearer way of describing it, since the physical memory will only get allocated once you use the memory (maybe never).


But is it not still limited by allocating the RAM + Page/Swap size?


Maybe I'm misunderstanding you but: no, you can allocate terabytes of address space on modern 64-bit Linux on a machine with only 8GB of RAM with overcommit. Try it; you can allocate 2^46 bytes of space (~= 100TB) today, with no problem. There is no limit to the allocation space in an overcommit system; there is only a limit to the actual working set, which is very different.


You can do it without overcommit -- you can just back the mmap with file


I don't think so, but it's difficult to find an actual reference. For sure it does overcommit like crazy. Here's an output from my mac:

% ps aux | sort -k5 -rh | head -1

xxxxxxxx 88273 1.2 0.9 1597482768 316064 ?? S 4:07PM 35:09.71 /Applications/Slack.app/Contents/Frameworks/Slack Helper (Renderer).app/...

Since ps displays vsz column in KiB, 1597482768 corresponds to 1TB+.


I don't know why but this really makes me laugh




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: