Provide RTS option to return memory non-lazily to the OS on Linux
Motivation
On Linux GHC returns memory to the OS by calling madvise(MADV_FREE) on pages it no longer needs.
As per madvise man page, this returns memory "lazily" -- only under memory pressure does Linux actually adjust the resident memory (RSS) of the process.
That is faster than prompt adjustments, but results high RSS numbers being shown in programs like top, htop and so on, even though GHC has already freed that memory. That can result in end users wrongly complaining about high memory usage, and make investigating memory usage trickier.
By the way
The amount of "memory to be lazily freed" can be observed in /proc/PID/smaps, with the LazyFree field. For example: grep LazyFree /proc/8429/smaps | grep -v '0 kB'. @nh2 made himself an ekg field to track it, in case you ever need too.
Proposal
Keep the current default, but add an RTS flag that switches to MADV_DONTNEED instead of MADV_FREE instead, which will result in prompt RSS adjustment. Do this only on Linux.
Go did did this in https://github.com/golang/go/issues/28466.