Tiered memory allocation restricts available memory

This was first noticed when running a binary produced with GHC 8.0.2 through a queueing system (Sun Grid Engine) that limits available memory by setting ulimit -v.

When running with **unlimited** address space, the binary reserves 1TB of RAM and everything works as expected. When running with **limited** address space, reservation happens in powers of 2 leading to significant unused memory at higher memory values.

The tests are summarized in the following table:

    Limit   VIRT    Unused  (all in GB)
    8       4       8
    10      8       2
    16      8       8
    32      16      16
    34      32      2
    64      32      32
    66      64      2
    128     64      64
    130     128     2
    256     128     128
    258     256     2
    512     256     256
    514     512     2
    1024    512     512

In the last line, setting ulimit -v to 1024GB (1073741824) causes the binary to only be able to reserve 512GB. If at runtime the software needs more than 512GB it will abort with "out of memory" instead of expanding to as close to the specified ulimit as possible.

PS: Is there any way to make the "out of memory" error a little bit more informative? Something that informs the user of how much memory was already being used and how much more was going to be reserved when the failure was seen?

Trac metadata

Trac field	Value
Version	8.0.2
Type	Bug
TypeOfFailure	OtherFailure
Priority	normal
Resolution	Unresolved
Component	Runtime System
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system
Architecture