Commit 1b5f9207 authored by bitonic's avatar bitonic Committed by Tamar Christina
Browse files

Make start address of `osReserveHeapMemory` tunable via command line -xb

Summary:
We stumbled upon a case where an external library (OpenCL) does not work
if a specific address (0x200000000) is taken.

It so happens that `osReserveHeapMemory` starts trying to mmap at 0x200000000:

```
        void *hint = (void*)((W_)8 * (1 << 30) + attempt * BLOCK_SIZE);
        at = osTryReserveHeapMemory(*len, hint);
```

This makes it impossible to use Haskell programs compiled with GHC 8
with C functions that use OpenCL.

See this example ​https://github.com/chpatrick/oclwtf for a repro.

This patch allows the user to work around this kind of behavior outside
our control by letting the user override the starting address through an
RTS command line flag.

Reviewers: bgamari, Phyx, simonmar, erikd, austin

Reviewed By: Phyx, simonmar

Subscribers: rwbarton, thomie

Differential Revision: https://phabricator.haskell.org/D2513
parent 65d9597d
......@@ -33,6 +33,10 @@ Compiler
initial cmm from STG-to-C-- code generation and :ghc-flag:`-ddump-cmm-verbose`
to obtain the intermediates from all C-- pipeline stages.
- The RTS :ghc-flag: `-xb` now reads the base heap address in any base,
defaulting to decimal, hexadecimal if the address starts with `0x`, and
octal if the address starts with `0`.
Runtime system
~~~~~~~~~~~~~~
......
......@@ -124,6 +124,8 @@ typedef uint32_t StgHalfWord;
#define FMT_HexWord FMT_HexWord64
#define FMT_Int FMT_Int64
#define strToStgWord strtoull
#elif SIZEOF_VOID_P == 4
typedef int32_t StgInt;
typedef uint32_t StgWord;
......@@ -139,6 +141,8 @@ typedef uint16_t StgHalfWord;
#define FMT_HexWord FMT_HexWord32
#define FMT_Int FMT_Int32
#define strToStgWord strtoul
#else
#error GHC untested on this architecture: sizeof(void *) != 4 or 8
#endif
......
......@@ -271,6 +271,9 @@ usage_text[] = {
" -O<size> Sets the minimum size of the old generation (default 1M)",
" -M<size> Sets the maximum heap size (default unlimited) Egs: -M256k -M1G",
" -H<size> Sets the minimum heap size (default 0M) Egs: -H24m -H1G",
" -xb<addr> Sets the address from which a suitable start for the heap memory",
" will be searched from. This is useful if the default address",
" clashes with some third-party library.",
" -m<n> Minimum % of heap which must be available (default 3%)",
" -G<n> Number of generations (default: 2)",
" -c<n> Use in-place compaction instead of copying in the oldest generation",
......@@ -1293,7 +1296,7 @@ error = rtsTrue;
OPTION_UNSAFE;
if (rts_argv[arg][3] != '\0') {
RtsFlags.GcFlags.heapBase
= strtol(rts_argv[arg]+3, (char **) NULL, 16);
= strToStgWord(rts_argv[arg]+3, (char **) NULL, 0);
} else {
errorBelch("-xb: requires argument");
error = rtsTrue;
......
......@@ -450,16 +450,18 @@ osTryReserveHeapMemory (W_ len, void *hint)
return start;
}
void *osReserveHeapMemory(W_ *len)
void *osReserveHeapMemory(void *startAddressPtr, W_ *len)
{
int attempt;
void *at;
/* We want to ensure the heap starts at least 8 GB inside the address space,
to make sure that any dynamically loaded code will be close enough to the
original code so that short relocations will work. This is in particular
important on Darwin/Mach-O, because object files not compiled as shared
libraries are position independent but cannot be loaded about 4GB.
since we want to reserve the address space below that address for code.
Specifically, we need to make sure that any dynamically loaded code will
be close enough to the original code so that short relocations will work.
This is in particular important on Darwin/Mach-O, because object files
not compiled as shared libraries are position independent but cannot be
loaded above 4GB.
We do so with a hint to the mmap, and we verify the OS satisfied our
hint. We loop, shifting our hint by 1 BLOCK_SIZE every time, in case
......@@ -472,6 +474,19 @@ void *osReserveHeapMemory(W_ *len)
*/
W_ minimumAddress = (W_)8 * (1 << 30);
// We don't use minimumAddress (0x200000000) as default because we know
// it can clash with third-party libraries. See ticket #12573.
W_ startAddress = 0x4200000000;
if (startAddressPtr) {
startAddress = (W_)startAddressPtr;
}
if (startAddress < minimumAddress) {
errorBelch(
"Provided heap start address %p is lower than minimum address %p",
(void*)startAddress, (void*)minimumAddress);
}
attempt = 0;
while (1) {
if (*len < MBLOCK_SIZE) {
......@@ -479,7 +494,7 @@ void *osReserveHeapMemory(W_ *len)
barf("osReserveHeapMemory: Failed to allocate heap storage");
}
void *hint = (void*)((W_)8 * (1 << 30) + attempt * BLOCK_SIZE);
void *hint = (void*)(startAddress + attempt * BLOCK_SIZE);
at = osTryReserveHeapMemory(*len, hint);
if (at == NULL) {
// This means that mmap failed which we take to mean that we asked
......@@ -487,7 +502,7 @@ void *osReserveHeapMemory(W_ *len)
// limits. In this case we reduce our allocation request by a factor
// of two and try again.
*len /= 2;
} else if ((W_)at >= ((W_)8 * (1 << 30))) {
} else if ((W_)at >= minimumAddress) {
// Success! We were given a block of memory starting above the 8 GB
// mark, which is what we were looking for.
break;
......@@ -498,6 +513,7 @@ void *osReserveHeapMemory(W_ *len)
sysErrorBelch("unable to release reserved heap");
}
}
attempt++;
}
return at;
......
......@@ -662,7 +662,11 @@ initMBlocks(void)
#else
size = (W_)1 << 40; // 1 TByte
#endif
void *addr = osReserveHeapMemory(&size);
void *startAddress = NULL;
if (RtsFlags.GcFlags.heapBase) {
startAddress = (void*) RtsFlags.GcFlags.heapBase;
}
void *addr = osReserveHeapMemory(startAddress, &size);
mblock_address_space.begin = (W_)addr;
mblock_address_space.end = (W_)addr + size;
......
......@@ -58,7 +58,10 @@ roundUpToPage (size_t x)
// to the amount of memory actually reserved.
//
// This function is called once when the block allocator is initialized.
void *osReserveHeapMemory(W_ *len);
//
// startAddress must be greater or equal than 8 * (1 << 30), and can be
// NULL, in which case a default will be picked by the RTS.
void *osReserveHeapMemory(void *startAddress, W_ *len);
// Commit (allocate memory for) a piece of address space, which must
// be within the previously reserved space After this call, it is safe
......
......@@ -430,19 +430,20 @@ void setExecutable (void *p, W_ len, rtsBool exec)
static void* heap_base = NULL;
void *osReserveHeapMemory (W_ *len)
void *osReserveHeapMemory (void *startAddress, W_ *len)
{
void *start;
heap_base = VirtualAlloc(NULL, *len + MBLOCK_SIZE,
heap_base = VirtualAlloc(startAddress, *len + MBLOCK_SIZE,
MEM_RESERVE, PAGE_READWRITE);
if (heap_base == NULL) {
if (GetLastError() == ERROR_NOT_ENOUGH_MEMORY) {
errorBelch("out of memory");
} else {
sysErrorBelch(
"osReserveHeapMemory: VirtualAlloc MEM_RESERVE %llu bytes failed",
len + MBLOCK_SIZE);
"osReserveHeapMemory: VirtualAlloc MEM_RESERVE %llu bytes \
at address %p bytes failed",
len + MBLOCK_SIZE, startAddress);
}
stg_exit(EXIT_FAILURE);
}
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment