GHC issueshttps://gitlab.haskell.org/ghc/ghc/-/issues2022-05-25T11:01:09Zhttps://gitlab.haskell.org/ghc/ghc/-/issues/21373test-cabal-reinstall nightly job failing2022-05-25T11:01:09ZMatthew Pickeringtest-cabal-reinstall nightly job failingJob: https://gitlab.haskell.org/ghc/ghc/-/jobs/1007220
```
$ cabal update --index=$HACKAGE_INDEX_STATE --project-file=cabal.project-reinstall
index-state must be a unix-timestamps (e.g. '@1474732068'), a ISO8601 UTC timestamp (e.g. '20...Job: https://gitlab.haskell.org/ghc/ghc/-/jobs/1007220
```
$ cabal update --index=$HACKAGE_INDEX_STATE --project-file=cabal.project-reinstall
index-state must be a unix-timestamps (e.g. '@1474732068'), a ISO8601 UTC timestamp (e.g. '2016-09-24T17:47:48Z'), or 'HEAD'
CallStack (from HasCallStack):
error, called at src/Distribution/ReadE.hs:42:24 in Cabal-3.6.2.0-inplace:Distribution.ReadE
Running after_script
```Matthew PickeringMatthew Pickeringhttps://gitlab.haskell.org/ghc/ghc/-/issues/19897Long GC pauses while collecting compact regions2022-05-02T14:17:09ZMarcin RzeźnickiLong GC pauses while collecting compact regions## Summary
In one of our projects, we noticed very long GC pauses. We have been able to pin that down to cases where long-lived caches have become garbage. We keep these caches in the form of compact regions, and they can grow quite lar...## Summary
In one of our projects, we noticed very long GC pauses. We have been able to pin that down to cases where long-lived caches have become garbage. We keep these caches in the form of compact regions, and they can grow quite large (the largest are just around 128GB). While we realized that the release of compact regions is linear in the region's block list length, the timings we saw looked rather quadratic e.g.
| Cache size [b] | GC pause [s] |
|-----------------|--------------|
| 2_533_449_728 | 0.123 |
| 2_962_022_400 | 0.169 |
| 3_768_795_136 | 0.214 |
| 5_052_579_840 | 0.286 |
| 9_247_870_976 | 1.016 |
| 14_269_587_456 | 3.470 |
At this rate, we expect a pause of approx. 400s for 128 GB, and that's what happens. The problem seems to be that `free_mblock_list` can grow in proportion to the region's block list (please see the perf samples). This behavior can be reliably reproduced, so it's probably not just us. To do so I used the following program:
```haskell
#!/usr/bin/env stack
{- stack
--resolver lts-17.11
script
--optimize
--package base
--package ghc-compact
--package primitive
--ghc-options -threaded
--ghc-options "-with-rtsopts=-N -I0 -qg -S"
-}
{-# LANGUAGE NumericUnderscores #-}
import Control.Monad (replicateM)
import Data.IORef
import Data.Primitive.ByteArray
import GHC.Compact
import System.Environment (getArgs)
import System.Mem (performGC)
main :: IO ()
main = do
a <- getArgs
let x = read $ head a
compactStuffRef <- newIORef =<< allocCompactStuff x
stuffSize <- compactSize =<< readIORef compactStuffRef
putStrLn $ "Compact size [b]: " ++ show stuffSize
putStrLn "*************************"
putStrLn "Press any key to start GC"
putStrLn "************************* "
_ <- getChar
writeIORef compactStuffRef undefined
performGC
putStrLn "*************************"
putStrLn "After GC"
putStrLn "************************* "
return ()
allocCompactStuff n = do
region <- compact' =<< chunk
go 0 region
where
chunk = replicateM 512 $ allocBytes 3276 -- alloc just below LARGE_OBJECT_THRESHOLD
go m region
| m == n = pure region
| otherwise = go (m + 1) =<< addChunk region
-- adds a lot of fragmentation to the free_mblock_list
addChunk region = compactAdd region . getCompact . head =<< replicateM 64 (compact' =<< chunk)
allocBytes :: Int -> IO ByteArray
allocBytes x = unsafeFreezeByteArray =<< newByteArray x
compact' :: a -> IO (Compact a)
compact' = compactSized 1_032_192 False
```
The goal I had in mind here is to "grow" the `free_mblock_list` proportionally to the length of some compact region (governed by the argument passed to the program) - it seems that this can be reliably achieved by allocating many smaller compacts and keeping only some of them. This more or less agrees with what we have seen in the actual program. If you measure GC times after releasing the "big" compact, it looks like this:
| x | Compact size [b]| GC pause [s] |
|------|-----------------|--------------|
|1_000 | 2_099_015_680 | 0.222 |
|2_000 | 4_196_167_680 | 0.465 |
|4_000 | 8_390_471_680 | 1.067 |
|8_000 | 16_779_079_680 | 3.065 |
|10_000| 20_973_383_680 | 4.078 |
|16_000| 33_550_295_680 | 9.059 |
If you attach `perf` after "Press any key ...", the following picture emerges:
![perf-bench.svg](/uploads/64b1e00bda953608e97da9d87d702644/perf-bench.svg)
which seems to favor the explanation of quadratic behavior emerging from loops in `free_mega_group`.
## Steps to reproduce
Please run the attached program
## Expected behavior
Keep GC pause linear in the size of compact region (perhaps a data structure better than linked-list would do)
## Environment
* GHC version used: 8.10.4, also reproduced on 8.8.4
Optional:
* Operating System: Linux
* System Architecture: x86_649.4.1Ben GamariBen Gamarihttps://gitlab.haskell.org/ghc/ghc/-/issues/18757Compact Region allocator creates unexpectedly small blocks2021-09-29T14:15:37ZFabian ThorandCompact Region allocator creates unexpectedly small blocks## Summary
When adding a large object of size `X` to a compact region which has an even larger default block size `Y`
(as specified in the initial `compactSized` call), then the compact region allocator allocates a compact
region block ...## Summary
When adding a large object of size `X` to a compact region which has an even larger default block size `Y`
(as specified in the initial `compactSized` call), then the compact region allocator allocates a compact
region block just large enough for accomodating for `X`, even though it could also just allocate a block
of the larger size `Y`, under the assumption that there most likely will be more object to be added to the
compact region.
There are some issues with having a lot of small compact region blocks:
- If the small blocks are spread out across more megablocks than stricly needed in terms of size,
fragmentation can prevent the RTS from returning memory to the OS (similar to the situation in https://www.well-typed.com/blog/2020/08/memory-fragmentation/, as compact region blocks are pinned)
- A lot of small blocks in the linked list of blocks can slow down allocation, when the nursery is not the last block (see https://gitlab.haskell.org/ghc/ghc/-/issues/17640)
- Smaller blocks mean that the nursery will be exhausted more often, resulting in more calls to `allocateForCompact` from the compaction itself.
## Steps to reproduce
The following Haskell program creates a new compact region with a default block size of one megablock
and then fills it with a few hundred large objects.
```haskell
{-# LANGUAGE MagicHash #-}
{-# LANGUAGE UnboxedTuples #-}
import Control.Monad
import GHC.Compact
import GHC.Compact.Serialized
import GHC.IO
import GHC.Prim
main :: IO ()
main = do
let
blocksPerMBlock, blockSize, dataBytesInMegablock :: Integral a => a
blocksPerMBlock = 252
blockSize = 4096
dataBytesInMegablock = blocksPerMBlock * blockSize
-- Create a new compact region with the maximum possible block size
-- (1 megablock per compact region block)
region <- compactSized dataBytesInMegablock False ()
largeObject <- newLargeObject
-- Add the large object a few times to our compact region:
replicateM 510 $ void $ compactAdd region largeObject
-- Now check how many blocks were allocated, and how much data they each contain
blockSizes <- withSerializedCompact region $ \serialized ->
pure $ map snd $ serializedCompactBlockList serialized
-- We'll find that the first block is full with the first half of the large objects,
-- but then there's one new block for each of the remaining large objects.
putStrLn $ "Number of blocks: " <> show (length blockSizes)
print blockSizes
-- UTIL:
data LargeObject = LargeObject ByteArray#
-- | Create an object larger than the large object threshold
newLargeObject :: IO LargeObject
newLargeObject = IO $ \s ->
case newByteArray# 4000# s of
(# s', arr #) -> case unsafeFreezeByteArray# arr s of
(# s'', frozenArr #) -> (# s'', LargeObject frozenArr #)
```
Running it prints
```
Number of blocks: 256
[1032192,4088,4088,4088,4040,4040, ... 250 more ...]
```
indicating that the compact region consists of 256 blocks, and all but the first are a lot smaller than the desired default block size.
## Expected behavior
Ideally, the reproducer would just allocate two compact region blocks, each one megablock large.
That way, there would be less memory fragmentation, and fewer blocks in the linked list of compact region blocks.
### Proposed Fix
I think the fix for this could just be to delete the special case for objects larger than the large object threshold from the `allocateForCompact` function,
i.e. to delete https://gitlab.haskell.org/ghc/ghc/-/blob/a9ce159ba58ca7e8946b46e19b1361588b677a26/rts/sm/CNF.c#L492-501
No further change would be necessary, the function would still be correct due to the fallback allocation
(which incidentally does take the default compact region block size into account):
https://gitlab.haskell.org/ghc/ghc/-/blob/a9ce159ba58ca7e8946b46e19b1361588b677a26/rts/sm/CNF.c#L529-538
And as far as I can tell, there is no obvious reason why large objects need to be treated specially in the compact region allocator:
As long as there is space in the nursery, a large object would just be allocated there anyway.
## Environment
* GHC version used: 8.8.4https://gitlab.haskell.org/ghc/ghc/-/issues/15653Both `Ptr a` in SerializedCompact are inaccurate because of the `a`2020-07-23T09:32:54ZchessaiBoth `Ptr a` in SerializedCompact are inaccurate because of the `a``SerializedCompactPtr` is defined as:
```hs
data SerializedCompact a = SerializedCompact
{ serializedCompactBlockList :: [(Ptr a, Word)]
, serializedCompactRoot :: Ptr a
}
```
But, these `Ptr a` values are a lie, because they don...`SerializedCompactPtr` is defined as:
```hs
data SerializedCompact a = SerializedCompact
{ serializedCompactBlockList :: [(Ptr a, Word)]
, serializedCompactRoot :: Ptr a
}
```
But, these `Ptr a` values are a lie, because they don't point to something of type 'a', which makes the documentation for `ghc-compact` sort of confusing to look at. A more accurate type would just be `Addr`.
The consequences of this being changes to `Addr` are
1: breaking API changes (though not many people use compact regions)
2: A dependency on primitive would be necessary, though I'm again unsure how big of a deal this is, given that ghc-compact already depends on bytestring. (`Addr` should probably be moved to base, and re-exported from primitive, which would avoid this issue.)
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------------ |
| Version | 8.4.3 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | libraries/compact |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | andrewthad, ezyang |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Both `Ptr a` in SerializedCompact are inaccurate because of the `a`","status":"New","operating_system":"","component":"libraries/compact","related":[],"milestone":"8.8.1","resolution":"Unresolved","owner":{"tag":"OwnedBy","contents":"ezyang"},"version":"8.4.3","keywords":["compact","ghc-compact,","regions"],"differentials":[],"test_case":"","architecture":"","cc":["andrewthad","ezyang"],"type":"Bug","description":"{{{SerializedCompactPtr}}} is defined as:\r\n\r\n{{{#!hs\r\ndata SerializedCompact a = SerializedCompact\r\n { serializedCompactBlockList :: [(Ptr a, Word)]\r\n , serializedCompactRoot :: Ptr a\r\n }\r\n}}}\r\n\r\nBut, these {{{Ptr a}}} values are a lie, because they don't point to something of type 'a', which makes the documentation for {{{ghc-compact}}} sort of confusing to look at. A more accurate type would just be {{{Addr}}}.\r\n\r\nThe consequences of this being changes to {{{Addr}}} are\r\n1: breaking API changes (though not many people use compact regions)\r\n2: A dependency on primitive would be necessary, though I'm again unsure how big of a deal this is, given that ghc-compact already depends on bytestring. ({{{Addr}}} should probably be moved to base, and re-exported from primitive, which would avoid this issue.)","type_of_failure":"OtherFailure","blocking":[]} -->9.0.1Edward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/16992compactFixupPointers# fails if compact region blocks are more than 2 GiB apart2020-07-08T17:18:09ZFabian ThorandcompactFixupPointers# fails if compact region blocks are more than 2 GiB apart## Summary
Importing a compact region using `importCompact` from the `ghc-compact` package fails if the addresses of the compact region blocks span across more than 2 GiB.
## Steps to reproduce
The smallest Haskell example for reprodu...## Summary
Importing a compact region using `importCompact` from the `ghc-compact` package fails if the addresses of the compact region blocks span across more than 2 GiB.
## Steps to reproduce
The smallest Haskell example for reproducing the bug I could make is the following:
```haskell
import qualified Data.Compact as Compact
import qualified Data.Compact.Serialize as CompactSerialize
-- | Minimal test case for reproducing compactFixupPointers# bug for large compact regions.
main :: IO ()
main = do
let
large = 1024 * 1024 * 128
largeString = replicate large 'A'
region <- Compact.compact largeString
Compact.compactSize region >>= print
CompactSerialize.writeCompact "/tmp/bug.compact" region
deserialized <- CompactSerialize.unsafeReadCompact "/tmp/bug.compact"
case deserialized of
Left err -> putStrLn err
Right largeString' -> print (Compact.getCompact largeString' == largeString)
```
It uses the `compact` library, but I could also reproduce it with a hand-rolled deserializer calling `GHC.Compact.Serialized.importCompact` directly.
Compiling and running that file results in `Failed to import compact` being printed to stdout.
I traced that error message (which comes from the `compact` library) back to a call to `compactFixupPointers#` in the `ghc-compact` library.
I had a quick look at the corresponding source code of the RTS (in `rts/sm/CNF.c`) and I suspect a faulty order predicate passed to `qsort` in `build_fixup_table` is the root cause of the issue. It is implemented as
```c
static int
cmp_fixup_table_item (const void *e1, const void *e2)
{
const StgWord *w1 = e1;
const StgWord *w2 = e2;
return *w1 - *w2;
}
```
However, on a 64 bit system, `sizeof(int)` is still just 4 bytes while the difference of two `StgWord`s is 8 bytes in size.
If the two pointers being compared are too far apart, an overflow causes the returned order to be the reverse of the actual order.
The following self-contained C program demonstrates this behavior (if compiled on a 64 bit architecture):
```c
#include <stdio.h>
#include <stdint.h>
typedef uint64_t StgWord;
static int
cmp_fixup_table_item (const void *e1, const void *e2)
{
const StgWord *w1 = e1;
const StgWord *w2 = e2;
return *w1 - *w2;
}
static void
check (const StgWord *e1, const StgWord *e2)
{
printf("*e1 = %ld *e2=%ld\n", *e1, *e2);
printf("*e1 < *e2 == %d\n", (int)(*e1 < *e2));
printf("cmp_fixup_table_item(e1,e2) == %d\n\n", cmp_fixup_table_item(e1, e2));
}
int main() {
StgWord examplePointer1 = 1234567;
StgWord examplePointer2 = examplePointer1 + 1234;
StgWord bigOffset = (1L << 31) + 1234L;
StgWord examplePointer3 = examplePointer1 + bigOffset;
check(&examplePointer1, &examplePointer2);
check(&examplePointer1, &examplePointer3);
return 0;
}
```
The output is
```
*e1 = 1234567 *e2=1235801
*e1 < *e2 == 1
cmp_fixup_table_item(e1,e2) == -1234
*e1 = 1234567 *e2=2148719449
*e1 < *e2 == 1
cmp_fixup_table_item(e1,e2) == 2147482414
```
Note how for pointers that are close to each other, it returns a negative number (as it should),
but if the pointers are too far apart, the sign switches.
## Expected behavior
The Haskell program above should print `True`, not an error message.
## Environment
* GHC version used: 8.6.3
Optional:
* Operating System: Ubuntu 18.04
* System Architecture: x86-648.10.2Ömer Sinan AğacanÖmer Sinan Ağacanhttps://gitlab.haskell.org/ghc/ghc/-/issues/17937Runtime panic in CNF test 'compact_gc' when compacting GC is enabled2020-04-09T20:50:31ZÖmer Sinan AğacanRuntime panic in CNF test 'compact_gc' when compacting GC is enabledProgram:
```haskell
import Control.Monad
import GHC.Compact
import qualified Data.Map as Map
main = do
let m = Map.fromList [(x, show x) | x <- [1 .. (10000 :: Int)]]
c <- compactWithSharing m
print =<< compactSize c
c <- fol...Program:
```haskell
import Control.Monad
import GHC.Compact
import qualified Data.Map as Map
main = do
let m = Map.fromList [(x, show x) | x <- [1 .. (10000 :: Int)]]
c <- compactWithSharing m
print =<< compactSize c
c <- foldM (\c _ -> do c <- compactWithSharing (getCompact c)
print =<< compactSize c
return c)
c [1..10]
print (length (show (getCompact c)))
print =<< compactSize c
```
Run with `+RTS -c`:
```
$ ghc-stage2 test.hs -rtsopts
Linking test ...
$ ./test +RTS -c
test: internal error: scavenge_mark_stack: unimplemented/strange closure type 0 @ 0x420012d5c8
(GHC version 8.11.0.20200319 for x86_64_unknown_linux)
Please report this as a GHC bug: https://www.haskell.org/ghc/reportabug
zsh: abort (core dumped) ./test +RTS -c
```
Backtrace:
```
>>> bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00007f91e4de7801 in __GI_abort () at abort.c:79
#2 0x0000000000661757 in rtsFatalInternalErrorFn (s=0x23a8cd "invalid closure, info=%p", ap=0x7fffd25b8840) at rts/RtsMessages.c:186
#3 0x000000000066130e in barf (s=0x23a8cd "invalid closure, info=%p") at rts/RtsMessages.c:48
#4 0x000000000069fd5e in evacuate (p=0x7fffd25b89d0) at rts/sm/Evac.c:592
#5 0x000000000069550a in evacuate_hash_entry (dat=0x7fffd25b8a60, key=283469929872, value=0x42001e5c71) at rts/sm/Scav.c:172
#6 0x00000000006771a6 in mapHashTable (table=0x12b6810, data=0x7fffd25b8a60, fn=0x6954d3 <evacuate_hash_entry>) at rts/Hash.c:458
#7 0x00000000006955b2 in scavenge_compact (str=0x42001f5018) at rts/sm/Scav.c:192
#8 0x0000000000697a7b in scavenge_one (p=0x42001f5018) at rts/sm/Scav.c:1561
#9 0x00000000006983e8 in scavenge_large (ws=0x827d80 <the_gc_thread+320>) at rts/sm/Scav.c:2016
#10 0x0000000000698556 in scavenge_find_work () at rts/sm/Scav.c:2086
#11 0x000000000069861c in scavenge_loop () at rts/sm/Scav.c:2155
#12 0x0000000000691fe4 in scavenge_until_all_done () at rts/sm/GC.c:1175
#13 0x0000000000690990 in GarbageCollect (collect_gen=1, do_heap_census=false, deadlock_detect=false, gc_type=0, cap=0x826500 <MainCapability>, idle_cap=0x0) at rts/sm/GC.c:449
#14 0x0000000000668cbf in scheduleDoGC (pcap=0x7fffd25b8f78, task=0x1217db0, force_major=false, deadlock_detect=false) at rts/Schedule.c:1855
#15 0x00000000006681e6 in schedule (initialCapability=0x826500 <MainCapability>, task=0x1217db0) at rts/Schedule.c:565
#16 0x00000000006696b7 in scheduleWaitThread (tso=0x4200105388, ret=0x0, pcap=0x7fffd25b9080) at rts/Schedule.c:2600
#17 0x000000000067ad18 in rts_evalLazyIO (cap=0x7fffd25b9080, p=0x6cb338, ret=0x0) at rts/RtsAPI.c:530
#18 0x000000000067b4bd in hs_main (argc=1, argv=0x7fffd25b9278, main_closure=0x6cb338, rts_config=...) at rts/RtsMain.c:72
#19 0x0000000000254034 in main ()
```
This happens pretty reliably, every run.
If I also pass `-DS` I get the same panic. Assuming `-DS` checks compact regions
correctly, this means that the bug caught in the same GC that breaks the
invariants/data structures or causes heap corruption.Ömer Sinan AğacanÖmer Sinan Ağacanhttps://gitlab.haskell.org/ghc/ghc/-/issues/17297CNF.c:insertCompactHash doesn't correctly dirty regions2019-11-03T20:24:49ZBen GamariCNF.c:insertCompactHash doesn't correctly dirty regionsConsider `CNF.c:insertCompactHash`:
```c
void
insertCompactHash (Capability *cap,
StgCompactNFData *str,
StgClosure *p, StgClosure *to)
{
insertHashTable(str->hash, (StgWord)p, (const void*)to);
...Consider `CNF.c:insertCompactHash`:
```c
void
insertCompactHash (Capability *cap,
StgCompactNFData *str,
StgClosure *p, StgClosure *to)
{
insertHashTable(str->hash, (StgWord)p, (const void*)to);
const StgInfoTable *strinfo = &str->header.info;
if (strinfo == &stg_COMPACT_NFDATA_CLEAN_info) {
strinfo = &stg_COMPACT_NFDATA_DIRTY_info;
recordClosureMutated(cap, (StgClosure*)str);
}
}
```
At first glance this looks reasonable. However, note how we are dirtying the region:
```
strinfo = &stg_COMPACT_NFDATA_DIRTY_info;
```
This does not do at all what we want; it simply sets the local variable, not the info table pointer.
For this reason compact regions with sharing preservation enabled get added to the `mut_list` once for every object in the region. Terrible!8.8.2Ben GamariBen Gamarihttps://gitlab.haskell.org/ghc/ghc/-/issues/17044Putting large objects in compact regions causes a segfault under the right co...2019-11-03T20:24:49ZFabian ThorandPutting large objects in compact regions causes a segfault under the right conditions## Summary
When putting a large object in a compact region, and then several smaller ones afterwards,
the program crashes with a segfault because the GC tries to follow an invalid pointer.
## Steps to reproduce
The following Ha...## Summary
When putting a large object in a compact region, and then several smaller ones afterwards,
the program crashes with a segfault because the GC tries to follow an invalid pointer.
## Steps to reproduce
The following Haskell program reliably causes a segfault for me:
```haskell
import Data.Traversable (for)
import Data.Primitive.ByteArray
import GHC.Compact
import System.Mem (performGC)
main :: IO ()
main = do
-- Make a fresh compact region. So far, it only contains a single almost empty small block
c <- compact ()
print =<< compactSize c -- prints 32768, as the default block size is 32K
-- Now add a big object to the compact region. The size is chosen so that the compact region
-- allocator will allocate two megablocks while actually leaving space in the first megablock.
let
totalSizeInWords = 129016
arraySizeInWords = totalSizeInWords - 2 {- header words -}
big <- newByteArray (arraySizeInWords * 8)
bigFrozen <- unsafeFreezeByteArray big
-- Add the big object to the compact region.
-- It is larger than the existing block, and also larger than the big object threshold.
-- The `ByteArray` wrapper is likely put still into the first block,
-- but the `ByteArray#` is put in a new block.
-- One can fit 129024 words into a megablock. Our object is 129016 words in size,
-- but the allocator reserves a bit too much space, so it needs to allocate 129025 words,
-- and therefore two megablocks.
_ <- compactAdd c bigFrozen
print =<< compactSize c -- prints 2113536, because it allocated a megablock group of 2 megablocks
-- Now we have to fill the first block, which is currently the nursery,
-- and eventually, one of the Ints will be added to the second block,
-- causing the segfault.
placeholders <- for [0 :: Int ..10000] $ \i -> do
r <- getCompact <$> compactAdd c i
print i
performGC
pure r
-- Keep the placeholders alive so the GC actually has a pointer to follow into the invalid
-- parts of the compact region.
print $ length placeholders
```
I dug a bit into the RTS source code, but I don't yet fully understand what is going on.
The segfault above is the result of an invariant about megablock groups that is violated by `allocateForCompact`.
In the example program, when we start adding the `Int`s in a loop, it will first allocate them from the nursery,
which at that point is still the first block of the compact region.
I think that those allocations happen in the `ALLOCATE` macro (https://gitlab.haskell.org/ghc/ghc/blob/ghc-8.6.3-release/rts/Compact.cmm#L25).
Then, once the nursery is full, `allocateForCompact` is called (https://gitlab.haskell.org/ghc/ghc/blob/ghc-8.6.3-release/rts/Compact.cmm#L29).
There, it first does the same check on the nursery as the `ALLOCATE` macro (https://gitlab.haskell.org/ghc/ghc/blob/ghc-8.6.3-release/rts/sm/CNF.c#L479), which obviously fails.
Then it proceeds, and it will find that the `Int` is not a large object, therefore skipping the special logic there (https://gitlab.haskell.org/ghc/ghc/blob/ghc-8.6.3-release/rts/sm/CNF.c#L490).
However, then it will find that the nursery is full (https://gitlab.haskell.org/ghc/ghc/blob/ghc-8.6.3-release/rts/sm/CNF.c#L501) and pick the next block that is not full as new nursery.
In the example above, there is a new block: the one that was allocated for the large array. It consists of 2 megablocks,
but it's `free` pointer still points within the first megablock, and there's enough room, so it picks that block.
But once that block has been chosen as a nursery, it will start allocating closures even in the second megablock of that group. If I understand the comment in BlockAlloc.c (https://gitlab.haskell.org/ghc/ghc/blob/ghc-8.6.3-release/rts/sm/BlockAlloc.c#L120) correctly, this violates the invariants of megablock groups, as closures that are allocated there don't have a valid `bdescr` of the block they've been placed into.
The remaining question is: why is there still space left in the first megablock, when the allocator actually allocated two megablocks?
I think this is because the size computation for large objects at https://gitlab.haskell.org/ghc/ghc/blob/ghc-8.6.3-release/rts/sm/CNF.c#L491.
It reserves room for an `StgCompactNFData` while only space for a `StgCompactNFDataBlock` is needed.
I tried to fix this by changing it to not picking megablock groups as nursery,
and that indeed fixed the issue above, but then we got other segfaults that
I wasn't able to reliably reproduce yet, as they occurred as part of a large application.
It could be that the loop that tries to fit the allocation in subsequent blocks (https://gitlab.haskell.org/ghc/ghc/blob/ghc-8.6.3-release/rts/sm/CNF.c#L517) is at fault, as it would still put more than one object into a megablock allocation, while https://gitlab.haskell.org/ghc/ghc/blob/ghc-8.6.3-release/rts/sm/BlockAlloc.c#L117 seems to imply that only a single object should be put in those.
Additionally, the `bdescr` initialization in `compactAllocateBlockInternal` (https://gitlab.haskell.org/ghc/ghc/blob/ghc-8.6.3-release/rts/sm/CNF.c#L247) seems to write into the data area of the allocation when the allocated area is larger than one megablock.
## Expected behavior
I expect the program to print the sizes of the compact regions and then all numbers from 0 to 10000,
and not segfault after printing number 2046.
## Environment
* GHC version used: 8.6.3
Optional:
* Operating System: Ubuntu 18.04
* System Architecture: x86_648.8.2https://gitlab.haskell.org/ghc/ghc/-/issues/11493Merge compact normal forms2019-07-07T18:30:11ZBen GamariMerge compact normal formsMerge Yang and Campagna's [http://ezyang.com/compact.html](Compact Normal Forms work).
There are several pieces to this,
- Types and construction logic for compact regions (see [D1264](https://phabricator.haskell.org/D1264))
- Serializ...Merge Yang and Campagna's [http://ezyang.com/compact.html](Compact Normal Forms work).
There are several pieces to this,
- Types and construction logic for compact regions (see [D1264](https://phabricator.haskell.org/D1264))
- Serialization and fixup logic (currently also in [D1264](https://phabricator.haskell.org/D1264))
- Striped allocator for heap sharing between processes (allocator in [D1434](https://phabricator.haskell.org/D1434), CNF support in [D1435](https://phabricator.haskell.org/D1435)).
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ----------------- |
| Version | 7.10.3 |
| Type | Task |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | libraries/compact |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Merge compact normal forms","status":"New","operating_system":"","component":"libraries/compact","related":[],"milestone":"8.2.1","resolution":"Unresolved","owner":{"tag":"OwnedBy","contents":"ezyang"},"version":"7.10.3","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Task","description":"Merge Yang and Campagna's [[Compact Normal Forms work|http://ezyang.com/compact.html]].\r\n\r\nThere are several pieces to this,\r\n\r\n * Types and construction logic for compact regions (see Phab:D1264)\r\n * Serialization and fixup logic (currently also in Phab:D1264)\r\n * Striped allocator for heap sharing between processes (allocator in Phab:D1434, CNF support in Phab:D1435).\r\n","type_of_failure":"OtherFailure","blocking":[]} -->8.2.1Edward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/13298Compact API design improvements2019-07-07T18:22:33ZEdward Z. YangCompact API design improvementsI took a look at the compact API today and I realized that there are a lot of improvements we should make to it:
1. `fmap getCompact . compact` is a pretty common thing to do, if you don't actually care about the compact pointer. We sho...I took a look at the compact API today and I realized that there are a lot of improvements we should make to it:
1. `fmap getCompact . compact` is a pretty common thing to do, if you don't actually care about the compact pointer. We should make a helper function for this.
1. The `SerializedCompact` data structure needs a bunch of instances. Especially, a user who wants to serialize the compact region needs to save this metadata somewhere, but we don't offer any help for doing this.
1. `importCompact` will always print a message to stderr saying pointer fixup is happening. Need to be able to suppress this message.
1. The serialization API is really unsafe; we should make it more safe by default by including some sort of fingerprint, at least.
1. There should be a convenience function for serializing to and from a file, and serializing to and from a handle. RDMA is always going to be delicate business (so keep the old API around) but for these cases we should pave the cowpaths.
Here is some sample code that might help:
```
{-# LANGUAGE ScopedTypeVariables #-}
import System.Environment (getArgs)
import qualified Data.Set as Set
import System.IO
import Data.Compact
import Data.Compact.Serialized
import Foreign.Ptr
import Foreign.Storable
import Foreign.Marshal.Alloc
import Control.Monad
main = do
[dict_file, out_file] <- getArgs
dict <- readFileLatin1 dict_file
c <- compact (Set.fromList (words dict))
withBinaryFile out_file WriteMode $ \h ->
withSerializedCompact c $ \sc -> do
-- Write out the metadata header
hPutStorable h (serializedCompactRoot sc)
forM_ (serializedCompactBlockList sc) $ \(ptr, l) -> do
hPutStorable h ptr
hPutStorable h l
hPutStorable h nullPtr
-- Write out the payload
forM_ (serializedCompactBlockList sc) $ \(ptr, l) ->
hPutBuf h ptr (fromIntegral l)
mb_r <- withBinaryFile out_file ReadMode $ \h -> do
-- Read out the metadata header
root <- hGetStorable h
let go h xs = do
ptr <- hGetStorable h
if ptr == nullPtr
then return (reverse xs)
else do l <- hGetStorable h
go h ((ptr, l):xs)
blocks <- go h []
let sc = SerializedCompact {
serializedCompactBlockList = blocks,
serializedCompactRoot = root
}
-- Read the payload into memory
importCompact sc $ \ptr l -> void $ hGetBuf h ptr (fromIntegral l)
print (fmap getCompact mb_r == Just (getCompact c))
hPutStorable :: forall a. Storable a => Handle -> a -> IO ()
hPutStorable h a =
alloca $ \ptr -> do
poke ptr a
hPutBuf h ptr (sizeOf (undefined :: a))
hGetStorable :: forall a. Storable a => Handle -> IO a
hGetStorable h =
alloca $ \ptr -> do
hGetBuf h ptr (sizeOf (undefined :: a))
peek ptr
readFileLatin1 f = do
h <- openFile f ReadMode
hSetEncoding h latin1
hGetContents h
```
I'm happy to do these but I want to make sure I'm not stepping on FB's toes, also I don't know if bgamari wants to take patches along these lines so late in the release cycle.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ----------------- |
| Version | 8.0.1 |
| Type | FeatureRequest |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | libraries/compact |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | bgamari, simonmar |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Compact API design improvements","status":"New","operating_system":"","component":"libraries/compact","related":[],"milestone":"8.2.1","resolution":"Unresolved","owner":{"tag":"OwnedBy","contents":"ezyang"},"version":"8.0.1","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":["bgamari","simonmar"],"type":"FeatureRequest","description":"I took a look at the compact API today and I realized that there are a lot of improvements we should make to it:\r\n\r\n1. `fmap getCompact . compact` is a pretty common thing to do, if you don't actually care about the compact pointer. We should make a helper function for this.\r\n\r\n2. The `SerializedCompact` data structure needs a bunch of instances. Especially, a user who wants to serialize the compact region needs to save this metadata somewhere, but we don't offer any help for doing this.\r\n\r\n3. `importCompact` will always print a message to stderr saying pointer fixup is happening. Need to be able to suppress this message.\r\n\r\n4. The serialization API is really unsafe; we should make it more safe by default by including some sort of fingerprint, at least.\r\n\r\n5. There should be a convenience function for serializing to and from a file, and serializing to and from a handle. RDMA is always going to be delicate business (so keep the old API around) but for these cases we should pave the cowpaths.\r\n\r\nHere is some sample code that might help:\r\n\r\n{{{\r\n{-# LANGUAGE ScopedTypeVariables #-}\r\nimport System.Environment (getArgs)\r\nimport qualified Data.Set as Set\r\nimport System.IO\r\nimport Data.Compact\r\nimport Data.Compact.Serialized\r\nimport Foreign.Ptr\r\nimport Foreign.Storable\r\nimport Foreign.Marshal.Alloc\r\nimport Control.Monad\r\n\r\nmain = do\r\n [dict_file, out_file] <- getArgs\r\n dict <- readFileLatin1 dict_file\r\n c <- compact (Set.fromList (words dict))\r\n withBinaryFile out_file WriteMode $ \\h ->\r\n withSerializedCompact c $ \\sc -> do\r\n -- Write out the metadata header\r\n hPutStorable h (serializedCompactRoot sc)\r\n forM_ (serializedCompactBlockList sc) $ \\(ptr, l) -> do\r\n hPutStorable h ptr\r\n hPutStorable h l\r\n hPutStorable h nullPtr\r\n -- Write out the payload\r\n forM_ (serializedCompactBlockList sc) $ \\(ptr, l) ->\r\n hPutBuf h ptr (fromIntegral l)\r\n\r\n mb_r <- withBinaryFile out_file ReadMode $ \\h -> do\r\n -- Read out the metadata header\r\n root <- hGetStorable h\r\n let go h xs = do\r\n ptr <- hGetStorable h\r\n if ptr == nullPtr\r\n then return (reverse xs)\r\n else do l <- hGetStorable h\r\n go h ((ptr, l):xs)\r\n blocks <- go h []\r\n let sc = SerializedCompact {\r\n serializedCompactBlockList = blocks,\r\n serializedCompactRoot = root\r\n }\r\n -- Read the payload into memory\r\n importCompact sc $ \\ptr l -> void $ hGetBuf h ptr (fromIntegral l)\r\n\r\n print (fmap getCompact mb_r == Just (getCompact c))\r\n\r\nhPutStorable :: forall a. Storable a => Handle -> a -> IO ()\r\nhPutStorable h a =\r\n alloca $ \\ptr -> do\r\n poke ptr a\r\n hPutBuf h ptr (sizeOf (undefined :: a))\r\n\r\nhGetStorable :: forall a. Storable a => Handle -> IO a\r\nhGetStorable h =\r\n alloca $ \\ptr -> do\r\n hGetBuf h ptr (sizeOf (undefined :: a))\r\n peek ptr\r\n\r\nreadFileLatin1 f = do\r\n h <- openFile f ReadMode\r\n hSetEncoding h latin1\r\n hGetContents h\r\n}}}\r\n\r\nI'm happy to do these but I want to make sure I'm not stepping on FB's toes, also I don't know if bgamari wants to take patches along these lines so late in the release cycle.","type_of_failure":"OtherFailure","blocking":[]} -->8.2.1Edward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/13508Clarify Some Restrictions on Compact Regions2019-07-07T18:21:34ZAndrew MartinClarify Some Restrictions on Compact RegionsI've been reading over the docs for using compact regions here: [https://phabricator-files.haskell.org/file/data/uqmgx4accjldseyd34xj/PHID-FILE-3sihhhdhl4gaeszvphzb/Compact.hs](https://phabricator-files.haskell.org/file/data/uqmgx4accjld...I've been reading over the docs for using compact regions here: [https://phabricator-files.haskell.org/file/data/uqmgx4accjldseyd34xj/PHID-FILE-3sihhhdhl4gaeszvphzb/Compact.hs](https://phabricator-files.haskell.org/file/data/uqmgx4accjldseyd34xj/PHID-FILE-3sihhhdhl4gaeszvphzb/Compact.hs). I'm pretty excited about this feature, but there's one thing that the docs don't make totally clear. Can MutableByteArray\# be placed in a compact region? Near the top, the docs specifically say that "object\[s\] with mutable pointer fields" cannot be compacted, but that doesn't rule out `MutableByteArray#`. Later down, in the docs for `compact`, this restriction is broadened to any mutable data.
I would like to see the docs clarify this better. The existence of `resizeMutableByteArray#` makes me think that it should not possible, but I'd like to be sure. Thanks.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ----------------- |
| Version | 8.1 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | lowest |
| Resolution | Unresolved |
| Component | libraries/compact |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Clarify Some Restrictions on Compact Regions","status":"New","operating_system":"","component":"libraries/compact","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"OwnedBy","contents":"ezyang"},"version":"8.1","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"I've been reading over the docs for using compact regions here: [https://phabricator-files.haskell.org/file/data/uqmgx4accjldseyd34xj/PHID-FILE-3sihhhdhl4gaeszvphzb/Compact.hs]. I'm pretty excited about this feature, but there's one thing that the docs don't make totally clear. Can MutableByteArray# be placed in a compact region? Near the top, the docs specifically say that \"object[s] with mutable pointer fields\" cannot be compacted, but that doesn't rule out `MutableByteArray#`. Later down, in the docs for `compact`, this restriction is broadened to any mutable data.\r\n\r\nI would like to see the docs clarify this better. The existence of `resizeMutableByteArray#` makes me think that it should not possible, but I'd like to be sure. Thanks.","type_of_failure":"OtherFailure","blocking":[]} -->8.2.1Edward Z. YangEdward Z. Yang