GHC issueshttps://gitlab.haskell.org/ghc/ghc/-/issues2022-01-18T14:24:41Zhttps://gitlab.haskell.org/ghc/ghc/-/issues/15111GHCi leaks the first modules loaded2022-01-18T14:24:41ZSimon MarlowGHCi leaks the first modules loadedHow to reproduce the leak:
```
cd nofib/real/veritas
ghci +RTS -S
Prelude> :load Main
*Main> System.Mem.performGC
10554832 32639712 38824584 0.048 0.058 2.317 8.386 0 0 (Gen: 1)
```
Live data is \~38Mb (3rd number). ...How to reproduce the leak:
```
cd nofib/real/veritas
ghci +RTS -S
Prelude> :load Main
*Main> System.Mem.performGC
10554832 32639712 38824584 0.048 0.058 2.317 8.386 0 0 (Gen: 1)
```
Live data is \~38Mb (3rd number). Now unload everything:
```
*Main> :load
Prelude> System.Mem.performGC
4005376 32681280 38850224 0.013 0.048 2.330 29.896 0 0 (Gen: 1)
```
Note the live data didn't go down.
Load the program again:
```
Prelude> :load Main
...
*Main> System.Mem.performGC
16344112 47790304 54799632 0.070 0.074 4.343 82.235 0 0 (Gen: 1)
```
Note the live memory is almost 2x what it was before.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------ |
| Version | 8.2.2 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"GHCi leaks the first modules loaded","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"8.6.1","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.2.2","keywords":["ghci"],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"How to reproduce the leak:\r\n\r\n{{{\r\ncd nofib/real/veritas\r\nghci +RTS -S\r\nPrelude> :load Main\r\n*Main> System.Mem.performGC\r\n 10554832 32639712 38824584 0.048 0.058 2.317 8.386 0 0 (Gen: 1)\r\n}}}\r\n\r\nLive data is ~38Mb (3rd number). Now unload everything:\r\n\r\n{{{\r\n*Main> :load\r\nPrelude> System.Mem.performGC\r\n 4005376 32681280 38850224 0.013 0.048 2.330 29.896 0 0 (Gen: 1)\r\n}}}\r\n\r\nNote the live data didn't go down.\r\n\r\nLoad the program again:\r\n\r\n{{{\r\nPrelude> :load Main\r\n...\r\n*Main> System.Mem.performGC\r\n 16344112 47790304 54799632 0.070 0.074 4.343 82.235 0 0 (Gen: 1)\r\n}}}\r\n\r\nNote the live memory is almost 2x what it was before.","type_of_failure":"OtherFailure","blocking":[]} -->8.6.1Simon MarlowSimon Marlowhttps://gitlab.haskell.org/ghc/ghc/-/issues/14970GHC.Err.errorWithoutStackTrace produces stack trace when profiling enabled2019-07-07T18:14:51ZrotaerkGHC.Err.errorWithoutStackTrace produces stack trace when profiling enabledWhen you run 'error', it shows the usual stack trace. When you enable profiling with the -prof flag to ghc, and you run 'error', it shows the usual stack trace plus an additional prof-specific stack trace.
When you run 'errorWithoutStac...When you run 'error', it shows the usual stack trace. When you enable profiling with the -prof flag to ghc, and you run 'error', it shows the usual stack trace plus an additional prof-specific stack trace.
When you run 'errorWithoutStackTrace' with profiling enabled, it strips the usual stack trace of 'error', but still displays the prof-specific stack trace.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 8.2.2 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | libraries/base |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"GHC.Err.errorWithoutStackTrace produces stack trace when profiling enabled","status":"New","operating_system":"","component":"libraries/base","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.2.2","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"When you run 'error', it shows the usual stack trace. When you enable profiling with the -prof flag to ghc, and you run 'error', it shows the usual stack trace plus an additional prof-specific stack trace.\r\n\r\nWhen you run 'errorWithoutStackTrace' with profiling enabled, it strips the usual stack trace of 'error', but still displays the prof-specific stack trace.","type_of_failure":"OtherFailure","blocking":[]} -->8.6.1Simon MarlowSimon Marlowhttps://gitlab.haskell.org/ghc/ghc/-/issues/14900Calling isByteArrayPinned# on compact ByteArray2022-12-27T16:11:44ZAndrew MartinCalling isByteArrayPinned# on compact ByteArrayThe documentation explaining the relationship between pinnedness and compact regions is incomplete. From `Data.Compact`:
> Pinned ByteArray\# objects cannot be compacted. This is for a good reason: the memory is pinned so that it can be...The documentation explaining the relationship between pinnedness and compact regions is incomplete. From `Data.Compact`:
> Pinned ByteArray\# objects cannot be compacted. This is for a good reason: the memory is pinned so that it can be referenced by address (the address might be stored in a C data structure, for example), so we can't make a copy of it to store in the Compact.
This is half-way true since it only considers one a the ways in which the GHC runtime tracks the pinnedness of an object. From experimenting with compact regions, it seems like there are two different notions of pinnedness:
1. Did the user explicitly ask for the `ByteArray` to be pinned?
1. Is the `ByteArray` pinned?
If (1) is true, then (2) must always be true, but if (1) is false, then could be true or false. `ByteArray`s over 3KB are pinned, and `ByteArray`s under 3KB are not (or somewhere around that number).
With that background information in place, here's the scenario I've encountered:
```
{-# LANGUAGE MagicHash #-}
import Data.Primitive
import Data.Compact
import GHC.Int
import GHC.Prim
main :: IO ()
main = do
ByteArray arr1# <- fmap getCompact $ newByteArray 65000 >>= unsafeFreezeByteArray >>= compact
ByteArray arr2# <- newByteArray 65000 >>= unsafeFreezeByteArray
print (I# (isByteArrayPinned# arr1#))
print (I# (isByteArrayPinned# arr2#))
putStrLn "Finished"
```
When compiled and run, this gives:
```
0
1
Finished
```
We can see that the 65KiB `ByteArray` that was not compacted let's the user know that it is pinned. The compacted `ByteArray` claims to not be pinned, but this is not true. The docs in `Data.Compact` claim:
> Data in a compact doesn't ever move, so compacting data is also a way to pin arbitrary data structures in memory.
I propose that the behavior of `compact` be modified to accurately convey the pinnedness of the `ByteArray`s that copies. This would mean that even small, previously unpinned, `ByteArray`s would also be designated as pinned. It's a small change, but it makes the information the runtime gives us more accurate. This is occasionally handy when dealing with the FFI.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ---------------- |
| Version | 8.2.2 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | ezyang, simonmar |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Calling isByteArrayPinned# on compact ByteArray","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.2.2","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":["ezyang","simonmar"],"type":"Bug","description":"The documentation explaining the relationship between pinnedness and compact regions is incomplete. From `Data.Compact`:\r\n\r\n> Pinned ByteArray# objects cannot be compacted. This is for a good reason: the memory is pinned so that it can be referenced by address (the address might be stored in a C data structure, for example), so we can't make a copy of it to store in the Compact.\r\n\r\nThis is half-way true since it only considers one a the ways in which the GHC runtime tracks the pinnedness of an object. From experimenting with compact regions, it seems like there are two different notions of pinnedness:\r\n\r\n1. Did the user explicitly ask for the `ByteArray` to be pinned?\r\n2. Is the `ByteArray` pinned?\r\n\r\nIf (1) is true, then (2) must always be true, but if (1) is false, then could be true or false. `ByteArray`s over 3KB are pinned, and `ByteArray`s under 3KB are not (or somewhere around that number).\r\n\r\nWith that background information in place, here's the scenario I've encountered:\r\n\r\n{{{\r\n{-# LANGUAGE MagicHash #-}\r\n\r\nimport Data.Primitive\r\nimport Data.Compact\r\nimport GHC.Int\r\nimport GHC.Prim\r\n\r\nmain :: IO ()\r\nmain = do\r\n ByteArray arr1# <- fmap getCompact $ newByteArray 65000 >>= unsafeFreezeByteArray >>= compact\r\n ByteArray arr2# <- newByteArray 65000 >>= unsafeFreezeByteArray\r\n print (I# (isByteArrayPinned# arr1#))\r\n print (I# (isByteArrayPinned# arr2#))\r\n putStrLn \"Finished\"\r\n}}}\r\n\r\nWhen compiled and run, this gives:\r\n\r\n{{{\r\n0\r\n1\r\nFinished\r\n}}}\r\n\r\nWe can see that the 65KiB `ByteArray` that was not compacted let's the user know that it is pinned. The compacted `ByteArray` claims to not be pinned, but this is not true. The docs in `Data.Compact` claim:\r\n\r\n> Data in a compact doesn't ever move, so compacting data is also a way to pin arbitrary data structures in memory.\r\n\r\nI propose that the behavior of `compact` be modified to accurately convey the pinnedness of the `ByteArray`s that copies. This would mean that even small, previously unpinned, `ByteArray`s would also be designated as pinned. It's a small change, but it makes the information the runtime gives us more accurate. This is occasionally handy when dealing with the FFI.","type_of_failure":"OtherFailure","blocking":[]} -->8.6.1Simon MarlowSimon Marlowhttps://gitlab.haskell.org/ghc/ghc/-/issues/9285IO manager startup procedure somewhat odd2019-07-07T18:41:03Zedsko@edsko.netIO manager startup procedure somewhat oddCurrently the RTS startup procedure for the threaded runtime with a single capability looks something like this:
**1. Capability gets initialized**
```
hs_main
real_main
hs_init_ghc
initScheduler
initCapabilities
``...Currently the RTS startup procedure for the threaded runtime with a single capability looks something like this:
**1. Capability gets initialized**
```
hs_main
real_main
hs_init_ghc
initScheduler
initCapabilities
```
**2. IO Manager is started**
```
hs_main
real_main
hs_init_ghc
ioManagerStart
```
**3. ensureIOManagerIsRunning**
The IO manager of course is mostly written in Haskell, so `ioManagerStart` does
```
cap = rts_lock();
rts_evalIO(&cap,&base_GHCziConcziIO_ensureIOManagerIsRunning_closure,NULL);
rts_unlock(cap);
```
**4. Bound task is created and we start running Haskell code**
Since this is the first call to `rts_lock`, it creates a bound task to correspond to the current OS thread.
**5. Haskell code does a safe FFI call: create worker task 1**
The first safe FFI call that the IO manager makes (ignoring `labelThread`) is to `c_setIOManagerControlFd`. At this point, the bound task gets suspended and we create a worker task.
**6. Worker task 1 starts executing `GHC.Event.Manager.loop`**
The worker tasks gets suspended due to an explicit call to `yield` in `GHC.Event.Manager.step`, at which point the bound task continues execution. It continues and finishes (`ensureIOManagerRunning` completes). The worker task takes over.
**7. Worker task 1 does a safe FFI call to `I.poll`, create new worker task 2**
(This safe call is from `GHC.Event.Manager.step`)
**8. Worker task 2 executes `GHC.Event.TimerManager.loop`, and does safe FFI call to `I.poll`; create worker task 3**
This worker task will remain spare for now.
**9. Start executing the actual main function**
The IO manager calls `rts_unlock`, which verifies if there are any spare workers available; it finds that there are and doesn't start a new one.
Then when we start `main` itself of course we go through this `rts_lock, rts_evalIO, rts_unlock` sequence *again*. This is somewhat confusing, and makes tracing the startup of the code more difficult to interpret. It seems that this could be simpler: only a single `rts_lock` should be necessary, in which we first call `ensureIOManagerIsRunning` and then `main`, followed by a single `rts_unlock`. A similar simplification should be possible in `forkProcess`.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------ |
| Version | 7.8.2 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | low |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"IO manager startup procedure somewhat odd","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"7.8.2","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"Currently the RTS startup procedure for the threaded runtime with a single capability looks something like this:\r\n\r\n'''1. Capability gets initialized'''\r\n\r\n{{{\r\nhs_main\r\n real_main\r\n hs_init_ghc\r\n initScheduler\r\n initCapabilities\r\n}}}\r\n\r\n'''2. IO Manager is started'''\r\n\r\n{{{\r\nhs_main\r\n real_main\r\n hs_init_ghc\r\n ioManagerStart\r\n}}}\r\n\r\n'''3. ensureIOManagerIsRunning'''\r\n\r\nThe IO manager of course is mostly written in Haskell, so `ioManagerStart` does\r\n\r\n{{{\r\n\tcap = rts_lock();\r\n rts_evalIO(&cap,&base_GHCziConcziIO_ensureIOManagerIsRunning_closure,NULL);\r\n\trts_unlock(cap);\r\n}}}\r\n\r\n'''4. Bound task is created and we start running Haskell code'''\r\n\r\nSince this is the first call to `rts_lock`, it creates a bound task to correspond to the current OS thread.\r\n\r\n'''5. Haskell code does a safe FFI call: create worker task 1'''\r\n\r\nThe first safe FFI call that the IO manager makes (ignoring `labelThread`) is to `c_setIOManagerControlFd`. At this point, the bound task gets suspended and we create a worker task. \r\n\r\n'''6. Worker task 1 starts executing `GHC.Event.Manager.loop`'''\r\n\r\nThe worker tasks gets suspended due to an explicit call to `yield` in `GHC.Event.Manager.step`, at which point the bound task continues execution. It continues and finishes (`ensureIOManagerRunning` completes). The worker task takes over.\r\n\r\n'''7. Worker task 1 does a safe FFI call to `I.poll`, create new worker task 2'''\r\n\r\n(This safe call is from `GHC.Event.Manager.step`)\r\n\r\n'''8. Worker task 2 executes `GHC.Event.TimerManager.loop`, and does safe FFI call to `I.poll`; create worker task 3'''\r\n\r\nThis worker task will remain spare for now.\r\n\r\n'''9. Start executing the actual main function'''\r\n\r\nThe IO manager calls `rts_unlock`, which verifies if there are any spare workers available; it finds that there are and doesn't start a new one.\r\n\r\nThen when we start `main` itself of course we go through this `rts_lock, rts_evalIO, rts_unlock` sequence ''again''. This is somewhat confusing, and makes tracing the startup of the code more difficult to interpret. It seems that this could be simpler: only a single `rts_lock` should be necessary, in which we first call `ensureIOManagerIsRunning` and then `main`, followed by a single `rts_unlock`. A similar simplification should be possible in `forkProcess`. ","type_of_failure":"OtherFailure","blocking":[]} -->8.6.1Simon MarlowSimon Marlowhttps://gitlab.haskell.org/ghc/ghc/-/issues/5611Asynchronous exception discarded after safe FFI call2019-07-07T18:54:23ZjoeyadamsAsynchronous exception discarded after safe FFI call**Note:** This bug appears to be fixed already, as it does not appear with GHC 7.2.1 . I'm submitting a bug report anyway, to document its presence.
The bug is: when an asynchronous exception is thrown to a thread making a (safe) foreig...**Note:** This bug appears to be fixed already, as it does not appear with GHC 7.2.1 . I'm submitting a bug report anyway, to document its presence.
The bug is: when an asynchronous exception is thrown to a thread making a (safe) foreign call, the thread throwing the exception blocks like it should, but then the exception isn't actually delivered. For example:
```haskell
{-# LANGUAGE ForeignFunctionInterface #-}
import Control.Concurrent
import Foreign.C
import System.IO
foreign import ccall safe "unistd.h sleep"
sleep :: CUInt -> IO CUInt
main :: IO ()
main = do
hSetBuffering stdout LineBuffering
tid <- forkIO $ do
putStrLn "child: Sleeping"
_ <- sleep 1
-- The following lines should not happen after the killThread from the
-- parent thread completes. However, they do...
putStrLn "child: Done sleeping"
threadDelay 1000000
putStrLn "child: Done waiting"
threadDelay 100000
putStrLn $ "parent: Throwing exception to thread " ++ show tid
throwTo tid $ userError "Exception delivered successfully"
putStrLn "parent: Done throwing exception"
threadDelay 2000000
```
When the bug is present, the program prints:
```
child: Sleeping
parent: Throwing exception to thread ThreadId 4
child: Done sleeping
parent: Done throwing exception
child: Done waiting
```
"child: Done waiting" should not be printed after completion of the throwTo, and the exception message should appear. On GHC 7.2.1, the program prints:
```
child: Sleeping
parent: Throwing exception to thread ThreadId 4
parent: Done throwing exception
ffi-sleep: user error (Exception delivered successfully)
```
This bug has been reproduced in GHC 7.0.3, on both Linux and Windows. It has also been reproduced on GHC 7.0.4.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.0.3 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Runtime System |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Asynchronous exception discarded after safe FFI call","status":"New","operating_system":"","component":"Runtime System","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"7.0.3","keywords":["FFI,","exception"],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"'''Note:''' This bug appears to be fixed already, as it does not appear with GHC 7.2.1 . I'm submitting a bug report anyway, to document its presence.\r\n\r\nThe bug is: when an asynchronous exception is thrown to a thread making a (safe) foreign call, the thread throwing the exception blocks like it should, but then the exception isn't actually delivered. For example:\r\n\r\n{{{\r\n\r\n{-# LANGUAGE ForeignFunctionInterface #-}\r\n\r\nimport Control.Concurrent\r\nimport Foreign.C\r\nimport System.IO\r\n\r\nforeign import ccall safe \"unistd.h sleep\"\r\n sleep :: CUInt -> IO CUInt\r\n\r\nmain :: IO ()\r\nmain = do\r\n hSetBuffering stdout LineBuffering\r\n\r\n tid <- forkIO $ do\r\n putStrLn \"child: Sleeping\"\r\n _ <- sleep 1\r\n\r\n -- The following lines should not happen after the killThread from the\r\n -- parent thread completes. However, they do...\r\n putStrLn \"child: Done sleeping\"\r\n threadDelay 1000000\r\n putStrLn \"child: Done waiting\"\r\n\r\n threadDelay 100000\r\n putStrLn $ \"parent: Throwing exception to thread \" ++ show tid\r\n throwTo tid $ userError \"Exception delivered successfully\"\r\n putStrLn \"parent: Done throwing exception\"\r\n\r\n threadDelay 2000000\r\n}}}\r\n\r\nWhen the bug is present, the program prints:\r\n\r\n{{{\r\nchild: Sleeping\r\nparent: Throwing exception to thread ThreadId 4\r\nchild: Done sleeping\r\nparent: Done throwing exception\r\nchild: Done waiting\r\n}}}\r\n\r\n\"child: Done waiting\" should not be printed after completion of the throwTo, and the exception message should appear. On GHC 7.2.1, the program prints:\r\n\r\n{{{\r\nchild: Sleeping\r\nparent: Throwing exception to thread ThreadId 4\r\nparent: Done throwing exception\r\nffi-sleep: user error (Exception delivered successfully)\r\n}}}\r\n\r\nThis bug has been reproduced in GHC 7.0.3, on both Linux and Windows. It has also been reproduced on GHC 7.0.4.","type_of_failure":"OtherFailure","blocking":[]} -->8.6.1Simon MarlowSimon Marlow