GHC issueshttps://gitlab.haskell.org/ghc/ghc/-/issues2019-07-07T18:42:29Zhttps://gitlab.haskell.org/ghc/ghc/-/issues/8972Investigate adding fast compare-and-swap Int type/primops2019-07-07T18:42:29ZtibbeInvestigate adding fast compare-and-swap Int type/primopsI've received reports that using `IORef Int` and `atomicModifyIORef` to implement an atomic counter in the ekg package has become a bottleneck for some of its users. These users update the counter thousands of times per second, using mul...I've received reports that using `IORef Int` and `atomicModifyIORef` to implement an atomic counter in the ekg package has become a bottleneck for some of its users. These users update the counter thousands of times per second, using multiple threads.
I will investigate whether adding a dedicated atomic `Int` reference type will offer significant speed improvements. Such a type can also be used to implement cheaper locks (by using bits in the int to represent different lock states, such as reader/write locks.)
Lets call this new type `AtomicIntRef` for now. This new type needs to support at least these functions:
```haskell
add :: AtomicIntRef -> Int -> IO Int
set :: AtomicIntRef -> Int -> IO ()
get :: AtomicIntRef -> IO Int
```
`add` would be implemented using the `lock` and `xaddq` instructions. `set` and `get` are just simple loads and stores on x86, as these are atomic.
We might also want to consider having other functions, such as a `cas`. Furthermore, there are subtleties with memory barriers that might motivate having barrier/barrier-less versions of some functions.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.9 |
| Type | FeatureRequest |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Investigate adding fast compare-and-swap Int type/primops","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"OwnedBy","contents":"tibbe"},"version":"7.9","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"FeatureRequest","description":"I've received reports that using `IORef Int` and `atomicModifyIORef` to implement an atomic counter in the ekg package has become a bottleneck for some of its users. These users update the counter thousands of times per second, using multiple threads.\r\n\r\nI will investigate whether adding a dedicated atomic `Int` reference type will offer significant speed improvements. Such a type can also be used to implement cheaper locks (by using bits in the int to represent different lock states, such as reader/write locks.)\r\n\r\nLets call this new type `AtomicIntRef` for now. This new type needs to support at least these functions:\r\n\r\n{{{#!haskell\r\nadd :: AtomicIntRef -> Int -> IO Int\r\nset :: AtomicIntRef -> Int -> IO ()\r\nget :: AtomicIntRef -> IO Int\r\n}}}\r\n\r\n`add` would be implemented using the `lock` and `xaddq` instructions. `set` and `get` are just simple loads and stores on x86, as these are atomic.\r\n\r\nWe might also want to consider having other functions, such as a `cas`. Furthermore, there are subtleties with memory barriers that might motivate having barrier/barrier-less versions of some functions.\r\n","type_of_failure":"OtherFailure","blocking":[]} -->7.10.1tibbetibbehttps://gitlab.haskell.org/ghc/ghc/-/issues/8923Add SmallArray# type2022-05-09T18:05:36ZtibbeAdd SmallArray# typeAdd a `SmallArray#` (and `SmallMutableArray#`) type that doesn't have a card table. This would
- save some memory (two words per array),
- make updates cheaper (no writing to the card table), and
- make GC faster (no traversing the card...Add a `SmallArray#` (and `SmallMutableArray#`) type that doesn't have a card table. This would
- save some memory (two words per array),
- make updates cheaper (no writing to the card table), and
- make GC faster (no traversing the card table).
The use case is the unordered-containers package, which uses small arrays to implement a hash array mapped trie.
A starting point would be \[changeset:0417404f5d1230c9d291ea9f73e2831121c8ec99/ghc\], which added the card table to the current array type.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.9 |
| Type | FeatureRequest |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Add SmallArray# type","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"OwnedBy","contents":"tibbe"},"version":"7.9","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"FeatureRequest","description":"Add a `SmallArray#` (and `SmallMutableArray#`) type that doesn't have a card table. This would\r\n\r\n * save some memory (two words per array),\r\n * make updates cheaper (no writing to the card table), and\r\n * make GC faster (no traversing the card table).\r\n\r\nThe use case is the unordered-containers package, which uses small arrays to implement a hash array mapped trie.\r\n\r\nA starting point would be [changeset:0417404f5d1230c9d291ea9f73e2831121c8ec99/ghc], which added the card table to the current array type.","type_of_failure":"OtherFailure","blocking":[]} -->7.10.1tibbetibbehttps://gitlab.haskell.org/ghc/ghc/-/issues/7460Double literals generated bad core2019-07-07T18:49:38ZtibbeDouble literals generated bad coreThe following code results in core containing expression like `doubleFromInteger (wordToInteger sc_s2pw)`:
```
{-# LANGUAGE CPP, BangPatterns #-}
module Main (main) where
#define VDIM 100
#define VNUM 100000
import Data.Array.Base
imp...The following code results in core containing expression like `doubleFromInteger (wordToInteger sc_s2pw)`:
```
{-# LANGUAGE CPP, BangPatterns #-}
module Main (main) where
#define VDIM 100
#define VNUM 100000
import Data.Array.Base
import Data.Array.ST
import Data.Array.Unboxed
import Control.Monad.ST
import GHC.Word
import Control.Monad
import Data.Bits
prng :: Word -> Word
prng w = w'
where
!w1 = w `xor` (w `shiftL` 13)
!w2 = w1 `xor` (w1 `shiftR` 7)
!w' = w2 `xor` (w2 `shiftL` 17)
type Vec s = STUArray s Int Double
kahan :: Vec s -> Vec s -> ST s ()
kahan s c = do
let inner w j
| j < VDIM = do
!cj <- unsafeRead c j
!sj <- unsafeRead s j
let !y = fromIntegral w - cj
!t = sj + y
!w' = prng w
unsafeWrite c j ((t-sj)-y)
unsafeWrite s j t
inner w' (j+1)
| otherwise = return ()
outer i | i < VNUM = inner i 0 >> outer (i + 1)
| otherwise = return ()
outer 0
calc :: ST s (Vec s)
calc = do
s <- newArray (0,VDIM-1) 0
c <- newArray (0,VDIM-1) 0
kahan s c
return s
main :: IO ()
main = print . elems $ runSTUArray calc
```
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ---------------------- |
| Version | 7.4.2 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | johan.tibell@gmail.com |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Double literals generated bad core","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"7.4.2","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":["johan.tibell@gmail.com"],"type":"Bug","description":"The following code results in core containing expression like `doubleFromInteger (wordToInteger sc_s2pw)`:\r\n\r\n{{{\r\n{-# LANGUAGE CPP, BangPatterns #-}\r\nmodule Main (main) where\r\n\r\n#define VDIM 100\r\n#define VNUM 100000\r\n\r\nimport Data.Array.Base\r\nimport Data.Array.ST\r\nimport Data.Array.Unboxed\r\nimport Control.Monad.ST\r\nimport GHC.Word\r\nimport Control.Monad\r\nimport Data.Bits\r\n\r\nprng :: Word -> Word\r\nprng w = w'\r\n where\r\n !w1 = w `xor` (w `shiftL` 13)\r\n !w2 = w1 `xor` (w1 `shiftR` 7)\r\n !w' = w2 `xor` (w2 `shiftL` 17)\r\n\r\ntype Vec s = STUArray s Int Double\r\n\r\nkahan :: Vec s -> Vec s -> ST s ()\r\nkahan s c = do\r\n let inner w j\r\n | j < VDIM = do\r\n !cj <- unsafeRead c j\r\n !sj <- unsafeRead s j\r\n let !y = fromIntegral w - cj\r\n !t = sj + y\r\n !w' = prng w\r\n unsafeWrite c j ((t-sj)-y)\r\n unsafeWrite s j t\r\n inner w' (j+1)\r\n | otherwise = return ()\r\n outer i | i < VNUM = inner i 0 >> outer (i + 1)\r\n | otherwise = return ()\r\n outer 0\r\n\r\ncalc :: ST s (Vec s)\r\ncalc = do\r\n s <- newArray (0,VDIM-1) 0\r\n c <- newArray (0,VDIM-1) 0\r\n kahan s c\r\n return s\r\n\r\nmain :: IO ()\r\nmain = print . elems $ runSTUArray calc\r\n}}}","type_of_failure":"OtherFailure","blocking":[]} -->7.10.1tibbetibbehttps://gitlab.haskell.org/ghc/ghc/-/issues/5443Errors when shutting down the event manager loop2019-07-07T18:55:14ZbasvandijkErrors when shutting down the event manager loopAs explained in [this thread on the GHC list](http://thread.gmane.org/gmane.comp.lang.haskell.glasgow.user/20573) I use the GHC event manager in my [usb library](https://github.com/basvandijk/usb). I create my own `EventManager` and star...As explained in [this thread on the GHC list](http://thread.gmane.org/gmane.comp.lang.haskell.glasgow.user/20573) I use the GHC event manager in my [usb library](https://github.com/basvandijk/usb). I create my own `EventManager` and start a thread which runs the event manager loop. When the library is finalized I automatically shutdown the loop. However this causes error messages from the RTS to be printed.
The following program shows the problem in isolation:
```
import Control.Concurrent
import GHC.Event
main = do
em <- new
tid <- forkIO $ loop em
threadDelay 2000000
shutdown em -- Note that 'killThread tid' has the same effect.
threadDelay 2000000
```
Make sure to build it with `-threaded` enabled:
`$ ghc -threaded --make eventManagerBug.hs`
Running it gives the following errors:
```
$ ./eventManagerBug
example: ioManagerWakeup: write: Bad file descriptor
example: ioManagerDie: write: Bad file descriptor
```
Note that these errors are printed in the `ioManagerWakeup` function in `rts/posix/Signals.c`.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ---------------------------------------- |
| Version | 7.2.1 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | bos@mailrank.com, johan.tibell@gmail.com |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Errors when shutting down the event manager loop","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"7.2.1","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":["bos@mailrank.com","johan.tibell@gmail.com"],"type":"Bug","description":"As explained in [http://thread.gmane.org/gmane.comp.lang.haskell.glasgow.user/20573 this thread on the GHC list] I use the GHC event manager in my [https://github.com/basvandijk/usb usb library]. I create my own `EventManager` and start a thread which runs the event manager loop. When the library is finalized I automatically shutdown the loop. However this causes error messages from the RTS to be printed.\r\n\r\nThe following program shows the problem in isolation:\r\n \r\n{{{\r\nimport Control.Concurrent\r\nimport GHC.Event\r\n\r\nmain = do\r\n em <- new\r\n tid <- forkIO $ loop em\r\n threadDelay 2000000\r\n shutdown em -- Note that 'killThread tid' has the same effect.\r\n threadDelay 2000000\r\n}}}\r\n\r\nMake sure to build it with `-threaded` enabled:\r\n\r\n`$ ghc -threaded --make eventManagerBug.hs`\r\n\r\nRunning it gives the following errors:\r\n\r\n{{{\r\n$ ./eventManagerBug\r\nexample: ioManagerWakeup: write: Bad file descriptor\r\nexample: ioManagerDie: write: Bad file descriptor\r\n}}}\r\n\r\nNote that these errors are printed in the `ioManagerWakeup` function in `rts/posix/Signals.c`.","type_of_failure":"OtherFailure","blocking":[]} -->7.10.1tibbetibbe