GHC issueshttps://gitlab.haskell.org/ghc/ghc/-/issues2019-07-07T18:03:46Zhttps://gitlab.haskell.org/ghc/ghc/-/issues/15604Profiling RTS flag `-po` does not work2019-07-07T18:03:46ZÖmer Sinan AğacanProfiling RTS flag `-po` does not workThe RTS flag documentation says this about `-po`:
```
empty: -p Time/allocation profile in tree format
empty: (output file <output prefix>.prof)
empty: -po<file> Override profiling output file name prefix (prog...The RTS flag documentation says this about `-po`:
```
empty: -p Time/allocation profile in tree format
empty: (output file <output prefix>.prof)
empty: -po<file> Override profiling output file name prefix (program name by default)
```
However it current doesn't work as advertised in 8.4 and HEAD. When I use `-po<file>` no file is generated. Reproducer:
```
$ cat empty.hs
main = return ()
$ ghc-stage2 empty.hs -fforce-recomp -rtsopts -prof
[1 of 1] Compiling Main ( empty.hs, empty.o )
Linking empty ...
$ ./empty +RTS -poprof
$ ls | grep prof
$
```
Just `-p` works:
```
$ ./empty +RTS -p
$ ls | grep prof
empty.prof
$
```
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------ |
| Version | 8.5 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Profiling |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Profiling RTS flag `-po` does not work","status":"New","operating_system":"","component":"Profiling","related":[],"milestone":"8.6.1","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.5","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"The RTS flag documentation says this about `-po`:\r\n\r\n{{{\r\nempty: -p Time/allocation profile in tree format\r\nempty: (output file <output prefix>.prof)\r\nempty: -po<file> Override profiling output file name prefix (program name by default)\r\n}}}\r\n\r\nHowever it current doesn't work as advertised in 8.4 and HEAD. When I use `-po<file>` no file is generated. Reproducer:\r\n\r\n{{{\r\n$ cat empty.hs\r\nmain = return ()\r\n$ ghc-stage2 empty.hs -fforce-recomp -rtsopts -prof\r\n[1 of 1] Compiling Main ( empty.hs, empty.o )\r\nLinking empty ...\r\n$ ./empty +RTS -poprof\r\n$ ls | grep prof\r\n$\r\n}}}\r\n\r\nJust `-p` works:\r\n\r\n{{{\r\n$ ./empty +RTS -p\r\n$ ls | grep prof\r\nempty.prof\r\n$\r\n}}}","type_of_failure":"OtherFailure","blocking":[]} -->8.6.1https://gitlab.haskell.org/ghc/ghc/-/issues/15602PAP invariant of pointer tagging does not hold in profiling builds2019-07-07T18:03:46ZÖmer Sinan AğacanPAP invariant of pointer tagging does not hold in profiling buildsThe PAP invariant of pointer tagging says
> the PAP entry code jumps to the function's entry code, so it must have a tagged pointer to the function closure in R1. We therefore assume that a PAP always contains a tagged pointer to the fu...The PAP invariant of pointer tagging says
> the PAP entry code jumps to the function's entry code, so it must have a tagged pointer to the function closure in R1. We therefore assume that a PAP always contains a tagged pointer to the function closure.
(from https://ghc.haskell.org/trac/ghc/wiki/Commentary/Rts/HaskellExecution/PointerTagging)
As discovered while debugging #15508, this currently does not hold. I tried to fix this in one PAP allocation site in [D5051](https://phabricator.haskell.org/D5051) but it somehow broke another test. We should review all PAP allocation sites and make sure the invariant holds, and then fix any bugs that this fix reveals.
Relevant commits:
- 6015a94f9108a502150565577b66c23650796639: Implements pointer tagging
- f9c6d53fe997f1c560cda6f346f4b201711df37c: Fixes a PAP allocation site (#13767)
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------ |
| Version | 8.5 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | simonmar |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"PAP invariant of pointer tagging does not hold","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"8.6.1","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.5","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":["simonmar"],"type":"Bug","description":"The PAP invariant of pointer tagging says\r\n\r\n> the PAP entry code jumps to the function's entry code, so it must have a tagged pointer to the function closure in R1. We therefore assume that a PAP always contains a tagged pointer to the function closure.\r\n\r\n(from https://ghc.haskell.org/trac/ghc/wiki/Commentary/Rts/HaskellExecution/PointerTagging)\r\n\r\nAs discovered while debugging #15508, this currently does not hold. I tried to fix this in one PAP allocation site in Phab:D5051 but it somehow broke another test. We should review all PAP allocation sites and make sure the invariant holds, and then fix any bugs that this fix reveals.\r\n\r\nRelevant commits:\r\n\r\n- 6015a94f9108a502150565577b66c23650796639: Implements pointer tagging\r\n- f9c6d53fe997f1c560cda6f346f4b201711df37c: Fixes a PAP allocation site (#13767)","type_of_failure":"OtherFailure","blocking":[]} -->8.6.1https://gitlab.haskell.org/ghc/ghc/-/issues/15108Panic: collectNBinders2023-09-01T12:15:47ZcdisselkoenPanic: collectNBindersGHC 8.4.2 panics with the error message `collectNBinders`, when compiling with profiling. This is a very similar bug to #15002, but although #15002 is fixed in 8.4.2, this panic persists.
Steps to reproduce:
`stack.yaml`:
```
resolver...GHC 8.4.2 panics with the error message `collectNBinders`, when compiling with profiling. This is a very similar bug to #15002, but although #15002 is fixed in 8.4.2, this panic persists.
Steps to reproduce:
`stack.yaml`:
```
resolver: nightly-2018-04-30
```
`bug.cabal`:
```
name: bug
version: 0.1.0.0
build-type: Simple
cabal-version: >=1.10
executable bug
main-is: Main.hs
build-depends: base
default-language: Haskell2010
```
`Main.hs`:
```
module Main where
main :: IO ()
main = do
_ <- return $ getInt Circle
return ()
newtype MyInt = MyInt Int
data Shape = Circle
| Square
getInt :: Shape -> MyInt
getInt sh =
case sh of
Circle ->
let (MyInt i) = MyInt 3
in myInt i
Square ->
let (MyInt i) = MyInt 2
in myInt i
where
myInt = MyInt
```
Then run:
```
$ stack ghc -- --version
The Glorious Glasgow Haskell Compilation System, version 8.4.2
$ stack build --profile
bug-0.1.0.0: configure (exe)
Configuring bug-0.1.0.0...
bug-0.1.0.0: build (exe)
Preprocessing executable 'bug' for bug-0.1.0.0..
Building executable 'bug' for bug-0.1.0.0..
[1 of 1] Compiling Main ( Main.hs, .stack-work/dist/x86_64-linux-tinfo6-nopie/Cabal-2.2.0.1/build/bug/bug-tmp/Main.p_o )
ghc: panic! (the 'impossible' happened)
(GHC version 8.4.2 for x86_64-unknown-linux):
collectNBinders
1
Call stack:
CallStack (from HasCallStack):
callStackDoc, called at compiler/utils/Outputable.hs:1150:37 in ghc:Outputable
pprPanic, called at compiler/coreSyn/CoreSyn.hs:2025:25 in ghc:CoreSyn
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
-- While building custom Setup.hs for package bug-0.1.0.0 using:
/usr/local/home/cdisselk/.stack/setup-exe-cache/x86_64-linux-tinfo6-nopie/Cabal-simple_mPHDZzAJ_2.2.0.1_ghc-8.4.2 --builddir=.stack-work/dist/x86_64-linux-tinfo6-nopie/Cabal-2.2.0.1 build exe:bug --ghc-options " -ddump-hi -ddump-to-file -fdiagnostics-color=always"
Process exited with code: ExitFailure 1
```8.6.1https://gitlab.haskell.org/ghc/ghc/-/issues/15063T3001-2 fails on i386 Linux2019-07-07T18:14:27ZBen GamariT3001-2 fails on i386 LinuxThe `T3001-2` testcase fails on i386/Linux:
```
Wrong exit code for T3001-2(prof_hb)(expected 0 , actual 134 )
Stderr ( T3001-2 ):
T3001-2: internal error: Invalid object in processHeapClosureForDead(): 7
(GHC version 8.5.20180419 f...The `T3001-2` testcase fails on i386/Linux:
```
Wrong exit code for T3001-2(prof_hb)(expected 0 , actual 134 )
Stderr ( T3001-2 ):
T3001-2: internal error: Invalid object in processHeapClosureForDead(): 7
(GHC version 8.5.20180419 for i386_unknown_linux)
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
Aborted
*** unexpected failure for T3001-2(prof_hb)
```
Closure type 7 is `CONSTR_NOCAF`, which is indeed invalid. Tracking this down will take some debugging.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------ |
| Version | 8.2.2 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Profiling |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"T3001-2 fails on i386 Linux","status":"New","operating_system":"","component":"Profiling","related":[],"milestone":"8.6.1","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.2.2","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"The `T3001-2` testcase fails on i386/Linux:\r\n\r\n{{{\r\nWrong exit code for T3001-2(prof_hb)(expected 0 , actual 134 )\r\nStderr ( T3001-2 ):\r\nT3001-2: internal error: Invalid object in processHeapClosureForDead(): 7\r\n (GHC version 8.5.20180419 for i386_unknown_linux)\r\n Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug\r\nAborted\r\n*** unexpected failure for T3001-2(prof_hb)\r\n}}}\r\n\r\nClosure type 7 is `CONSTR_NOCAF`, which is indeed invalid. Tracking this down will take some debugging.","type_of_failure":"OtherFailure","blocking":[]} -->8.6.1https://gitlab.haskell.org/ghc/ghc/-/issues/10915Statistical profiling support in the RTS2023-11-25T22:03:34ZBen GamariStatistical profiling support in the RTSNow since GHC can produce useful debugging information (e.g. DWARF annotations and source notes) in compiled objects, it would great if we could leverage this for more efficient profiling.
While ideally we would be able to rely on exter...Now since GHC can produce useful debugging information (e.g. DWARF annotations and source notes) in compiled objects, it would great if we could leverage this for more efficient profiling.
While ideally we would be able to rely on external tools like `perf` for this task, this would require that the STG stack be walkable like a standard C stack.
A more feasible alternative is to incorporate a simple statistical profiler into the RTS.
Profiling is defined loosely here as the collection of information describing the current state of evaluation in response to certain triggers. In practice this will be the collection of the current symbol being evaluated triggered on a few events of interest,
- Memory allocation
- Periodic timers for time-based profiling
- Blackhole blocking for locating performance problems due to sharing in parallel programs8.6.1Ben GamariBen Gamarihttps://gitlab.haskell.org/ghc/ghc/-/issues/7836"Invalid object in processHeapClosureForDead" when profiling with -hb2019-07-07T18:47:53Zhyperthunk"Invalid object in processHeapClosureForDead" when profiling with -hbRunning the attached program, compiled with `-threaded -Wall -auto-all -caf-all -fforce-recomp -fprof-auto-top -fprof-auto-calls` - with the following flags: `+RTS -hc -hbdrag,void -RTS`
The output is as follows:
```
leaks: internal er...Running the attached program, compiled with `-threaded -Wall -auto-all -caf-all -fforce-recomp -fprof-auto-top -fprof-auto-calls` - with the following flags: `+RTS -hc -hbdrag,void -RTS`
The output is as follows:
```
leaks: internal error: Invalid object in processHeapClosureForDead(): 60
(GHC version 7.4.2 for i386_apple_darwin)
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
Abort trap: 6
```8.6.1