"access violation" when building Decimal with 8.8.3 on windows

added Phigh Windows linking labels and removed needs triage label

Hmm, strangely I cannot reproduce this with Decimal-0.5.1 and GHC 8.8.3 from downloads.haskell.org using cabal-install to build.

@simonmichael, it looks like the log expired. Can you perhaps attach it to the ticket?

Actually, I was able to find another log. The failing command-line is

      C:\Users\runneradmin\AppData\Roaming\stack\setup-exe-cache\x86_64-windows\Cabal-simple_Z6RU0evB_3.0.1.0_ghc-8.8.3.exe --builddir=.stack-work\dist\29cc6475 configure --user --package-db=clear --package-db=global --package-db=C:\Users\runneradmin\AppData\Roaming\stack\snapshots\ef66253a\pkgdb --libdir=C:\Users\runneradmin\AppData\Roaming\stack\snapshots\ef66253a\lib --bindir=C:\Users\runneradmin\AppData\Roaming\stack\snapshots\ef66253a\bin --datadir=C:\Users\runneradmin\AppData\Roaming\stack\snapshots\ef66253a\share --libexecdir=C:\Users\runneradmin\AppData\Roaming\stack\snapshots\ef66253a\libexec --sysconfdir=C:\Users\runneradmin\AppData\Roaming\stack\snapshots\ef66253a\etc --docdir=C:\Users\runneradmin\AppData\Roaming\stack\snapshots\ef66253a\doc\Decimal-0.5.1 --htmldir=C:\Users\runneradmin\AppData\Roaming\stack\snapshots\ef66253a\doc\Decimal-0.5.1 --haddockdir=C:\Users\runneradmin\AppData\Roaming\stack\snapshots\ef66253a\doc\Decimal-0.5.1 --dependency=base=base-4.13.0.0 --dependency=deepseq=deepseq-1.4.4.0 --extra-include-dirs=C:\Users\runneradmin\AppData\Local\Programs\stack\x86_64-windows\msys2-20180531\mingw64\include --extra-lib-dirs=C:\Users\runneradmin\AppData\Local\Programs\stack\x86_64-windows\msys2-20180531\mingw64\lib --extra-lib-dirs=C:\Users\runneradmin\AppData\Local\Programs\stack\x86_64-windows\msys2-20180531\mingw64\bin --exact-configuration --ghc-option=-fhide-source-paths

Sadly, this being a Setup.hs invocation, it's quite difficult to reproduce.

However, the fact that 8.8.2 works is interesting. While there were quite a few Windows-specific changes in process, these really shouldn't be able to cause a segfault. AFAIK, Setup.hs doesn't rely on the runtime linker so I doubt that 8b193877 is culpable. The only other potentially relevant changes are in CorePrep. However, this really wouldn't be platform specific.

Hmmmm.

@simonmichael on failure could you zip up the build directory? or perhaps pass to GHC +RTS --generate-crash-dumps, if this is crashing in a custom setup like @bgamari suggested you can add to your cabal file ghc-options: -debug -rtsopts -with-rtsopts="--generate-crash-dumps"

The dumps are created in %TEMP%\*.dmp.

I can't reproduce it locally and I currently lack way too much information to be able to determine what this is.. Why do you suspect the linker @bgamari ? Decimal doesn't seem to use TH so there should have been no need for the runtime linker..

Indeed the linker was an initial guess but I agree that it doesn't fit the facts.

removed linking label

Not sure if that will help, but for me 8.8.3 will cause an Access violation for the following hello world Haskell program:

Main.hs:

main = putStrLn "Hello World"

D:\Source\Repos\stack-sanity-check> D:\Haskell\ghc-8.8.3\bin\ghc.exe D:\Source\Repos\stack-sanity-check\Main.hs -no-use
r-package-db
Linking D:\Source\Repos\stack-sanity-check\Main.exe ...

Access violation in generated code when writing 0x0

 Attempting to reconstruct a stack trace...

   Frame        Code address
 * 0x461da80    0x37f7b66 D:\Haskell\ghc-8.8.3\bin\ghc.exe+0x33f7b66
 * 0x461da88    0x3277bb9 D:\Haskell\ghc-8.8.3\bin\ghc.exe+0x2e77bb9
 * 0x461da90    0x3
 * 0x461da98    0x69fa651
 * 0x461daa0    0x3d80c90 D:\Haskell\ghc-8.8.3\bin\ghc.exe+0x3980c90
 * 0x461daa8    0x67a2c70

D:\Source\Repos\stack-sanity-check>

Doesn't this look like dereferencing NULL somewhere within ghc.exe?

Is this using the Haskell.org x86-64 Windows binary distribution?

Which Windows version?

@jakzale Maybe, but the error is too generic. I really would like a dump otherwise we're just guessing... can you call ghc with +RTS --generate-crash-dumps and upload the dump file.

If it doesn't generate one use procdump. https://docs.microsoft.com/en-us/sysinternals/downloads/procdump and using procdump.exe -t -ma -e 1 -x . <program>.exe. This would really make things go a lot faster... since I can't reproduce it on my machine it's highly likely that something environmental if affecting it.

GHC Version is Haskell.org x86-64 windows binary distribution.

OS Version is Windows 10 Version 2004, build 19041.153.

As the crash dump was around ~84 Mb I uploaded it to Dropbox.

While the crashdump was uploading, I run analysis on it and here are the results:

0:000> !analyze -v
*******************************************************************************
*                                                                             *
*                        Exception Analysis                                   *
*                                                                             *
*******************************************************************************


KEY_VALUES_STRING: 1

    Key  : AV.Dereference
    Value: NullPtr

    Key  : AV.Fault
    Value: Write

    Key  : Analysis.CPU.Sec
    Value: 3

    Key  : Analysis.DebugAnalysisProvider.CPP
    Value: Create: 8007007e on DESKTOP-HQV75QJ

    Key  : Analysis.DebugData
    Value: CreateObject

    Key  : Analysis.DebugModel
    Value: CreateObject

    Key  : Analysis.Elapsed.Sec
    Value: 350

    Key  : Analysis.Memory.CommitPeak.Mb
    Value: 110

    Key  : Analysis.System
    Value: CreateObject

    Key  : Timeline.Process.Start.DeltaSec
    Value: 1


ADDITIONAL_XML: 1

APPLICATION_VERIFIER_FLAGS:  0

CONTEXT:  (.ecxr)
rax=0000000000000000 rbx=0000000006587b20 rcx=129774d5617b0000
rdx=0000000000000000 rsi=0000000003d46012 rdi=0000000006722ef0
rip=000000000676c700 rsp=000000000461dab0 rbp=00000000068f0cd8
 r8=0000000000000000  r9=000000000461d988 r10=0000000000000011
r11=000000000461d970 r12=0000000006731dd0 r13=0000000003d76898
r14=0000000006722aca r15=00000000068e90c0
iopl=0         nv up ei pl nz na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010206
00000000`0676c700 ??              ???
Resetting default scope

EXCEPTION_RECORD:  (.exr -1)
ExceptionAddress: 00000000037f7b66 (ghc+0x00000000033f7b66)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 0000000000000001
   Parameter[1]: 0000000000000000
Attempt to write to address 0000000000000000

PROCESS_NAME:  ghc.exe

WRITE_ADDRESS:  0000000000000000 

ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.

EXCEPTION_CODE_STR:  c0000005

EXCEPTION_PARAMETER1:  0000000000000001

EXCEPTION_PARAMETER2:  0000000000000000

FAULTING_THREAD:  ffffffff

IP_ON_HEAP:  000000000676c700
The fault address in not in any loaded module, please check your build's rebase
log at <releasedir>\bin\build_logs\timebuild\ntrebase.log for module which may
contain the address if it were loaded.

STACK_TEXT:  
00000000`037f7b66 00000000`037f7b66 ghc+0x33f7b66


STACK_COMMAND:  .ecxr ; kb ; ** Pseudo Context ** Pseudo ** Value: 1e41c6fe3b0 ** ; kb

SYMBOL_NAME:  ghc+33f7b66

MODULE_NAME: ghc

IMAGE_NAME:  ghc.exe

FAILURE_BUCKET_ID:  NULL_POINTER_WRITE_c0000005_ghc.exe!Unknown

OS_VERSION:  10.0.19041.1

BUILDLAB_STR:  vb_release

OSPLATFORM_TYPE:  x64

OSNAME:  Windows 10

FAILURE_ID_HASH:  {22332386-e0fd-7738-b7f8-6a447359de7c}

Followup:     MachineOwner
---------

Thanks @jakzale i'll take a look today

Now I'm seeing the notifications from this. @Phyx, I tried but haven't successfully captured .dmp files. At https://github.com/simonmichael/hledger/actions/runs/59047658 the dumps.zip artifact is the full contents of C:\Users\runneradmin\AppData\Local\Temp, after doing stack --no-terminal build --only-dependencies || true, with

ghc-options:
  "$locals": -debug -rtsopts -with-rtsopts="--generate-crash-dumps"

in stack.yaml. I don't know if it has anything useful.

this could be an indication that the crash is happening during unwinding.. let's see if @jakzale can produce a crash dump.

We are getting a similar problem in Agda in the same environment (GHC 8.8.3, invoked via stack 2.1.3, Windows Server 2019 ("windows-latest" in github actions)). Also the problem doesn't happen with GHC 8.8.2:

alex                > Preprocessing executable 'alex' for alex-3.2.5..
alex                > Building executable 'alex' for alex-3.2.5..
alex                > 
alex                > Access violation in generated code when writing 0x0
alex                > 
alex                >  Attempting to reconstruct a stack trace...
alex                > 
alex                >    Frame	Code address
alex                >  * 0x461da80	0x37f7b66 C:\Users\runneradmin\AppData\Local\Programs\stack\x86_64-windows\ghc-8.8.3\bin\ghc.exe+0x33f7b66
alex                >  * 0x461da88	0x3277bb9 C:\Users\runneradmin\AppData\Local\Programs\stack\x86_64-windows\ghc-8.8.3\bin\ghc.exe+0x2e77bb9
alex                >  * 0x461da90	0x5
alex                >  * 0x461da98	0x5
alex                >  * 0x461daa0	0x3d7dc22 C:\Users\runneradmin\AppData\Local\Programs\stack\x86_64-windows\ghc-8.8.3\bin\ghc.exe+0x397dc22
alex                >  * 0x461daa8	0x718b230

The GitHub log is here. I also don't have a machine with Windows for reproducing the issue.

We have a similar error building ghcide dependencies. when compiling base-compat:

base-compat                      > Preprocessing library for base-compat-0.11.1..
base-compat                      > Building library for base-compat-0.11.1..
base-compat                      > 
base-compat                      > Access violation in generated code when writing 0x0
base-compat                      > 
base-compat                      >  Attempting to reconstruct a stack trace...
base-compat                      > 
base-compat                      >    Frame	Code address
base-compat                      >  * 0x461da80	0x37f7b66 C:\Users\VssAdministrator\AppData\Local\Programs\stack\x86_64-windows\ghc-8.8.3\bin\ghc.exe+0x33f7b66
base-compat                      >  * 0x461da88	0x3277bb9 C:\Users\VssAdministrator\AppData\Local\Programs\stack\x86_64-windows\ghc-8.8.3\bin\ghc.exe+0x2e77bb9
base-compat                      >  * 0x461da90	0x5
base-compat                      >  * 0x461da98	0x5
base-compat                      >  * 0x461daa0	0x3d7dc22 C:\Users\VssAdministrator\AppData\Local\Programs\stack\x86_64-windows\ghc-8.8.3\bin\ghc.exe+0x397dc22
base-compat                      >  * 0x461daa8	0x6f5a1f0
base-compat                      >

The error is thrown consistently building the package in azure pipelines (Windows Server 2019): https://dev.azure.com/jneira/haskell-language-server/_build/results?buildId=736&view=logs&j=c161c675-1417-57ed-29bf-7d19b694489b&t=3813d704-4df5-5eff-d5f1-7414d6aa39b0

I can't reproduce it locally in my windows 10 though.

Update: the error is not reproduced if you use base-compat-0.11.0 instead base-compat-0.11.1: https://dev.azure.com/jneira/haskell-language-server/_build/results?buildId=738&view=logs&j=c161c675-1417-57ed-29bf-7d19b694489b&t=3813d704-4df5-5eff-d5f1-7414d6aa39b0

But then is shake the lib which hrows the error...

EDIT: Downgrading shake to 0.8.14 instead 0.8.15 make the build succesful... https://dev.azure.com/jneira/haskell-language-server/_build/results?buildId=740&view=logs&j=c161c675-1417-57ed-29bf-7d19b694489b

TL;DR it seems that you need very recent Windows environment (which hosted CIs seems to have more often than user machines).

The issue started happening in Cabal CI (failing consistently, but on what looks to be arbitrary dependency) recently.

The GitHub actions image notes are at: https://github.com/actions/virtual-environments/releases/tag/win19%2F20200319.1
previous image was https://github.com/actions/virtual-environments/releases/tag/win19%2F20200308.0

I cannot spot other difference than upgrade of Visual Studio (I have no idea how that might affect things), from Version: VisualStudio/16.4.5+29806.167 to Version: VisualStudio/16.5.0+29911.84. It might be that msys stuff is not compatible? (e.g. see https://github.com/actions/virtual-environments/issues/625 - suggests that some tools need to be updated).

There's also next GH Actions windows image (https://github.com/actions/virtual-environments/releases/tag/win19%2F20200330.1), I'll comment whether it "fixes" the issue or not.

It looks like ghcide is running into this in their CI.

@phadej notes that Cabal has also seen a similar crash in 8.6.5. Consequently bisection likely isn't worthwhile.

I believe @phadej managed to reproduce it on AppVeyor, so I should be able to remote desktop there to debug further. The dumpfile was a good start but the line it claims causes this is in a file that doesn't seem to exist in the compiler.

Then again the compiler is statically linked so it could have come from some library. But being able to reproduce makes it very hard to debug.

We are getting a similar problem in Agda in the same environment (GHC 8.8.3, invoked via stack 2.1.3, Windows Server 2019 ("windows-latest" in github actions)). Also the problem doesn't happen with GHC 8.8.2.

FYI, the problem disappeared in Agda when using GHC 8.10.1.

I am glad to hear it. I am going to optimistically assume that the issue was fixed by the something the windows stabilization work that is present in 8.10 (perhaps the toolchain bump?). Do let us if you encounter this again the future.

closed

I note that GHC 8.8.1, 8.8.2 and 8.8.3 are all mostly unusable on Windows (the first two the Cabal linking bug, the last one that everything segfaults when building). I appreciate 8.10 fixes it, but to a large extent this is leaving Windows unsupported for GHC 8.8.

"access violation" when building Decimal with 8.8.3 on windows

Summary

Steps to reproduce

Expected behavior

Environment

Child items ...

Activity

"access violation" when building Decimal with 8.8.3 on windows

Summary

Steps to reproduce

Expected behavior

Environment

Relates to

Activity