Object code produced by GHC is non-deterministic

This is a continuation of #4012 (closed), which was resolved in 8.0.2 by making interface files deterministic. Now someone needs to make object code deterministic.

mentioned in issue #4012 (closed)

mentioned in issue #7231 (closed)

mentioned in issue #10037

mentioned in issue #12262 (closed)

changed weight to 5

added Tbug Trac import labels

changed the description

Trac metadata

Trac field	Value
Version	8.0.1 → 8.0.2

What's involved in fixing this? Is it just plugging away at a list of known sources of non-determinism or is it more complicated than that?

It's not an entirely trivial task. We use uniques quite often to name symbols in code produced by the native codegen. Fixing this without compromising compiler performance is rather tricky. I don't know if niteria has thought much about how to approach this.

mentioned in issue #14769

Attached file objdump-headers-1.txt (download).

Attached file objdump-headers-1.2.txt (download).

Attached file objdump-headers-2.txt (download).

I have attached some files, objdump-headers-1.txt and objdump-headers-1.txt which are the outputs of objdump --all-headers Example.o as produced by ghc --make -j1 of a project of mine.

Have a look at them in a diff viewer.

At the top you will find probably what was talked about before: Uniques making it into symbol names, like cZoQ_info vs cQJU_info.

But further down you see

0000000000002f10 R_X86_64_PC32     .data.rel.ro+0x00000000000000e8

vs

0000000000002f10 R_X86_64_PC32     .data.rel.ro+0x00000000000000f0

which is the result of some values in the RELOCATION RECORDS FOR [.data.rel.ro] having their positions swapped:

0000000000000100 R_X86_64_64       monozmtraversablezm1zi0zi2zi1zm4aNtRX1lKHJF1rNRp4LmtV_DataziMonoTraversable_zdfMonoFoldableZMZN_closure
0000000000000108 R_X86_64_64       monozmtraversablezm1zi0zi2zi1zm4aNtRX1lKHJF1rNRp4LmtV_DataziMonoTraversableziUnprefixed_any_closure

vs

0000000000000100 R_X86_64_64       monozmtraversablezm1zi0zi2zi1zm4aNtRX1lKHJF1rNRp4LmtV_DataziMonoTraversableziUnprefixed_any_closure
0000000000000108 R_X86_64_64       monozmtraversablezm1zi0zi2zi1zm4aNtRX1lKHJF1rNRp4LmtV_DataziMonoTraversable_zdfMonoFoldableZMZN_closure

These are the same things, they have exactly the same symbol names, just their order swapped.

How might this happen?

Trac metadata

Trac field	Value
CC	jb55, nomeata → jb55, nh2, nomeata

It's not an entirely trivial task. We use uniques quite often to name symbols in code produced by the native codegen. Fixing this without compromising compiler performance is rather tricky.

@bgamari Question:

Why cannot the uniques be predetermined by a seed that's dependent on the module name?

My understanding so far is this: We need an infinite stream of unused strings to generate symbols. Parallel compilation with -j and "partial" compilation of the projects (resumes, incremental compilation), means some number or pseudorandom generator has a different initial state when we come by the file next time.

So couldn't we initialise the seed of that pseudorandom generator as hash(module name), so that each time a module is compiled, it always generates the same uniques in the same order (as within a module there is no parallel compilation)? And when a module name changes, we have to recompile anyway and it's probably OK if the object code changes when that happens.

(Edit: Looks like this is the same question asked but unanswered here: #4012 (closed)##12935)