Skip to content
Snippets Groups Projects
Commit f5f09617 authored by Hannes Siebenhandl's avatar Hannes Siebenhandl
Browse files

Avoid unneccessarily re-serialising the `ModIface`

To reduce memory usage of `ModIface`, we serialise `ModIface` to an
in-memory byte array, which implicitly shares duplicated values.

This serialised byte array can be reused to avoid work when we actually
write the `ModIface` to disk.
We introduce a new field to `ModIface` which allows us to save the byte
array, and write it direclty to disk if the `ModIface` wasn't changed
after the initial serialisation.

This requires us to change absolute offsets, for example to jump to the
deduplication table for `Name` or `FastString` with relative offsets, as
the deduplication byte array doesn't contain header information, such as
fingerprints.
To allow us to dump the binary blob to disk, we need to replace all
absolute offsets with relative ones.

We introduce additional helpers for `ModIface` binary serialisation, which
construct relocatable binary blobs. We say the binary blob is relocatable,
if the binary representation can be moved and does not contain any
absolute offsets.

Further, we introduce new primitives for `Binary` that allow to create
relocatable binaries, such as `forwardGetRel` and `forwardPutRel`.

-------------------------
Metric Decrease:
    MultiLayerModulesDefsGhcWithCore
Metric Increase:
    MultiComponentModules
    MultiLayerModules
    T10421
    T12150
    T12234
    T12425
    T13035
    T13253-spj
    T13701
    T13719
    T14697
    T15703
    T16875
    T18140
    T18304
    T18698a
    T18730
    T18923
    T20049
    T24582
    T5837
    T6048
    T9198
    T9961
    mhu-perf
-------------------------

These metric increases may look bad, but they are all completely benign,
we simply allocate 1 MB per module for `shareIface`. As this allocation
is quite quick, it has a negligible impact on run-time performance.
In fact, the performance difference wasn't measurable on my local
machine. Reducing the size of the pre-allocated 1 MB buffer avoids these
test failures, but also requires us to reallocate the buffer if the
interface file is too big. These reallocations *did* have an impact on
performance, which is why I have opted to accept all these metric
increases, as the number of allocated bytes is merely a guidance.

This 1MB allocation increase causes a lot of tests to fail that
generally have a low allocation number. E.g., increasing from 40MB to
41MB is a 2.5% increase.
In particular, the tests T12150, T13253-spj, T18140, T18304, T18698a,
T18923, T20049, T24582, T5837, T6048, and T9961 only fail on i386-darwin
job, where the number of allocated bytes seems to be lower than in other
jobs.
The test T16875 fails on i386-linux-debian10 for the same reason.
parent c5e7b418
No related branches found
No related tags found
No related merge requests found
Pipeline #96783 canceled
Showing
with 853 additions and 242 deletions
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment