Skip to content
  • Hannes Siebenhandl's avatar
    Add run-time configurability of `.hi` file compression · 36aa7cf1
    Hannes Siebenhandl authored and Marge Bot's avatar Marge Bot committed
    Introduce the flag `-fwrite-if-compression=<n>` which allows to
    configure the compression level of writing .hi files.
    
    The motivation is that some deduplication operations are too expensive
    for the average use case. Hence, we introduce multiple compression
    levels with variable impact on performance, but still reduce the
    memory residency and `.hi` file size on disk considerably.
    
    We introduce three compression levels:
    
    * `1`: `Normal` mode. This is the least amount of compression.
        It deduplicates only `Name` and `FastString`s, and is naturally the
        fastest compression mode.
    * `2`: `Safe` mode. It has a noticeable impact on .hi file size and is
      marginally slower than `Normal` mode. In general, it should be safe to
      always use `Safe` mode.
    * `3`: `Full` deduplication mode. Deduplicate as much as we can,
      resulting in minimal .hi files, but at the cost of additional
      compilation time.
    
    Reading .hi files doesn't need to know the initial compression level,
    and can always deserialise a `ModIface`, as we write out a byte that
    indicates the next value has been deduplicated.
    This allows users to experiment with different compression levels for
    packages, without recompilation of dependencies.
    
    Note, the deduplication also has an additional side effect of reduced
    memory consumption to implicit sharing of deduplicated elements.
    See #24540 for example where
    that matters.
    
    -------------------------
    Metric Decrease:
        MultiLayerModulesDefsGhciWithCore
        T16875
        T21839c
        T24471
        hard_hole_fits
        libdir
    -------------------------
    36aa7cf1