Skip to content
Snippets Groups Projects
  • Matthew Pickering's avatar
    7d46115b
    perf: Group together packages by repo when verifying tarballs · 7d46115b
    Matthew Pickering authored
    verifyFetchedTarball has the effect of deserialising the index tarball
    (see call to Sec.withIndex).
    
    verifyFetchedTarball is called individually for each package in the
    build plan (see ProjectPlanning.hs). Not once per repo.
    
    The hackage tarball is now 880mb so it takes a non significant amount of
    time to deserialise this (much better after haskell/tar#95).
    
    This code path is important as it can add 1s with these 38 calls on to
    the initial load of a project and scales linearly with the size of your
    build tree.
    
    Reproducer: Simple project with "lens" dependency deserialises the index tarball 38 times.
    
    Solution: Refactor verifyFetchedTarball to run once per repo rather than once per package.
    
    In future it would be much better to refactor this function so that the
    items are not immediately grouped and ungrouped but I didn't want to
    take that on immediately.
    
    Fixes #10110
    7d46115b
    History
    perf: Group together packages by repo when verifying tarballs
    Matthew Pickering authored
    verifyFetchedTarball has the effect of deserialising the index tarball
    (see call to Sec.withIndex).
    
    verifyFetchedTarball is called individually for each package in the
    build plan (see ProjectPlanning.hs). Not once per repo.
    
    The hackage tarball is now 880mb so it takes a non significant amount of
    time to deserialise this (much better after haskell/tar#95).
    
    This code path is important as it can add 1s with these 38 calls on to
    the initial load of a project and scales linearly with the size of your
    build tree.
    
    Reproducer: Simple project with "lens" dependency deserialises the index tarball 38 times.
    
    Solution: Refactor verifyFetchedTarball to run once per repo rather than once per package.
    
    In future it would be much better to refactor this function so that the
    items are not immediately grouped and ungrouped but I didn't want to
    take that on immediately.
    
    Fixes #10110
Code owners
Assign users and groups as approvers for specific file changes. Learn more.