Skip to content
  • Hannes Siebenhandl's avatar
    5f213bff
    Make GHCi commands compatible with multiple home units · 5f213bff
    Hannes Siebenhandl authored
    === Design
    
    We enable all GHCi features that were previously guarded by the `inMulti`
    option.
    
    GHCi supported multiple home units up to a certain degree for quite a while now.
    The supported feature set was limited, due to a design impasse:
    One of the home units must be "active", e.g., there must be one `HomeUnit`
    whose `UnitId` is "active" which is returned when calling
    
    ```haskell
    do
      hscActiveUnitId <$> getSession
    ```
    
    This makes sense in a GHC session, since you are always compiling a particular
    Module, but it makes less intuitive sense in an interactive session.
    Given an expression to evaluate, we can't easily tell in which "context" the expression
    should be parsed, typechecked and evaluated.
    That's why initially, most of GHCi features, except for `:reload`ing were disabled
    if the GHCi session had more than one `HomeUnitEnv`.
    
    We lift this restriction, enabling all features of GHCi for the multiple home unit case.
    To do this, we fundamentally change the `HomeUnitEnv` graph to be multiple home unit first.
    Instead of differentiating the case were we have a single home unit and multiple,
    we now always set up a multiple home unit session that scales seamlessly to an arbitrary
    amount of home units.
    
    We introduce two new `HomeUnitEnv`s that are always added to the `HomeUnitGraph`.
    They are:
    
    The "interactive-ghci", called the `interactiveGhciUnit`, contains the same
    `DynFlags` that are used by the `InteractiveContext` for interactive evaluation
    of expressions.
    This `HomeUnitEnv` is only used on the prompt of GHCi, so we may refer to it as
    "interactive-prompt" unit.
    See Note [Relation between the `InteractiveContext` and `interactiveGhciUnitId`]
    for discussing its role.
    
    And the "interactive-session"", called `interactiveSessionUnit` or
    `interactiveSessionUnitId`, which is used for loading Scripts into
    GHCi that are not `Target`s of any home unit, via `:load` or `:add`.
    
    Both of these "interactive" home units depend on all other `HomeUnitEnv`s that
    are passed as arguments on the cli.
    Additionally, the "interactive-ghci" unit depends on `interactive-session`.
    
    We always evaluate expressions in the context of the
    "interactive-ghci" session.
    Since "interactive-ghci" depends on all home units, we can import any `Module`
    from the other home units with ease.
    
    As we have a clear `HomeUnitGraph` hierarchy, we can set `interactiveGhciUnitId`
    as the active home unit for the full duration of the GHCi session.
    In GHCi, we always set `interactiveGhciUnitId` to be the currently active home unit.
    
    === Implementation Details
    
    Given this design idea, the implementation is relatively straight
    forward.
    The core insight is that a `ModuleName` is not sufficient to identify a
    `Module` in the `HomeUnitGraph`. Thus, large parts of the PR is simply
    about refactoring usages of `ModuleName` to prefer `Module`, which has a
    `Unit` attached and is unique over the `HomeUnitGraph`.
    
    Consequentially, most usages of `lookupHPT` are likely to be incorrect and have
    been replaced by `lookupHugByModule` which is keyed by a `Module`.
    
    In `GHCi/UI.hs`, we make sure there is only one location where we are
    actually translating `ModuleName` to a `Module`:
    
    * `lookupQualifiedModuleName`
    
    If a `ModuleName` is ambiguous, we detect this and report it to the
    user.
    
    To avoid repeated lookups of `ModuleName`s, we store the `Module` in the
    `InteractiveImport`, which additionally simplifies the interface
    loading.
    
    A subtle detail is that the `DynFlags` of the `InteractiveContext` are
    now stored both in the `HomeUnitGraph` and in the `InteractiveContext`.
    In UI.hs, there are multiple code paths where we are careful to update
    the `DynFlags` in both locations.
    Most importantly in `addToProgramDynFlags`.
    
    ---
    
    There is one metric increase in this commit:
    
    -------------------------
    Metric Increase:
        T4029
    -------------------------
    
    It is an increase from 14.4 MB to 16.1 MB (+11.8%) which sounds like a
    pretty big regression at first.
    However, we argue this increase is solely caused by using more data
    structures for managing multiple home units in the GHCi session.
    In particular, due to the design decision of using three home units, the
    base memory usage increases... but by how much?
    
    A big contributor is the `UnitState`, of which we have three now, which
    on its own 260 KB per instance. That makes an additional memory usage of
    520 KB, already explaining a third of the overall memory usage increase.
    Then we store more elements in the `HomeUnitGraph`, we have more
    `HomeUnitEnv` entries, etc...
    
    While we didn't chase down each byte, we looked at the memory usage over time
    for both `-hi` and `-hT` profiles and can say with confidence while the memory
    usage increased slightly, we did not introduce any space leak, as
    the graph looks almost identical as the memory usage graph of GHC HEAD.
    
    ---
    
    Adds testcases for GHCi multiple home units session
    
    * Test truly multiple home unit sessions, testing reload logic and code evaluation.
    * Test that GHCi commands such as `:all-types`, `:browse`, etc., work
    * Object code reloading for home modules
    * GHCi debugger multiple home units session
    5f213bff
    Make GHCi commands compatible with multiple home units
    Hannes Siebenhandl authored
    === Design
    
    We enable all GHCi features that were previously guarded by the `inMulti`
    option.
    
    GHCi supported multiple home units up to a certain degree for quite a while now.
    The supported feature set was limited, due to a design impasse:
    One of the home units must be "active", e.g., there must be one `HomeUnit`
    whose `UnitId` is "active" which is returned when calling
    
    ```haskell
    do
      hscActiveUnitId <$> getSession
    ```
    
    This makes sense in a GHC session, since you are always compiling a particular
    Module, but it makes less intuitive sense in an interactive session.
    Given an expression to evaluate, we can't easily tell in which "context" the expression
    should be parsed, typechecked and evaluated.
    That's why initially, most of GHCi features, except for `:reload`ing were disabled
    if the GHCi session had more than one `HomeUnitEnv`.
    
    We lift this restriction, enabling all features of GHCi for the multiple home unit case.
    To do this, we fundamentally change the `HomeUnitEnv` graph to be multiple home unit first.
    Instead of differentiating the case were we have a single home unit and multiple,
    we now always set up a multiple home unit session that scales seamlessly to an arbitrary
    amount of home units.
    
    We introduce two new `HomeUnitEnv`s that are always added to the `HomeUnitGraph`.
    They are:
    
    The "interactive-ghci", called the `interactiveGhciUnit`, contains the same
    `DynFlags` that are used by the `InteractiveContext` for interactive evaluation
    of expressions.
    This `HomeUnitEnv` is only used on the prompt of GHCi, so we may refer to it as
    "interactive-prompt" unit.
    See Note [Relation between the `InteractiveContext` and `interactiveGhciUnitId`]
    for discussing its role.
    
    And the "interactive-session"", called `interactiveSessionUnit` or
    `interactiveSessionUnitId`, which is used for loading Scripts into
    GHCi that are not `Target`s of any home unit, via `:load` or `:add`.
    
    Both of these "interactive" home units depend on all other `HomeUnitEnv`s that
    are passed as arguments on the cli.
    Additionally, the "interactive-ghci" unit depends on `interactive-session`.
    
    We always evaluate expressions in the context of the
    "interactive-ghci" session.
    Since "interactive-ghci" depends on all home units, we can import any `Module`
    from the other home units with ease.
    
    As we have a clear `HomeUnitGraph` hierarchy, we can set `interactiveGhciUnitId`
    as the active home unit for the full duration of the GHCi session.
    In GHCi, we always set `interactiveGhciUnitId` to be the currently active home unit.
    
    === Implementation Details
    
    Given this design idea, the implementation is relatively straight
    forward.
    The core insight is that a `ModuleName` is not sufficient to identify a
    `Module` in the `HomeUnitGraph`. Thus, large parts of the PR is simply
    about refactoring usages of `ModuleName` to prefer `Module`, which has a
    `Unit` attached and is unique over the `HomeUnitGraph`.
    
    Consequentially, most usages of `lookupHPT` are likely to be incorrect and have
    been replaced by `lookupHugByModule` which is keyed by a `Module`.
    
    In `GHCi/UI.hs`, we make sure there is only one location where we are
    actually translating `ModuleName` to a `Module`:
    
    * `lookupQualifiedModuleName`
    
    If a `ModuleName` is ambiguous, we detect this and report it to the
    user.
    
    To avoid repeated lookups of `ModuleName`s, we store the `Module` in the
    `InteractiveImport`, which additionally simplifies the interface
    loading.
    
    A subtle detail is that the `DynFlags` of the `InteractiveContext` are
    now stored both in the `HomeUnitGraph` and in the `InteractiveContext`.
    In UI.hs, there are multiple code paths where we are careful to update
    the `DynFlags` in both locations.
    Most importantly in `addToProgramDynFlags`.
    
    ---
    
    There is one metric increase in this commit:
    
    -------------------------
    Metric Increase:
        T4029
    -------------------------
    
    It is an increase from 14.4 MB to 16.1 MB (+11.8%) which sounds like a
    pretty big regression at first.
    However, we argue this increase is solely caused by using more data
    structures for managing multiple home units in the GHCi session.
    In particular, due to the design decision of using three home units, the
    base memory usage increases... but by how much?
    
    A big contributor is the `UnitState`, of which we have three now, which
    on its own 260 KB per instance. That makes an additional memory usage of
    520 KB, already explaining a third of the overall memory usage increase.
    Then we store more elements in the `HomeUnitGraph`, we have more
    `HomeUnitEnv` entries, etc...
    
    While we didn't chase down each byte, we looked at the memory usage over time
    for both `-hi` and `-hT` profiles and can say with confidence while the memory
    usage increased slightly, we did not introduce any space leak, as
    the graph looks almost identical as the memory usage graph of GHC HEAD.
    
    ---
    
    Adds testcases for GHCi multiple home units session
    
    * Test truly multiple home unit sessions, testing reload logic and code evaluation.
    * Test that GHCi commands such as `:all-types`, `:browse`, etc., work
    * Object code reloading for home modules
    * GHCi debugger multiple home units session
Loading