GHCi multi-unit package-db stack merge treats reversed packageDBFlags as command-line order, dropping other units' package dbs (#27640) · Issues · Glasgow Haskell Compiler / GHC · GitLab

GHCi multi-unit package-db stack merge treats reversed packageDBFlags as command-line order, dropping other units' package dbs

## Summary Commit 728662de20 ("Use home unit package db stacks in GHCi prompt and session unit", backported to 9.14.2-rc as 18b5da32fd) computes the package-db stack for the GHCi prompt/session units of a multi-home-unit session by merging the per-unit stacks with `concatPackageDbStacksUsingLongestCommonPrefix` (`ghc/GHCi/UI.hs`): ```haskell concatPackageDbStacksUsingLongestCommonPrefix :: [[PackageDBFlag]] -> [PackageDBFlag] concatPackageDbStacksUsingLongestCommonPrefix stacks = let longestCommonPrefix = map List.head . List.takeWhile ((List.all . (==) . List.head) <*> List.tail) . List.transpose prefix = longestCommonPrefix stacks in prefix ++ nubOrd (concatMap (List.drop (length prefix)) stacks) ``` The algorithm is written for command-line order — the commit message describes cabal/stack stacks as sharing a common *leading* part (`-no-user-package-db -package-db $store`) and differing only in a trailing per-unit local db. But `packageDBFlags` is stored in *reverse* command-line order: every setter prepends (`addPkgDbRef`, `removeUserPkgDb`, `removeGlobalPkgDb`, `clearPkgDb` in `compiler/GHC/Driver/Session.hs`), and the consumer `getUnitDbRefs` documents and relies on that ("The list of flags is reversed (later ones first)", `compiler/GHC/Unit/State.hs`). In the reversed representation the differing per-unit local dbs come *first*, so whenever the stacks differ at all the computed prefix is empty and the function degenerates to `nubOrd (concat stacks)` over the reversed lists. Two concrete consequences: * With cabal-style stacks `-clear-package-db -global-package-db -package-db $store -package-db $localX`, the stored lists are `[local_A, store, global, Clear]` and `[local_B, store, global, Clear]`; the merge yields `[local_A, store, global, Clear, local_B]`. `getUnitDbRefs` folds this right-to-left, so `ClearPackageDBs` (from unit A's portion) wipes `local_B` — unit B's package db is silently dropped and its package dependencies are unavailable to the prompt unit. * With the `-no-user-package-db` variant, the merged `[local_A, store, NoUser, local_B]` resolves (right shadows left) with the store db shadowing unit B's local db — the inverse of the intended precedence, so the wrong version of a dependency can win. Had the lists been in command-line order the algorithm would produce exactly the intended `[Clear, global, store] ++ [local_A, local_B]`. This defeats the commit's own purpose (making home-unit dependencies importable at the prompt with correct precedence) precisely in the non-degenerate case the longest-common-prefix logic was written for. The new tests cannot see it: `prog-mhu006`'s units have identical single-flag stacks (the merge is the identity) and `prog025` has a single home unit. ## Steps to reproduce Two home units, each depending on a package registered in its own package db, with a common leading stack: * unitA: `-clear-package-db -global-package-db -package-db db-a -package-id pkga-1.0` * unitB: `-clear-package-db -global-package-db -package-db db-b -package-id pkgb-1.0` `run.sh` below builds the packages and dbs and runs two sessions. Control (both dbs listed in both units, i.e. identical stacks): ``` == control: identical per-unit db stacks (works) == pkga and pkgb imported OK ``` Bug case (stacks differing in the last entry): ``` == bug: per-unit db stacks differing in the last entry == <command line>: cannot satisfy -package-id pkgb-1.0 (use -v for more information) ``` GHCi fails to start at all: the prompt unit is given `-package-id` flags for every home-unit dependency, but `db-b` has been dropped from its merged db stack, so `pkgb-1.0` cannot be resolved. Verified on x86_64-linux with a master build (10.1.20260710, d415f38a75); the function is identical on the ghc-9.14 branch after the 9.14.2 preparations. <details><summary>run.sh</summary> ```sh #!/bin/sh # Reproducer for: GHCi prompt unit-state merge misreads reversed packageDBFlags. # # Two home units, each with a cabal-style package-db stack that shares a # common leading part (-clear-package-db -global-package-db) and differs # only in the trailing per-unit local db: # # unitA: -clear-package-db -global-package-db -package-db db-a (contains pkga) # unitB: -clear-package-db -global-package-db -package-db db-b (contains pkgb) # # Expected: `ghci -unit @unitA -unit @unitB` starts, and both `import PkgA` # and `import PkgB` work at the prompt. # Actual: db-b is dropped from the merged stack, so pkgb-1.0 cannot be # found when setting up the GHCi prompt unit. set -e GHC=${GHC:-ghc} GHC_PKG=${GHC_PKG:-ghc-pkg} here=$(cd "$(dirname "$0")" && pwd) work=$here/work rm -rf "$work" mkdir -p "$work" cd "$work" LIBDIR=$("$GHC" --print-libdir) VER=$("$GHC" --numeric-version) GLOBALDB="$LIBDIR/package.conf.d" BASE_ID=$("$GHC_PKG" --global-package-db="$GLOBALDB" field base id | head -n1 | sed 's/^id: *//') # --- two tiny packages, each registered in its own package db ------------ mk_pkg () { # $1 = pkg name (pkga), $2 = module name (PkgA), $3 = db dir mkdir -p "$1" printf 'module %s where\n\nvalue :: String\nvalue = "%s"\n' "$2" "$1" > "$1/$2.hs" "$GHC" -c -dynamic-too -this-unit-id "$1-1.0" -hidir "$1" -odir "$1" "$1/$2.hs" ar cqs "$1/libHS$1-1.0.a" "$1/$2.o" "$GHC" -shared -dynamic -this-unit-id "$1-1.0" \ -o "$1/libHS$1-1.0-ghc$VER.so" "$1/$2.dyn_o" cat > "$1.conf" <<EOF name: $1 version: 1.0 id: $1-1.0 key: $1-1.0 exposed: True exposed-modules: $2 import-dirs: $work/$1 library-dirs: $work/$1 dynamic-library-dirs: $work/$1 hs-libraries: HS$1-1.0 depends: $BASE_ID EOF "$GHC_PKG" init "$3" "$GHC_PKG" --no-user-package-db --global-package-db="$GLOBALDB" \ --package-db="$3" register "$1.conf" } mk_pkg pkga PkgA db-a mk_pkg pkgb PkgB db-b # --- two home units, differing only in the last package db --------------- mkdir -p unitA unitB printf 'module A where\nimport PkgA\na :: String\na = value\n' > unitA/A.hs printf 'module B where\nimport PkgB\nb :: String\nb = value\n' > unitB/B.hs cat > unitA.rsp <<EOF -i$work/unitA -this-unit-id unitA -clear-package-db -global-package-db -package-db $work/db-a -package-id pkga-1.0 $work/unitA/A.hs EOF cat > unitB.rsp <<EOF -i$work/unitB -this-unit-id unitB -clear-package-db -global-package-db -package-db $work/db-b -package-id pkgb-1.0 $work/unitB/B.hs EOF cat > f1.script <<EOF import PkgA import PkgB putStrLn (PkgA.value ++ " and " ++ PkgB.value ++ " imported OK") EOF # Control: same two units, but with *identical* package-db stacks # (both dbs in each unit). The merge degenerates to the identity and # everything works, showing the failure below is due to the merge. sed "s#-package-db $work/db-a#-package-db $work/db-a\n-package-db $work/db-b#" \ unitA.rsp > unitA-ctl.rsp sed "s#-package-db $work/db-b#-package-db $work/db-a\n-package-db $work/db-b#" \ unitB.rsp > unitB-ctl.rsp echo "== control: identical per-unit db stacks (works) ==" "$GHC" --interactive -v0 -ignore-dot-ghci \ -unit @"$work"/unitA-ctl.rsp -unit @"$work"/unitB-ctl.rsp < f1.script \ || echo "control failed unexpectedly" echo echo "== bug: per-unit db stacks differing in the last entry ==" "$GHC" --interactive -v0 -ignore-dot-ghci \ -unit @"$work"/unitA.rsp -unit @"$work"/unitB.rsp < f1.script ``` </details> (The failure mode depends on the shape of the stacks: with `-clear-package-db` in the shared part, GHCi fails to start as above or drops a unit's db; with `-no-user-package-db`, sessions start but a shared/store db shadows a unit's local db, so a wrong dependency version can be picked up silently.) ## Suggested fix Do the merge in command-line order: reverse each stack, compute the longest common prefix, append the `nubOrd` of the rests, and reverse the result — equivalently, compute the longest common *suffix* of the stored (reversed) stacks and prepend the union of the remainders. A test with two home units whose db stacks actually differ (the case the function exists for) would cover this; neither `prog-mhu006` nor `prog025` does. ## Provenance Found by [GlaHiCo](https://gitlab.haskell.org/sjakobi/GlaHiCo), an LLM-driven review pipeline, while reviewing the 9.14.2 backports [18b5da32fd](https://gitlab.haskell.org/ghc/ghc/-/commit/18b5da32fddced8205442007b3800767ada1259e) / [f4afeba1ce](https://gitlab.haskell.org/ghc/ghc/-/commit/f4afeba1ce2ed2767b57beb96c433e378aafc048) (master: [728662de20](https://gitlab.haskell.org/ghc/ghc/-/commit/728662de20) / [286f1adff3](https://gitlab.haskell.org/ghc/ghc/-/commit/286f1adff3)). The published review is at https://gitlab.haskell.org/sjakobi/GlaHiCo/-/snippets/6134 (finding f1). This draft was prepared by an agent (which also built and ran the reproducer) and reviewed by a human before filing.

issue

Admin message