Skip to content
  • Rodrigo Mesquita's avatar
    addbcbfb
    Add extraLibDirs to runtime lib search paths of library · addbcbfb
    Rodrigo Mesquita authored
    Runtime search paths are hard. Here's the current picture to understand
    why this patch exists:
    
    * When linking a shared library, GHC will include in the rpath entries
      of the shared library all the paths listed in the library dirs section
      of the installed package info of all packages the shared library
      depends on.
        * On darwin, GHC has special logic to inject the library dirs listed
          in the installed dependent packages info into the rpath section
          instead of passing the dirs as -rpath flags to the linker.
          However, only the dirs where used libraries are found are
          actually injected. The others are ignored. This works around
          limitations of the darwin loader.
    
    * Cabal, in addition, passes directly to the linker (via
      -optl-Wl,-rpath,...) the library dirs of packages the
      shared library for the package being built depends on.
        * In a vanilla cabal installation, this will typically only be the
          path to the cabal store and the path to the installed GHC's boot
          libraries store.
        * When using nix there will a different library dir per installed
          package. Since these lib dirs are passed directly to the linker as
          rpaths, we bypass the darwin loader logic and, for very big
          packages, on darwin, we could end up reaching the load command
          limit and fail linking. We don't address this situation in this MR.
    
    When we specify `extra-lib-dirs` in Cabal, these extra-lib-dirs will be
    added to the library dirs listed in the installed package info of the
    library they were specified for. Furthermore, when building a shared
    library, extra-lib-dirs will be passed as `-L` flags to the linker
    invocation. However, the same extra-lib-dirs will not be passed as
    `-rpath` to the linker.
    
    The end situation is as follows:
    
        1. The shared library `libA` built for a package `A` will be linked
           against some libraries `libExtra` found in extra-lib-dirs
           `extraA`.
    
        2. The RPATH section of `A` will NOT contain `extraA`, because we
           don't pass -rpath extra-lib-dirs when linking the library, but it
           will depend on `libExtra`.
    
        3. The installed package info of that package `A` will contain, in
           the library dirs section, the extra-lib-dirs `extraA` and the
           path to `libA`.
    
        4. When a package `B` depends on package `A`, it will include in the
           RPATH section of the shared library `libB` the lib dirs from the
           installed package info of `A`, i.e. `/path/to/libA` and `extraA`,
           and depends on `libA` and, transitively, on `libExtra`.
    
    The conclusion is:
    
        5. When we load `libB`, we will load `libA`, which is found in
           `/path/to/libA`, and, transitively, load `libExtra` which is
           found in `extraA` -- they are both found because both
           `/path/to/libA` and `extraA` are listed in the RPATH entries.
    
        6. However, if we load `libA` directly we will /NOT/ find
           `libExtra`, because `extraA` is not included in the RPATH
           entries.
    
    So, ultimately, what this commit fixes, is the failure described in (6),
    caused by the incorrect behaviour of (2), by specifying `-rpath
    extra-lib-dirs` when linking the shared library of a package, to include
    the extra lib dirs in the RPATH entries of that shared library (even
    though dependents of this library would already get the extra-lib-dirs
    in their RPATH, the library itself didn't, resulting in cabal#7339 and
    ghc#19350)
    
    Fixes #7339
    Fixes ghc#19350
    addbcbfb
    Add extraLibDirs to runtime lib search paths of library
    Rodrigo Mesquita authored
    Runtime search paths are hard. Here's the current picture to understand
    why this patch exists:
    
    * When linking a shared library, GHC will include in the rpath entries
      of the shared library all the paths listed in the library dirs section
      of the installed package info of all packages the shared library
      depends on.
        * On darwin, GHC has special logic to inject the library dirs listed
          in the installed dependent packages info into the rpath section
          instead of passing the dirs as -rpath flags to the linker.
          However, only the dirs where used libraries are found are
          actually injected. The others are ignored. This works around
          limitations of the darwin loader.
    
    * Cabal, in addition, passes directly to the linker (via
      -optl-Wl,-rpath,...) the library dirs of packages the
      shared library for the package being built depends on.
        * In a vanilla cabal installation, this will typically only be the
          path to the cabal store and the path to the installed GHC's boot
          libraries store.
        * When using nix there will a different library dir per installed
          package. Since these lib dirs are passed directly to the linker as
          rpaths, we bypass the darwin loader logic and, for very big
          packages, on darwin, we could end up reaching the load command
          limit and fail linking. We don't address this situation in this MR.
    
    When we specify `extra-lib-dirs` in Cabal, these extra-lib-dirs will be
    added to the library dirs listed in the installed package info of the
    library they were specified for. Furthermore, when building a shared
    library, extra-lib-dirs will be passed as `-L` flags to the linker
    invocation. However, the same extra-lib-dirs will not be passed as
    `-rpath` to the linker.
    
    The end situation is as follows:
    
        1. The shared library `libA` built for a package `A` will be linked
           against some libraries `libExtra` found in extra-lib-dirs
           `extraA`.
    
        2. The RPATH section of `A` will NOT contain `extraA`, because we
           don't pass -rpath extra-lib-dirs when linking the library, but it
           will depend on `libExtra`.
    
        3. The installed package info of that package `A` will contain, in
           the library dirs section, the extra-lib-dirs `extraA` and the
           path to `libA`.
    
        4. When a package `B` depends on package `A`, it will include in the
           RPATH section of the shared library `libB` the lib dirs from the
           installed package info of `A`, i.e. `/path/to/libA` and `extraA`,
           and depends on `libA` and, transitively, on `libExtra`.
    
    The conclusion is:
    
        5. When we load `libB`, we will load `libA`, which is found in
           `/path/to/libA`, and, transitively, load `libExtra` which is
           found in `extraA` -- they are both found because both
           `/path/to/libA` and `extraA` are listed in the RPATH entries.
    
        6. However, if we load `libA` directly we will /NOT/ find
           `libExtra`, because `extraA` is not included in the RPATH
           entries.
    
    So, ultimately, what this commit fixes, is the failure described in (6),
    caused by the incorrect behaviour of (2), by specifying `-rpath
    extra-lib-dirs` when linking the shared library of a package, to include
    the extra lib dirs in the RPATH entries of that shared library (even
    though dependents of this library would already get the extra-lib-dirs
    in their RPATH, the library itself didn't, resulting in cabal#7339 and
    ghc#19350)
    
    Fixes #7339
    Fixes ghc#19350
Loading