-Wunused-packages doesn't understand backpack semantics

Summary

-Wunused-packages warns that signatures don't need their dependencies and that indefinite packages don't need their signatures.

Steps to reproduce

Consider this simple backpack project:

==> bug.cabal <==
cabal-version: 3.0
name:          bug
version:       0.1.0.0

common default
  ghc-options:        -Werror -Wunused-packages
  default-language:   Haskell2010
  default-extensions: NoImplicitPrelude
  build-depends:      base

library sig
  import:         default
  signatures:     Sig
  build-depends:  semigroups
  hs-source-dirs: sig

library lib
  import:          default
  exposed-modules: Lib
  hs-source-dirs:  lib
  build-depends:   sig

library imp
  import:          default
  exposed-modules: Imp
  build-depends:   semigroups
  hs-source-dirs:  imp

executable app
  import:         default
  main-is:        App.hs
  hs-source-dirs: app
  build-depends:
    , imp
    , lib

  mixins:         imp (Imp as Sig)

==> sig/Sig.hsig <==
signature Sig (T, t, f) where

import Data.List.NonEmpty (NonEmpty)
import Prelude (Show)

data T

instance Show T

t :: T
f :: NonEmpty T -> T

==> lib/Lib.hs <==
module Lib (u) where

import Sig (T, f, t)

u :: T
u = f t

==> imp/Imp.hs <==
module Imp (T, t, f) where

import Data.List.NonEmpty (NonEmpty, head)
import Prelude (Show)

data T = L | R deriving (Show)

t :: T
t = L

f :: NonEmpty T -> T
f = head

==> app/App.hs <==
import Lib (u)
import Prelude (print, IO)

main :: IO ()
main = print u

Then cabal build sig and imp complain that

<no location info>: error: [-Wunused-packages, -Werror=unused-packages]
    The following packages were specified via -package or -package-id flags,
    but were not needed for compilation:
      - semigroups-0.19.1

If we suppress this by passing -Wno-unused-packages to the sig build, then cabal build lib complains that

<no location info>: error: [-Wunused-packages, -Werror=unused-packages]
    The following packages were specified via -package or -package-id flags,
    but were not needed for compilation:
      - bug-0.1.0.0:sig[Sig=<Sig>]

Full error logs have been posted at ix.io, for no suppression, sig suppression and sig+lib suppression

Expected behavior

This shouldn't complain at all. However, judging by #13493, this might be a bit difficult to implement, so at least having this check have a toggle to switch between off/no-backpack/on would help.

Environment

• GHC version used: 8.10.1