family instance consistency checks are too pessimistic
FamInst has this logic for checking type family instance consistency among imports:
Note [Checking family instance consistency] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ For any two family instance modules that we import directly or indirectly, we check whether the instances in the two modules are consistent, *unless* we can be certain that the instances of the two modules have already been checked for consistency during the compilation of modules that we import. [...] How do we know which pairs of modules have already been checked? Any pair of modules where both modules occur in the `HscTypes.dep_finsts' set (of the `HscTypes.Dependencies') of one of our directly imported modules must have already been checked. Everything else, we check now. (So that we can be certain that the modules in our `HscTypes.dep_finsts' are consistent.)
However, suppose one of the modules
A we import directly is itself a type family instance module. Then it too has been checked for consistency with its dependencies
C, etc., so we should skip checking the pairs
The current behavior means that whenever we directly import a type family instance module, we still have to load the interface files for all its type family module dependencies, which largely defeats the purpose of the optimization in this case.