This patch fixes a long-standing lurking bug, but it surfaced when I
was adding specialisation for imported Ids.

See Note [ImpRuleUsage], which explains the issue.   The solution
seems more complicated than the problem really deserves, but I
could not think of a simpler way, so I just bit the bullet and
wrote the code.  Improvements welcome.