So now

  :load foo.hs       loads bytecode for foo.hs, even if foo.o exists
  :load foo          is just shorthand for :load foo.hs
  :load M            loads a module M, as object code if possible
                     (no change here)

  :set -fobject-code
  :load foo.hs       loads foo.hs as object code; an existing foo.o
                     can be used.
  
This turned out to be very straightforward: when building the
ModSummary for a file (summariseFile) we just ignore the object file
unless -fobject-code is on.