Support for external build system from MkDepend mode, i.e. ghc -M or equivalent
Motivation
GHC MkDepend mode analyzes the source structure and dependencies, and generates Makefile. However, it is currently (and for a long time) appending very rudimentary artifact file-level dependency graph serialized as creating (or appending if exists) Makefile.
The resultant dependency graph is very useful for many build-related tasks and it has many contextual information. Therefore, making this output in a structured format with stable schema will be very helpful.
One immediate practical use case would be a big Haskell project with bazel build system [1,2]. With a tool called gazelle and haskell plugin gazelle_haskell_modules [3], such a build system automatically generates module-level Haskell dependencies, but at this moment, it internally sets up a temporary GHC session and calls GHC API directly to get such information [4], which makes the maintenance and upgrade of such tools very hard. So we want, for example, to get those module graph dependencies in json format and also get some additional analysis results like whether a module uses Template Haskell, from the output of CLI call of ghc -M (or a new flag)
Certainly, this information should not be tied to any specific build system. the same information can be useful for buck2 or even cabal (if cabal supports different mode of compilation using GHC one-shot mode for example).
In addition, as a prospective improvement out of the new serialized formats of dependency analysis, GHC can also make use of the cached result of the analysis in a separate file and bypass the downsweep step of compilation to save time. The dependency analysis overhead can be significant in a project with more than a few thousands of modules in a single CompManager mode compilation, this will benefit many big project development.
Proposal
From simple CLI invocation with additional flags accompanied with ghc -M, GHC generates output files in a structured format with well-defined schema containing the ModGraph dependency info and file-level dependency chains, such as .hs, .hi, .hie, .o, .so/dylib and executables, with human-readable contextual information about dependency analysis, so that external build tools can make a build plan from the result and diagnose problems with understandable messages.
Immediate minimal testable use case goal would be to replace all of internal GHC API calls with external CLI invocation in gazelle-haskell-modules for generating bazel BUILD plan for Haskell projects.
This will also directly benefits the GHC project itself by further modernizing the build of GHC using this feature (of course, that task is out of the scope of this ticket, but a follow-up task).