WIP: Make threaded RTS the default (#16126)

This makes -threaded the default and adds -non-threaded for the case when there is a need to rollback, as suggested in #16126. I think this should be considered as WIP but I'd like to have some feedback.

Questions

Should we fix everything concerning -threaded in the GHC source code to reflect switching the default? Or can we go gradually?

Conceptually, I think, there are three sources that have to deal with the switch.

• API documentation, especially in base -- I haven't look into this so far.

• Tests -- I tried to fix some, but there are some failing, and also I'm not sure my fixes make sense. See below.

• GHCi -- I noticed that when dealing with the tests: GHCi warns you when it notices the threaded way. There are warnings in tests: "Warning: -debug, -threaded and -ticky are ignored by GHCi". But the warning doesn't seem to show up when GHCi is run manually -- this I don't get.

At a lower level, we might try to grep -threaded and find several sources of it in the GHC source tree, listed below in the order of decreasing confidence as to what to do:

• docs/user_guide -- (hopefully) updated.
• utils -- mostly compile-time flag. Can be easily removed, by I haven't got there as of yet.
• rts -- mostly make sence after switching the default, I think, so leaving as is.
• nofib -- sometimes just the compile-time flag, but many times some notes on single-vs-multi-threaded performance. Haven't analysed that.
• libraries -- comes in two flavors: haddocks and compile-time flag. I'm unsure about weather we should update everything here. Certainly not submodules? Updating only base might be a reasonable start.
• testsuite -- mostly compile-time flags.