Skip to content

RTS Linker does not handle initialization of cyclic objects correctly

While working on #21068 (closed) I stumbled across a subtle bug in the RTS linker's current treatment of initializers. Specifically, we currently mark a loaded object's text as executable (e.g. by flushing its m32 allocator) in ocTryLoad, which is the same function which is responsible for running initializers. This can be problematic in the case where we have a situation where a static library contains a set of recursive objects:

  • object A has depends upon symbols in object B
  • object B has an initializer that depends upon object A
  • we try to load object A

The linker will:

  1. start resolving object A
  2. encounter the reference to object B, loading it
  3. resolve object B
  4. run object B's initializer
  5. the initializer will attempt to call into object A, which hasn't been fully resolved (and therefore protected)

This situation results from the handling of C stubs described in !7528 (closed). The solution here is to introduce a new phase of linking, "initialization", after resolution.

Edited by Ben Gamari
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information