Consider reverting "Use fix-sized arithmetic primops for fixed size boxed types"

We reverted !9920 (closed) and !9904 (closed) because they caused perf regressions through partial register stalls.

I assume !5572 (closed) can cause regressions for much the same reason and therefore should also be reverted or changed along the lines which #22233 lays out.