Revert "Refactor SpecConstr to use treat bindings uniformly"
This reverts commit 415468fe.
This refactoring introduced quite a severe residency regression (900MB live from 650MB live when compiling mmark), see #21993 (closed) for a reproducer and more discussion.
Ticket #21993 (closed)