Fix literals for unregisterized backend of small types

All credit to @hsyl20, who in !4717 (comment 338560) figured out this was a problem.

To fix this, we use casts instead of suffixes. It might make for more verbose code, I don't think that matters too muc

Progress towards #19026 (closed)