I suspect it's not about mp4 in general, since it's happening to my .flv-turned-.mp4 . To record, I used the open recording software OBS
https://obsproject.com/download .
My older videos were done in what they now call "OBS Classic", version: 0.659b . This created .flv files natively, but apparently with missing key indexes. It's in these videos that I cannot seek, even after they're converted to .mp4.
I recently obtained the newer OBS Studio, which natively creates .mp4 files with key indexes, and I have no issues seeking the generated files, even if they are big.
icy`