It is not supposed to be done outside lavc.
This is basically a revert of 818062f2f346df30f4ec0c0c1f54e8025cc3a80a.
It is unclear what issue this was supposed to fix, if it reappears again
it will have to be fixed in a more proper place.
The wtv-demux test change is because the sample starts with a B-frame.