Decoding a Y4M encoded bitstream do not retain framerate

[mindfreeze]

When we encode which is in y4m input, the frame rate and other input metadata are present in the header. This is also expected when we decode a bitstream to y4m.

Background

Currently in VVC, when we try to decode a video (bitstream) to Y4M format, it suggests there is no framerate info in the bitstream and fallback to the default 50fps. This is not the correct behaviour. If it is YUV, we don't care about the frame rate inside the decoded stream, it is not the case here.

Steps to reproduce

1. Encode a video which is in y4m

EncoderAppStatic -i FourPeople_480x270_60.y4m -c vvc-vtm/encoder_randomaccess_vtm.cfg --InputBitDepth=10 --FramesToBeEncoded=1 --QP=50 --ReconFile=Fourpeople-recon.yuv -b fourpeople.bin
The sample used in this example: ​https://media.xiph.org/video/aomctc/test_set/a5_270p/FourPeople_480x270_60.y4m

2. Decode video to y4m format

./bin/DecoderAppStatic -b fourpeople.bin -o four.y4m

3. Inspect the input headers and output headers to cross-check

SRC: head -1 /media/aomctc-a5-270p/FourPeople_480x270_60.y4m
SRC: YUV4MPEG2 W480 H270 F60:1 Ip A0:0 C420jpeg XYSCSS=420JPEG
ENC_STREAM: head -1 four.y4m
ENC_STREAM: YUV4MPEG2 W480 H270 F50:1 Ip A0:0 C420p10

Samples
Any y4m samples would be sufficient,

Workarounds
If we explicitly signal HRDParameters with SEIPictureTiming and BufferPeriod, we get the approximate frame-rate (no idea why not same for all the clips) in the decoded y4m file,
--HrdParametersPresent=1 --SEIPictureTiming=1 --SEIBufferingPeriod=1 --RCCpbSize=1000

YUV4MPEG2 W480 H270 F60000:1001 Ip A0:0 C420p10
It suggests 59.94 instead of 60fps for this sample clip.
So I do not know if this is the correct way to handle frame-rate information in VVC, I have little idea as a whole for VVC.
At the same time, for some other clips which has fractional frame rates (only tested with 2-4 clips, so not yet conclusive) it gives the correct framerate in the output y4m.
Another broken example,
Input: ​https://media.xiph.org/video/aomctc/test_set/a2_2k/MountainBike_1920x1080_30fps_8bit.y4m
Encode-settings: same as above
SRC: YUV4MPEG2 W1920 H1080 F30:1 Ip A1:1 C420jpeg XYSCSS=420JPEG
ENC_STREAM: YUV4MPEG2 W1920 H1080 F30000:1001 Ip A0:0 C420p10
Another working example,
Input: ​https://media.xiph.org/video/aomctc/test_set/a5_270p/Vertical_Bayshore_270x480_2997.y4m
SRC: YUV4MPEG2 W270 H480 F30000:1001 Ip A0:0 C420jpeg XYSCSS=420JPEG
ENC_STREAM: YUV4MPEG2 W270 H480 F30000:1001 Ip A0:0 C420p10

Question
I would assume it falls back to 420p10 due to the internal 10-bit pipeline in the configuration of VVC. Would be beneficial to have a clarification for that.

Best,
V

[fbossen]

This should include frame rate info in the bitstream:
​https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/merge_requests/2587

As for the 420p10 indicator, it is a result of the 10-bit coding pipeline. It is possible to modify the bit depth in the decoded file by using the --OutputBitDepth option in the decoder

[mindfreeze]

Hi,

This is amazing,

I tried to apply the patch locally,

For integer frame rates this works correctly, and previously it was not (only fractional framerate used to work with HRD params).
Now, I am getting denom(frameRateScale I believe) always as 1 even if it is a fractional framerate.

If you use the patch and try to encode this, ​https://media.xiph.org/video/aomctc/test_set/a5_270p/Vertical_Bayshore_270x480_2997.y4m,
for instance, we are getting,
YUV4MPEG2 W270 H480 F30000:1 Ip A0:0 C420p10

[fbossen]

My bad. I pushed an update. Thanks for sharing the 29.97Hz sequence.

[mindfreeze]

The V2 of the patchset,

On my testing, it works well for both fractional and integer framerates.

I have tested for

1. 23.98
2. 24
3. 29.97
4. 30
5. 50
6. 59.97
7. 60

I would let the experts review the rest, but this solves the problem for FPS passthrough for Y4M:).

Thanks again for the quick fix

[XiangLi]

The fix (!2587) has been merged.