Encoder fails to start encoding under CTC for high bit-depth

[tsukuba.takeshi]

The encoder of VTM-14.0rc fails to start encoding under CTC for high bit-depth using the following command:

EncoderAppStatic -c cfg/encoder_intra_vtm.cfg -q -13 -i PQ.BalloonFestival_422.yuv -b str.bin -o rec.yuv --InputChromaFormat=422 -fr 24 -wdt 1920 -hgt 1080 --Level=4.1 -f 16 -ip 1 --SEIDecodedPictureHash=1 --PrintHexPSNR=1 -c cfg/per-class/classH1.cfg -c cfg/hbd/hbd.cfg -c cfg/hbd/12-bit.cfg

When a macro JVET_W2005_RANGE_EXTENSION_PROFILES is turned off, encoding starts.

[ksuehring]

I submitted !2116 to fix a null pointer dereference in the case that no profile is set (Profile=none)

​https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/merge_requests/2116

This should fix the command line above. I could not fully test because I didn't have the 422 input file locally.

But additionally the high bit depth CTC config files should set a proper profile. I can create a patch, but would need some input what profiles would be most suitable.

[tsukuba.takeshi]

Thanks a lot. The above fix works well for the high bit-depth CTC.

[Tomohiro Ikai]

Thank you very much for the report and fix. I had tested it with --Profile=auto but not with --Profile=none (my failure).

But additionally the high bit depth CTC config files should set a proper profile.

--Profile=auto should set appropriate profiles.

I believe,
One option is to include --Profile=auto in hbd.cfg.
the other option would be --Profile=main_12_444 --GeneralLowerBitRateConstraintFlag=1 in 12-bit.cfg and --Profile=main_16_444 --GeneralLowerBitRateConstraintFlag=1 in 16-bit.cfg.

Note: --GeneralLowerBitRateConstraintFlag=0 is not allowed for non Intra profiles.

[Tomohiro Ikai]

It seems VTM cfg files use Profile:auto as default. Thus I changed Profile setting from none to auto in hbd.cfg and created a merge request (the first option in the above comment).

Note: HM uses "Profile : main-RExt" but "constraint flags are either manually specified, or calculated via the other supplied settings." So it can be said specific profiles are to set auto in HM too.

[ksuehring]

Profile setting merged as suggested in !2117

​https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/merge_requests/2117