Exception thrown in 'estIntraPredChromaQT' function

[fbarbier]

The following encoding test is failing :

git checkout 312ae1ef406ca37a545dafcb6019487acfa112dc
cd build
cmake ..
make -j 8
cd ..

bin/EncoderAppStatic --SourceWidth=120 --SourceHeight=88 --InputChromaFormat=420 --InputBitDepth=10 --Profile=next --FrameRate=60 --FramesToBeEncoded=2 --GOPSize=4 '--Frame1=B 1 5 -6.5 0.2590 0 0 1.0 0 0 0 4 4 -1 -5 -9 -13 0' '--Frame2=B 2 4 -6.5 0.2590 0 0 1.0 0 0 0 4 4 -1 -2 -6 -10 1 -1 5 1 1 1 0 1' '--Frame3=B 3 5 -6.5 0.2590 0 0 1.0 0 0 0 4 4 -1 -3 -7 -11 1 -1 5 0 1 1 1 1' '--Frame4=B 4 1 0.0 0.0 0 0 1.0 0 0 0 4 4 -1 -4 -8 -12 1 -1 5 0 1 1 1 1' --FastSearch=2 --MaxCUWidth=24 --MaxCUHeight=12 --MaxPartitionDepth=5 --TULog2MaxSize=6 --PCMEnabledFlag=1 --DisableIntraInInter=0 -i ./source.yuv -b ./encoded.vvc

terminate called after throwing an instance of 'Exception'

what():

ERROR: In function "estIntraPredChromaQT" in ./source/Lib/EncoderLib/IntraSearch.cpp:1070: bestCostSoFar must be positive!

/source/Lib/EncoderLib/IntraSearch.cpp:1070: bestCostSoFar must be positive!

[ksuehring]

A fix is submitted as MR !342

​https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/merge_requests/342

[forayr]

There is an issue with this fix when qtbtt_dual_tree_intra_flag = 1 and we are inside a coding_unit with treeMode = DUAL_TREE_CHROMA.
Indeed, in this case, cu.lwidth() and cu.lheight() return 0 (as the value is taken from Y()).
This cause a mismatch with the specification where cbWidth and cbHeight are used.

[forayr]

The regression is fixed in bb3c8389c6b6d1f1ad3db20fc0889ae6c7910334.

But the original issue is still present in 09cc47722f31884118de830ee9aaede29637436a:

git checkout 09cc47722f31884118de830ee9aaede29637436a
cd build
cmake ..
make -j 8
cd ..
bin/EncoderAppStatic --SourceWidth=112 --SourceHeight=104 --InputChromaFormat=420 --InputBitDepth=8 --Profile=next --FrameRate=60 --FramesToBeEncoded=3 --GOPSize=4 '--Frame1=B 1 5 -6.5 0.2590 0 0 1.0 0 0 0 4 4 -1 -5 -9 -13 0' '--Frame2=B 2 4 -6.5 0.2590 0 0 1.0 0 0 0 4 4 -1 -2 -6 -10 1 -1 5 1 1 1 0 1' '--Frame3=B 3 5 -6.5 0.2590 0 0 1.0 0 0 0 4 4 -1 -3 -7 -11 1 -1 5 0 1 1 1 1' '--Frame4=B 4 1 0.0 0.0 0 0 1.0 0 0 0 4 4 -1 -4 -8 -12 1 -1 5 0 1 1 1 1' --FastSearch=1 --MaxCUWidth=24 --MaxCUHeight=12 --MaxPartitionDepth=5 --PCMEnabledFlag=1 --SAO=1 --LoopFilterDisable=1 --ALF=0 --DisableIntraInInter=1 --LCTUFast=1 --DepQuant=1 --DualITree=0 --LMChroma=0 --TransformSkip=1 --MaxDeltaQP=2 --MTS=0 --IBC=1 --IMV=0 -i ./source2.yuv -b ./encoded.vvc

ERROR: In function "estIntraPredChromaQT" in /home/remyf/work/jvet/vtm2/source/Lib/EncoderLib/IntraSearch.cpp:895: bestCostSoFar must be positive!