Dead loop in reshaper code

[XiangLi]

In function void EncReshape::constructReshaperSDR(), following code sometimes lead to dead loop

else if (resCW > (cwReduce1 + cwReduce2))
{

resCW -= (cwReduce1 + cwReduce2);
int idx = 0;
while (resCW > 0)
{

if (m_binCW[idx] > 0 && m_binCW[idx] < (histLenth + 1))
{

m_binCW[idx]--;
resCW--;

}
idx++;
if (idx == histBins)

idx = 0;

}

The issue may be duplicated with following command line and attached YUV file. Note the resolution is 352x288.

Encoder.exe -c ..\cfg\encoder_randomaccess_vtm.cfg -c ..\cfg\per-class\classF.cfg -c SlideShow_cif.cfg -v 6 -dph 1 -f 17 -q 37 -ip 16 -i ..\..\..\Sequences\SlideShow_#160-#180_352x288_25Hz_8bit_P420.yuv

[XiangLi]

YUV clip

[taoranlu]

Hi Xiang,
Ticket 231 is reporting same issue in nonCTC case: https://jvet.hhi.fraunhofer.de/trac/vvc/ticket/231

We are working on the fix and will provide MR soon.

[XiangLi]

A suggested fix has been provided by proponent in ​https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/merge_requests/534.

[XiangLi]

Fixed.