Opened 5 years ago

Closed 5 years ago

#254 closed defect (fixed)

Dead loop in reshaper code

Reported by: XiangLi Owned by:
Priority: minor Milestone:
Component: VTM Version: VTM-5.0rc1
Keywords: Cc: ksuehring, XiangLi, fbossen, jvet@…


In function void EncReshape::constructReshaperSDR(), following code sometimes lead to dead loop

else if (resCW > (cwReduce1 + cwReduce2))

resCW -= (cwReduce1 + cwReduce2);
int idx = 0;
while (resCW > 0)

if (m_binCW[idx] > 0 && m_binCW[idx] < (histLenth + 1))


if (idx == histBins)

idx = 0;


The issue may be duplicated with following command line and attached YUV file. Note the resolution is 352x288.

Encoder.exe -c ..\cfg\encoder_randomaccess_vtm.cfg -c ..\cfg\per-class\classF.cfg -c SlideShow_cif.cfg -v 6 -dph 1 -f 17 -q 37 -ip 16 -i ..\..\..\Sequences\SlideShow_#160-#180_352x288_25Hz_8bit_P420.yuv

Attachments (1)

SlideShow_#160-#180_352x288_25Hz_8bit_P420.7z (200.6 KB) - added by XiangLi 5 years ago.
YUV clip

Download all attachments as: .zip

Change history (4)

Changed 5 years ago by XiangLi

YUV clip

comment:1 Changed 5 years ago by taoranlu

Hi Xiang,
Ticket 231 is reporting same issue in nonCTC case:

We are working on the fix and will provide MR soon.

comment:2 Changed 5 years ago by XiangLi

A suggested fix has been provided by proponent in

comment:3 Changed 5 years ago by XiangLi

  • Resolution set to fixed
  • Status changed from new to closed


Note: See TracTickets for help on using tickets.