Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#36 closed defect (fixed)

Floating-point QP not working

Reported by: yyasugi Owned by:
Priority: minor Milestone: HM-16.6-JEM-4.1
Component: JEM Version: HM-16.6-JEM-4.0
Keywords: Cc: ksuehring, XiangLi, jvet@…


Version: HM-16.6-JEM-4.0

I noticed encoding failed with floating point QP in most cases.
The encoder was terminated by accessing an array with index -1.

It would seem that NUM_QP_PROB is not enough and TComStats::aaQPUsed runs out of space during encoding.

The issue seems to be fixed by increasing JVET_B0039_INC_NUM_QP_PROB from 7 to 14 as its comment says, but I'm not sure what value is appropriate.

Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00000000004b5b36 in TEncSbac::loadContextsFromPrev (this=0x7ffe173f3980, apcStats=0x83b8670, eSliceType=B_SLICE, iQPIdx=-1, bFromGloble=false, iQPIdxRst=-1, bAfterLastISlice=false)

at /.../source/Lib/TLibEncoder/TEncSbac.cpp:3273

#1 0x0000000000456a2e in TEncSlice::encodeSlice (this=0x7ffe173f66e0, pcPic=0x2ba23b668010, pcSubstreams=0x11902a80, numBinsCoded=@0x7ffe173d0680: 0, alfParam=...)

at /.../source/Lib/TLibEncoder/TEncSlice.cpp:1290

Attachments (1)

Fix_Ticket36.patch (2.2 KB) - added by lizhang 4 years ago.

Download all attachments as: .zip

Change history (5)

comment:1 Changed 4 years ago by sier

I notice that when we use the switch DeltaQpRD, the total number of appeared QP values will be larger than 7, and this will cause that TComStats::aaQPUsed runs out of space during encoding.

In order to solve this problem, it is necessary to take every QP value into consideration during the encoding process, especially for muliti QP optimization. Therefore, the parameter JVET_B0039_INC_NUM_QP_PROB should be set as 52.

I have tested the performance by setting JVET_B0039_INC_NUM_QP_PRO as 52,and the results show that there is no luma nor chroma BD-rate reduction over JEM-3.0 for RA cases with neglected encoding time increase.

Changed 4 years ago by lizhang

comment:2 Changed 4 years ago by lizhang

The total number of used QP values is changed to the maximum number of positive QP values allowed in HEVC, i.e., JVET_B0039_INC_NUM_QP_PRO is changed from 7 to 52. In addition, when JVET_B0039_INC_NUM_QP_PROB is not used, the macro NUM_QP_PROB is also modified to be 52 for potential non-CTC tests.

comment:3 Changed 4 years ago by XiangLi

  • Resolution set to fixed
  • Status changed from new to closed

Fixed in r354.

comment:4 Changed 4 years ago by XiangLi

  • Milestone set to HM-16.6-JEM-4.1
  • Version set to HM-16.6-JEM-4.0
Note: See TracTickets for help on using tickets.