Opened 6 years ago

Closed 6 years ago

#44 closed defect (fixed)

Inconsistent implementation of zero-out for large transforms

Reported by: XiangLi Owned by:
Priority: minor Milestone: HM-16.6-JEM-5.1
Component: JEM Version: HM-16.6-JEM-5.0
Keywords: Cc: cjianle@…, H.Huang@…, Ohji.Nakagami@…, Takeshi.Tsukuba@…, vzakharc, yuwenhe, jvet@…


The implementations of zero-out for large transforms are not consistent in JEM-5 and earlier versions. In one place, the non-zero-out region is checked to be the top-left quarter of a block,


else if ( (uiLog2BlockWidth + uiLog2BlockHeight) > TH_LOG2TBAREASIZE &&

(!pcCU->getTransformSkip(compID) && !pcCU->getCUTransquantBypass(uiAbsPartIdx) ))


if ( iCGPosY >= (codingParameters.heightInGroups / 2)
iCGPosX >= (codingParameters.widthInGroups / 2) )


coded_sbk_flag(iCGX,iCGY) shall be equal to 0
assert(0 == uiSigCoeffGroupFlag[iCGBlkPos]);




While in another place, the non-zero-out region is checked to be no larger than 32x32 block, as below

assert((posLastX < JVET_C0024_ZERO_OUT_TH) && (posLastY < JVET_C0024_ZERO_OUT_TH));

Moreover, duplicated variables are declared for the thresholds, i.e., JVET_C0046_ZERO_OUT_TH and JVET_C0024_ZERO_OUT_TH.

It is proposed to make consistent implementation (always use threshold so that the max non-zero-out region is 32x32) to avoid confusion and clean redundant macros and code. A patch is attached for the proposed changes.

Attachments (1)

fix_zeroout_clean.patch (61.3 KB) - added by XiangLi 6 years ago.
fix and cleanning

Download all attachments as: .zip

Change history (3)

Changed 6 years ago by XiangLi

fix and cleanning

comment:1 Changed 6 years ago by XiangLi

  • Component changed from 360Lib to JEM

comment:2 Changed 6 years ago by XiangLi

  • Milestone set to HM-16.6-JEM-5.1
  • Resolution set to fixed
  • Status changed from new to closed

Fixed in r458

Note: See TracTickets for help on using tickets.