Opened 4 years ago

Closed 4 years ago

#44 closed defect (fixed)

Inconsistent implementation of zero-out for large transforms

Reported by: XiangLi Owned by:
Priority: minor Milestone: HM-16.6-JEM-5.1
Component: JEM Version: HM-16.6-JEM-5.0
Keywords: Cc: cjianle@…, H.Huang@…, Ohji.Nakagami@…, Takeshi.Tsukuba@…, vzakharc, yuwenhe, jvet@…

Description

The implementations of zero-out for large transforms are not consistent in JEM-5 and earlier versions. In one place, the non-zero-out region is checked to be the top-left quarter of a block,

#if JVET_C0046_ZO_ASSERT && JVET_C0046_ZO_ASSERT_CODED_SBK_FLAG

else if ( (uiLog2BlockWidth + uiLog2BlockHeight) > TH_LOG2TBAREASIZE &&

(!pcCU->getTransformSkip(compID) && !pcCU->getCUTransquantBypass(uiAbsPartIdx) ))

{

if ( iCGPosY >= (codingParameters.heightInGroups / 2)
iCGPosX >= (codingParameters.widthInGroups / 2) )

{

coded_sbk_flag(iCGX,iCGY) shall be equal to 0
assert(0 == uiSigCoeffGroupFlag[iCGBlkPos]);

}

}

#endif

While in another place, the non-zero-out region is checked to be no larger than 32x32 block, as below

assert((posLastX < JVET_C0024_ZERO_OUT_TH) && (posLastY < JVET_C0024_ZERO_OUT_TH));

Moreover, duplicated variables are declared for the thresholds, i.e., JVET_C0046_ZERO_OUT_TH and JVET_C0024_ZERO_OUT_TH.

It is proposed to make consistent implementation (always use threshold so that the max non-zero-out region is 32x32) to avoid confusion and clean redundant macros and code. A patch is attached for the proposed changes.

Attachments (1)

fix_zeroout_clean.patch (61.3 KB) - added by XiangLi 4 years ago.
fix and cleanning

Download all attachments as: .zip

Change history (3)

Changed 4 years ago by XiangLi

fix and cleanning

comment:1 Changed 4 years ago by XiangLi

  • Component changed from 360Lib to JEM

comment:2 Changed 4 years ago by XiangLi

  • Milestone set to HM-16.6-JEM-5.1
  • Resolution set to fixed
  • Status changed from new to closed

Fixed in r458

Note: See TracTickets for help on using tickets.