Because of separated luma CTB and chroma CTB for I Slice, TEncCu::xCompressCU is invoked twice in TEncCu::compressCtu, the first time for the luma CTB encoding and the second time for the chroma CTB encoding. On the second invocation, four bits/distortion related values of CTU in pCtu (function parameter of TEncCu::compressCtu): m_dTotalCost, m_uiTotalDistortion, m_uiTotalBits, and m_uiNumPartition, are overwritten. It leads to the four values of a CTU are that of chroma CTB in the CTU, and that of luma CTB in the CTU are excluded since JEM-3.0. The mistake has two influences:

1. R-D related values of I slice are that of chroma components, and the slice level delta QP RDO process does wrong QP selection when DeltaQpRD is on.
2. The remaining bit number of current I frame is updated incorrectly when CTU level rate control is on. More specially, the remaining bit number of current I frame is only subtracted by chroma CTB bits.

The original code of encoding chroma CTB is as below: #if JVET_C0024_QTBT if (pCtu->getSlice()->isIntra()) {

… xCompressCU(/*long arguments*/);

} #endif A simple solution to this bug is as below: #if JVET_C0024_QTBT if (pCtu->getSlice()->isIntra()) {

… Double dLumaTotalCost = pCtu->getTotalCost(); Distortion dLumaTotalDistortion = pCtu->getTotalDistortion(); UInt uiLumaTotalBits = pCtu->getTotalBits(); UInt uiLumaTotalBins = pCtu->getTotalBins(); xCompressCU(/*long arguments*/); pCtu->getTotalCost() += dLumaTotalCost; pCtu->getTotalDistortion() += dLumaTotalDistortion; pCtu->getTotalBits() += uiLumaTotalBits; pCtu->getTotalBins() += uiLumaTotalBins;

} #endif The bug still exit in latest software version (JEM-4.1), and has no impact under CTC.

In JEM4 software, the following code in TEncSearch.cpp causes different coding performance in some particular sequences/QP when D0127 is on/off.

#if JVET_D0127_REDUNDANCY_REMOVAL

updateCandList(uiMode, uiSad, numModesForFullRD+2, uiHadModeList, CandHadList);

We found that the intra mode candidate lists in the uiHadModeList array could be diffrent from the original order if the HAD cost of two intra modes are the same. In the situation, the coding performance may be different when the NSST index is equal to 3. The situation will cause the syntax change. Suggest to correct it.

The implementations of zero-out for large transforms are not consistent in JEM-5 and earlier versions. In one place, the non-zero-out region is checked to be the top-left quarter of a block,

#if JVET_C0046_ZO_ASSERT && JVET_C0046_ZO_ASSERT_CODED_SBK_FLAG

else if ( (uiLog2BlockWidth + uiLog2BlockHeight) > TH_LOG2TBAREASIZE &&

(!pcCU->getTransformSkip(compID) && !pcCU->getCUTransquantBypass(uiAbsPartIdx) ))

{

if ( iCGPosY >= (codingParameters.heightInGroups / 2)

iCGPosX >= (codingParameters.widthInGroups / 2) )

{

coded_sbk_flag(iCGX,iCGY) shall be equal to 0 assert(0 == uiSigCoeffGroupFlag[iCGBlkPos]);

}

}

#endif

While in another place, the non-zero-out region is checked to be no larger than 32x32 block, as below

assert((posLastX < JVET_C0024_ZERO_OUT_TH) && (posLastY < JVET_C0024_ZERO_OUT_TH));

Moreover, duplicated variables are declared for the thresholds, i.e., JVET_C0046_ZERO_OUT_TH and JVET_C0024_ZERO_OUT_TH.

It is proposed to make consistent implementation (always use threshold so that the max non-zero-out region is 32x32) to avoid confusion and clean redundant macros and code. A patch is attached for the proposed changes.