Opened 8 years ago
Closed 8 years ago
#42 closed defect (fixed)
CTU bits/distortion calculation mistake for I slice when QTBT is enabled
Reported by: | FangliangSong | Owned by: | |
---|---|---|---|
Priority: | minor | Milestone: | HM-16.6-JEM-4.2 |
Component: | JEM | Version: | HM-16.6-JEM-4.0 |
Keywords: | CTU, bits, distortion, I slice, QTBT | Cc: | ksuehring, XiangLi, jvet@… |
Description
Because of separated luma CTB and chroma CTB for I Slice, TEncCu::xCompressCU is invoked twice in TEncCu::compressCtu, the first time for the luma CTB encoding and the second time for the chroma CTB encoding. On the second invocation, four bits/distortion related values of CTU in pCtu (function parameter of TEncCu::compressCtu): m_dTotalCost, m_uiTotalDistortion, m_uiTotalBits, and m_uiNumPartition, are overwritten. It leads to the four values of a CTU are that of chroma CTB in the CTU, and that of luma CTB in the CTU are excluded since JEM-3.0. The mistake has two influences:
- R-D related values of I slice are that of chroma components, and the slice level delta QP RDO process does wrong QP selection when DeltaQpRD is on.
- The remaining bit number of current I frame is updated incorrectly when CTU level rate control is on. More specially, the remaining bit number of current I frame is only subtracted by chroma CTB bits.
The original code of encoding chroma CTB is as below:
#if JVET_C0024_QTBT
if (pCtu->getSlice()->isIntra())
{
…
xCompressCU(/*long arguments*/);
}
#endif
A simple solution to this bug is as below:
#if JVET_C0024_QTBT
if (pCtu->getSlice()->isIntra())
{
…
Double dLumaTotalCost = pCtu->getTotalCost();
Distortion dLumaTotalDistortion = pCtu->getTotalDistortion();
UInt uiLumaTotalBits = pCtu->getTotalBits();
UInt uiLumaTotalBins = pCtu->getTotalBins();
xCompressCU(/*long arguments*/);
pCtu->getTotalCost() += dLumaTotalCost;
pCtu->getTotalDistortion() += dLumaTotalDistortion;
pCtu->getTotalBits() += uiLumaTotalBits;
pCtu->getTotalBins() += uiLumaTotalBins;
}
#endif
The bug still exit in latest software version (JEM-4.1), and has no impact under CTC.
Attachments (2)
Change history (4)
comment:1 Changed 8 years ago by FangliangSong
Changed 8 years ago by FangliangSong
Changed 8 years ago by FangliangSong
replace variable name "dLumaCTBTotalDistortion" with "uiLumaCTBTotalDistortion"
comment:2 Changed 8 years ago by XiangLi
- Milestone set to HM-16.6-JEM-4.2
- Resolution set to fixed
- Status changed from new to closed
Fixed in r404.
Four related variables is TComDataCU::m_dTotalCost, TComDataCU::m_uiTotalDistortion, TComDataCU::m_uiTotalBits, and TComDataCU::m_uiTotalBins. "m_uiNumPartition" in above description should be m_uiTotalBins. I'm sorry for miswriting it.