Opened 4 years ago
Closed 4 years ago
#1100 closed defect (invalid)
Mismatch spec/VTM in PDPC process for wT and wL calculation
Reported by: | audrey.turquin | Owned by: | |
---|---|---|---|
Priority: | minor | Milestone: | |
Component: | VTM | Version: | |
Keywords: | Cc: | ksuehring, XiangLi, fbossen, jvet@… |
Description
In VTM, in IntraPrediction.cpp, in function predIntraAng, there is the following code :
if (m_ipaParam.applyPDPC) { PelBuf dstBuf = piPred; const int scale = ((floorLog2(iWidth) - 2 + floorLog2(iHeight) - 2 + 2) >> 2); CHECK(scale < 0 || scale > 31, "PDPC: scale < 0 || scale > 31"); if (uiDirMode == PLANAR_IDX || uiDirMode == DC_IDX) { for (int y = 0; y < iHeight; y++) { const int wT = 32 >> std::min(31, ((y << 1) >> scale)); const Pel left = srcBuf.at(y + 1, 1); for (int x = 0; x < iWidth; x++) { const int wL = 32 >> std::min(31, ((x << 1) >> scale)); const Pel top = srcBuf.at(x + 1, 0); const Pel val = dstBuf.at(x, y); dstBuf.at(x, y) = val + ((wL * (left - val) + wT * (top - val) + 32) >> 6); } } } }
The calculation of wT and wL are not aligned with the one in the specification.
In the spec, in section 8.4.5.2.14, part where predModeIntra is equal to INTRA_PLANAR or INTRA_DC, it is :
wT[ y ] = 32 >> ( ( y << 1 ) >> nScale )
wL[ x ] = 32 >> ( ( x << 1 ) >> nScale )
Change history (2)
comment:1 Changed 4 years ago by geertv
comment:2 Changed 4 years ago by audrey.turquin
- Resolution set to invalid
- Status changed from new to closed
Ok, thanks for the clarification.
Note: See TracTickets for help on using tickets.
The right shift >> std::min(31,...) is a software implementation issue with >> operator having undefined behavior if value is > 31 for integer type. It is nonnormative.