Mismatch spec/VTM in PDPC process for wT and wL calculation

[audrey.turquin]

In VTM, in IntraPrediction.cpp, in function predIntraAng, there is the following code :

  if (m_ipaParam.applyPDPC)                                                                                                                                                                                                                  
  {                                                                                                                                                                                                                                          
    PelBuf dstBuf = piPred;                                                                                                                                                                                                                  
    const int scale = ((floorLog2(iWidth) - 2 + floorLog2(iHeight) - 2 + 2) >> 2);                                                                                                                                                           
    CHECK(scale < 0 || scale > 31, "PDPC: scale < 0 || scale > 31");                                                                                                                                                                         
                                                                                                                                                                                                                                             
    if (uiDirMode == PLANAR_IDX || uiDirMode == DC_IDX)                                                                                                                                                                                      
    {                                                                                                                                                                                                                                        
      for (int y = 0; y < iHeight; y++)                                                                                                                                                                                                      
      {                                                                                                                                                                                                                                      
        const int wT   = 32 >> std::min(31, ((y << 1) >> scale));                                                                                                                                                                            
        const Pel left = srcBuf.at(y + 1, 1);                                                                                                                                                                                                
        for (int x = 0; x < iWidth; x++)                                                                                                                                                                                                     
        {                                                                                                                                                                                                                                    
          const int wL    = 32 >> std::min(31, ((x << 1) >> scale));                                                                                                                                                                         
          const Pel top   = srcBuf.at(x + 1, 0);                                                                                                                                                                                             
          const Pel val   = dstBuf.at(x, y);                                                                                                                                                                                                 
          dstBuf.at(x, y) = val + ((wL * (left - val) + wT * (top - val) + 32) >> 6);                                                                                                                                                        
        }                                                                                                                                                                                                                                    
      }                                                                                                                                                                                                                                      
    }                                                                                                                                                                                                                                        
  }

The calculation of wT and wL are not aligned with the one in the specification.
In the spec, in section 8.4.5.2.14, part where predModeIntra is equal to INTRA_PLANAR or INTRA_DC, it is :

wT[ y ] = 32 >> ( ( y << 1 ) >> nScale )
wL[ x ] = 32 >> ( ( x << 1 ) >> nScale )

[geertv]

The right shift >> std::min(31,...) is a software implementation issue with >> operator having undefined behavior if value is > 31 for integer type. It is nonnormative.

[audrey.turquin]

Ok, thanks for the clarification.