Mismatch between the VVC spec draft and VTM: bugs in equations (335) and (345)

[afilippov]

The following two pieces of specification do not match with the VTM SW. The bug is in the negative value of x in expression "x * invAngle + 256" of equations (335) and (345)

– If intraPredAngle is less than 0, the main reference sample array is extended as follows:
ref[ x ] = p[ −1 − refIdx ][ −1 − refIdx + Min( ( x * invAngle + 256 ) >> 9, nTbH ) ],

with x = −nTbH..−1 (335)

– If intraPredAngle is less than 0, the main reference sample array is extended as follows:
ref[ x ] = p[ −1 − refIdx + Min( ( x * invAngle + 256 ) >> 9, nTbW ) ][ −1 − refIdx ],

with x = −nTbW..−1 (345)

Please, compare with VTM:

   int sizeSide = bIsModeVer ? height : width;
    for (int k = -sizeSide; k <= -1; k++)
    {
      refMain[k] = refSide[std::min((-k * invAngle + 256) >> 9, sizeSide)];
    }

Two alternative fixes are suggested:

1. Substitute "x" with "-x" inside "Min()":

– If intraPredAngle is less than 0, the main reference sample array is extended as follows:
ref[ x ] = p[ −1 − refIdx ][ −1 − refIdx + Min( ( (-x) * invAngle + 256 ) >> 9, nTbH ) ],

with x = −nTbH..−1 (335)

2. Substitute "x" with "-x" in the result indexing and negate the x range defined in "with x=..." part:

– If intraPredAngle is less than 0, the main reference sample array is extended as follows:
ref[ -x ] = p[ −1 − refIdx ][ −1 − refIdx + Min( ( x * invAngle + 256 ) >> 9, nTbH ) ],

with x = 1..nTbH (335)

[fbossen]

This was previously discussed in #1057.

In the described case, the value of invAngle is negative in the spec and positive in the software, so there is no mismatch. The best thing to do here may be to rename the variable in software from invAngle to absInvAngle to avoid confusion.

[afilippov]

The best thing to do here may be to rename the variable in software from invAngle to absInvAngle to avoid confusion.

Yes, it would be good to do. Thank you!