Mismatch on getting the down-sampled neighbouring top luma samples in CCLM for 4:2:2

[wangyang.cs]

In JVET-P2001-vE, when sps_chroma_vertical_collocated_flag is equal to 0, the down-sampled neighbouring top luma samples in CCLM for 4:2:2 format videos are derived as follows:

8.4.5.2.13 Specification of INTRA_LT_CCLM, INTRA_L_CCLM and INTRA_T_CCLM intra prediction mode
...

-The one-dimensional filter coefficients array F1 and F2, and the 2-dimensional filter coefficients arrays F3 and F4 are specified as follows.

F1[ 0 ] = 2, F1[ 1 ] = 1 (356)
F2[ 0 ] = 1, F2[ 1 ] = 2, F2[ 2 ] = 1 (357)
F3[ i ][ j ] = F4[ i ][ j ] = 0, with i = 0..2, j = 0..2 (358)

-If both SubWidthC and SubHeightC are equal to 2, the following applies:

F1[ 0 ] = 1, F1[ 1 ] = 1 (359)
F3[ 0 ][ 1 ] = 1, F3[ 1 ][ 1 ] = 4, F3[ 2 ][ 1 ] = 1, F3[ 1 ][ 0 ] = 1, F3[ 1 ][ 2 ] = 1 (360)
F4[ 0 ][ 1 ] = 1, F4[ 1 ][ 1 ] = 2, F4[ 2 ][ 1 ] = 1 (361)
F4[ 0 ][ 2 ] = 1, F4[ 1 ][ 2 ] = 2, F4[ 2 ][ 2 ] = 1 (362)

-Otherwise, the following applies:

F3[ 1 ][ 1 ] = 8 (363)
F4[ 0 ][ 1 ] = 2, F4[ 1 ][ 1 ] = 4, F4[ 2 ][ 1 ] = 2, (364)

…

-Otherwise (sps_chroma_vertical_collocated_flag is equal to 0), the following applies:

-If x is greater than 0, the following applies:
-If bCTUboundary is equal to FALSE, the following applies:

pSelDsY[ idx ] = ( F4[ 0 ][ 1 ] * pY[ SubWidthC x − 1 ][ −2 ] +

F4[ 0 ][ 2 ] * pY[ SubWidthC * x − 1 ][ −1 ] +
F4[ 1 ][ 1 ] * pY[ SubWidthC * x ][ −2 ] + (388)
F4[ 1 ][ 2 ] * pY[ SubWidthC * x ][ −1] +
F4[ 2 ][ 1 ] * pY[ SubWidthC * x + 1 ][ −2 ] +
F4[ 2 ][ 2 ] * pY[ SubWidthC * x + 1 ][ −1 ] + 4 ) >> 3

-Otherwise (bCTUboundary is equal to TRUE), the following applies:

pSelDsY[ idx ] = ( F2[ 0 ] * pY[ SubWidthC * x − 1 ][ −1 ] +

F2[ 1 ] * pY[ SubWidthC * x ][ −1 ] + (389)
F2[ 2 ] * pY[ SubWidthC * x + 1][ −1 ] + 2 ) >> 2

-Otherwise (x is equal to 0), the following applies:

-If availTL is equal to TRUE and bCTUboundary is equal to FALSE, the following applies:

pSelDsY[ idx ] = ( F4[ 0 ][ 1 ] * pY[ −1 ][ −2 ] + F4[ 0 ][ 2 ] * pY[ −1 ][ −1 ] +

F4[ 1 ][ 1 ] * pY[ 0 ][ −2 ] + F4[ 1 ][ 2 ] * pY[ 0 ][ −1 ] + F4[ 2 ][ 1 ] * pY[ 1 ][ −2 ] + F4[ 2 ][ 2 ] * pY[ 1 ][ −1 ] + 4 ) >> 3

(390)
…

The 2nd line above current block is utilized to get the down-sampled neighbouring top luma samples in (388) and (390).
However, the 1st line is utilized in VTM7.0.

In void IntraPrediction::xGetLumaRecPixels()
…

switch (pu.chromaFormat)
{

case CHROMA_422: overwrite filter coefficient values for 422

c0_2tap = 1, c1_2tap = 0, offset_2tap = 0, shift_2tap = 0; sum = 1
c0_3tap = 2, c1_3tap = 1, c2_3tap = 1, offset_3tap = 2, shift_3tap = 2; sum = 4
c0_5tap = 0, c1_5tap = 1, c2_5tap = 0, c3_5tap = 0, c4_5tap = 0, offset_5tap = 0, shift_5tap = 0; sum = 1
c0_6tap = 2, c1_6tap = 1, c2_6tap = 1, c3_6tap = 0, c4_6tap = 0, c5_6tap = 0, offset_6tap = 2, shift_6tap = 2; sum = 4
break;

…

}

…

if( bAboveAvaillable )
{

…

for (int i = 0; i < uiCWidth + addedAboveRight; i++)
{

…

else
{

piSrc = pRecSrc0 - iRecStride2;

if ((i == 0 && !bLeftAvaillable)

(i == uiCWidth + addedAboveRight - 1 + logSubWidthC))

{

pDst[i] = (piSrc[mult * i] * c0_2tap + piSrc[mult * i + strOffset] * c1_2tap + offset_2tap) >> shift_2tap;

}

else
{

pDst[i] = ((piSrc[mult * i] * c0_6tap + piSrc[mult * i - 1] * c1_6tap + piSrc[mult * i + 1] * c2_6tap)

+ (piSrc[mult * i + strOffset] * c3_6tap + piSrc[mult * i - 1 + strOffset] * c4_6tap + piSrc[mult * i + 1 + strOffset] * c5_6tap)
+ offset_6tap) >> shift_6tap;

}

}

}

}

Suggested fix is to align getting the down-sampled neighbouring top luma samples for CCLM in spec with VTM.
The modified draft is described as follows:

8.4.5.2.13 Specification of INTRA_LT_CCLM, INTRA_L_CCLM and INTRA_T_CCLM intra prediction mode
...

-The one-dimensional filter coefficients array F1 and F2, and the 2-dimensional filter coefficients arrays F3 and F4 are specified as follows.

F1[ 0 ] = 2, F1[ 1 ] = 1 (356)
F2[ 0 ] = 1, F2[ 1 ] = 2, F2[ 2 ] = 1 (357)
F3[ i ][ j ] = F4[ i ][ j ] = 0, with i = 0..2, j = 0..2 (358)

-If both SubWidthC and SubHeightC are equal to 2, the following applies:

F1[ 0 ] = 1, F1[ 1 ] = 1 (359)
F3[ 0 ][ 1 ] = 1, F3[ 1 ][ 1 ] = 4, F3[ 2 ][ 1 ] = 1, F3[ 1 ][ 0 ] = 1, F3[ 1 ][ 2 ] = 1 (360)
F4[ 0 ][ 1 ] = 1, F4[ 1 ][ 1 ] = 2, F4[ 2 ][ 1 ] = 1 (361)
F4[ 0 ][ 2 ] = 1, F4[ 1 ][ 2 ] = 2, F4[ 2 ][ 2 ] = 1 (362)

-Otherwise, the following applies:

F3[ 1 ][ 1 ] = 8 (363)
F4[ 0 ][ 1 ] = 2, F4[ 1 ][ 1 ] = 4, F4[ 2 ][ 1 ] = 2, (364)

…

-Otherwise (sps_chroma_vertical_collocated_flag is equal to 0), the following applies:

-If x is greater than 0, the following applies:
-If bCTUboundary is equal to FALSE, the following applies:

pSelDsY[ idx ] = ( F4[ 0 ][ 1 ] * pY[ SubWidthC x − 1 ][ −21 ] +

F4[ 0 ][ 2 ] * pY[ SubWidthC * x − 1 ][ −12 ] +
F4[ 1 ][ 1 ] * pY[ SubWidthC * x ][ −21 ] + (388)
F4[ 1 ][ 2 ] * pY[ SubWidthC * x ][ −12] +
F4[ 2 ][ 1 ] * pY[ SubWidthC * x + 1 ][ −21 ] +
F4[ 2 ][ 2 ] * pY[ SubWidthC * x + 1 ][ −12 ] + 4 ) >> 3

-Otherwise (bCTUboundary is equal to TRUE), the following applies:

pSelDsY[ idx ] = ( F2[ 0 ] * pY[ SubWidthC * x − 1 ][ −1 ] +

F2[ 1 ] * pY[ SubWidthC * x ][ −1 ] + (389)
F2[ 2 ] * pY[ SubWidthC * x + 1][ −1 ] + 2 ) >> 2

-Otherwise (x is equal to 0), the following applies:

-If availTL is equal to TRUE and bCTUboundary is equal to FALSE, the following applies:

pSelDsY[ idx ] = ( F4[ 0 ][ 1 ] * pY[ −1 ][ −21 ] + F4[ 0 ][ 2 ] * pY[ −1 ][ −12 ] +

F4[ 1 ][ 1 ] * pY[ 0 ][ −21 ] + F4[ 1 ][ 2 ] * pY[ 0 ][ −12 ] + F4[ 2 ][ 1 ] * pY[ 1 ][ −21 ] + F4[ 2 ][ 2 ] * pY[ 1 ][ −12 ] + 4 ) >> 3

(390) )
…

[bbross]

Thanks for reporting!

This is fixed in JVET-Q2001-v3.