Bug in down-sampled collocated luma samples derivation in CCLM

[lphamvan]

- In the current spec, the first row and first column of luma reference block is not assigned in YUV444.

3. The down-sampled collocated luma samples pDsY[ x ][ y ] with x = 0..nTbW − 1, y = 0..nTbH − 1 are derived as follows:

– If both SubWidthC and SubHeightC are equal to 1, the following applies:
– pDsY[ x ][ y ] with x = 1..nTbW − 1, y = 1..nTbH − 1 is derived as follows:
pDstY[ x ][ y ] = pY[ x ][ y ] (355)

It is suggested to fix as:
pDsY[ x ][ y ] with x = 10..nTbW − 1, y = 10..nTbH − 1 is derived as follows:
pDstY[ x ][ y ] = pY[ x ][ y ] (355)

The index in (382) and (384) is incorrect.

pSelDsY[ idx ] = ( F3[ 1 ][ 0 ] * pY[ −1 ][ − 1 − SubHeightC ] + ...

In YUV420, − 1 − SubHeightC = -1 - 2 = -3. However, the line -3 of pSelDsY is never assigned.

It is suggested to fix this bug also.

[bbross]

Thanks for reporting!

The first issue has been solved in JVET-Q2001-v3.

Does the second issue still exists?
Please confirm otherwise I would close this ticket.

[lphamvan]

Hi,

I believe the bug has not been solved. I will provide a fix my earliest convenience.

Best regards,
Luong

[lphamvan]

Hi Benjamin,

As I checked in JVET-Q2001-vB, the first issue has not been solved.

To fix the second issue, the line with y = -3 should be set by modifying the following text:

– When numSampT is greater than 0, the neighbouring top luma samples pY[ x ][ y ] with x = 0..SubWidthC * numSampT − 1, y = −1, −23, are set equal to the reconstructed luma samples prior to the deblocking filter process at the locations ( xTbY+ x, yTbY + y ).

In the attached file, I provide a revised version of section 8.4.5.2.13 with the following change:

• Remove F1 definition since it is not used.
• Move the definition of filter coefficients to the outside of section 3, because the coefficients is used in other sections also.
• Revise section 5, If bCTUboundary is equal to TRUE, the filter is the same whatever sps_chroma_vertical_collocated_flag is equal 0 or 1.

Please take a look at the attachment to see if it is necessary.

Thanks,

Best regards,
Luong

[bbross]

Good catch, thanks!

This is fixed in vB of D9, ie. JVET-R2001-v11.