Editorial suggestion in section 8.8.5.3

[nanhu]

In section 8.8.5.3:

1. The variable applyAlfLineBufBoundary is not used, it is suggested to remove this variable in this section.
2. In equation (1456), ac[ x >> 2 ][ y >> 2 ] is either 64 or 96 (multiples of 32), multiplying ac[ x >> 2 ][ y >> 2 ] and then doing right shift is redundant.

Suggest to edit vE as below:

– The variables applyAlfLineBufBoundary, minY, maxY, and ac[ x ][ y ] are derived as follows:
– If y4 is equal to ( CtbSizeY − 8 ) and one of the following condition is true, minY is set equal to −2, maxY is set equal to 3, and ac[ x ][ y ] is set equal to 3.
– The bottom boundary of the current coding tree block is the bottom boundary of the picture and pic_height_in_luma_samples − yCtb > CtbSizeY − 4.
– The bottom boundary of the current coding tree block is not the bottom boudary of the picture.
– Otherwise, if y4 is equal to ( CtbSizeY − 4 ) and one of the following condition is true, minY is set equal to 0, maxY is set equal to 5, and ac[ x ][ y ] is set equal to 3.
– The bottom boundary of the current coding tree block is the bottom boundary of the picture and pic_height_in_luma_samples − yCtb > CtbSizeY − 4.
– The bottom boundary of the current coding tree block is not the bottom boudary of the picture
– Otherwise, minY is set equal to −2 and maxY is set equal to 5, and ac[ x ][ y ] is set equal to 2.

(1456):
avgVar[ x ][ y ] = varTab[ Clip3( 0, 15,

( sumOfHV[ x >> 2 ][ y >> 2 ] * ac[ x >> 2 ][ y >> 2 ] ) >> ( BitDepth - 1 ) ) ]

[bbross]

Thanks for reporting.

This has been fixed in JVET-Q2001-v8.