Possible mismatch spec/VTM CC-ALF

[sdchi]

I believe there is mismatch between the spec and VTM regarding the CC ALF. The best match I could find for statement 1506 in the spec:

sum = ( SubHeightC == 1 && ( y == CtbSizeY − 3 | | y == CtbSizeY − 4 ) ) ? curr : curr + scaledSum (1506)

is at AdaptiveLoopFilter.cpp:1340 in VTM:

int pos = ((startHeight + i + ii) << scaleY) & (vbCTUHeight - 1);
if (scaleY == 0 && (pos == vbPos || pos == vbPos + 1))
{
  continue;
}

I assume that y is actually yL, and this is a typo. But even with this the SubHeightC and scaleY do not match. In the spec it is skipped for 420, but in the software for 422/444. The spec behavior would make more sense for me.

As a side note, the y == CtbSizeY − 3 is a bit redundant as this does not occur with SubHeightC == 1.

[fbossen]

Not seeing a mismatch or typo here.
The use of y appears correct in the spec. Using yL would be incorrect (because it is a global position), as would be removing the y == CtbSizeY - 3 condition (it does occur when SubHeightC is equal to 1).

Note that for 422 and 444 formats SubHeightC is equal to 1 and scaleY is equal to 0. For the 420 format SubHeightC is equal to 2 and scaleY is equal to 1.

"Skipping" is meant to occur only when SubHeightC is equal to 1 (422 and 444 case) and processing the 3rd and 4th line above the bottom boundary of a CTU.

[sdchi]

I see, thank you for the clarification. I mixed up the meaning of scaleY and SubHeightC (shift, divisor), and am thrown off a bit by the irregular border rules of CC-ALF.

What I meant with y is that is a chroma coordinate and it is compared to CtbSizeY which is a luma block size. Should this not be y * subHeightC then?

[sdchi]

Nevermind, as SubHeightC is equal to 1 it would not matter.

[fbossen]

Indeed, it doesn't matter whether you use y or y * subHeightC here. The end result is the same.