Opened 3 years ago

Closed 3 years ago

#1470 closed defect (invalid)

Possible mismatch spec/VTM CC-ALF

Reported by: sdchi Owned by:
Priority: major Milestone:
Component: VTM Version: VTM-12.0
Keywords: cc alf Cc: ksuehring, XiangLi, fbossen, jvet@…


I believe there is mismatch between the spec and VTM regarding the CC ALF. The best match I could find for statement 1506 in the spec:

sum = ( SubHeightC == 1 && ( y == CtbSizeY − 3 | | y == CtbSizeY − 4 ) ) ? curr : curr + scaledSum (1506)

is at AdaptiveLoopFilter.cpp:1340 in VTM:

int pos = ((startHeight + i + ii) << scaleY) & (vbCTUHeight - 1);
if (scaleY == 0 && (pos == vbPos || pos == vbPos + 1))

I assume that y is actually yL, and this is a typo. But even with this the SubHeightC and scaleY do not match. In the spec it is skipped for 420, but in the software for 422/444. The spec behavior would make more sense for me.

As a side note, the y == CtbSizeY − 3 is a bit redundant as this does not occur with SubHeightC == 1.

Change history (5)

comment:1 Changed 3 years ago by sdchi

  • Version set to VTM-12.0

comment:2 Changed 3 years ago by fbossen

Not seeing a mismatch or typo here.
The use of y appears correct in the spec. Using yL would be incorrect (because it is a global position), as would be removing the y == CtbSizeY - 3 condition (it does occur when SubHeightC is equal to 1).

Note that for 422 and 444 formats SubHeightC is equal to 1 and scaleY is equal to 0. For the 420 format SubHeightC is equal to 2 and scaleY is equal to 1.

"Skipping" is meant to occur only when SubHeightC is equal to 1 (422 and 444 case) and processing the 3rd and 4th line above the bottom boundary of a CTU.

comment:3 Changed 3 years ago by sdchi

I see, thank you for the clarification. I mixed up the meaning of scaleY and SubHeightC (shift, divisor), and am thrown off a bit by the irregular border rules of CC-ALF.

What I meant with y is that is a chroma coordinate and it is compared to CtbSizeY which is a luma block size. Should this not be y * subHeightC then?

comment:4 Changed 3 years ago by sdchi

Nevermind, as SubHeightC is equal to 1 it would not matter.

comment:5 Changed 3 years ago by fbossen

  • Resolution set to invalid
  • Status changed from new to closed

Indeed, it doesn't matter whether you use y or y * subHeightC here. The end result is the same.

Note: See TracTickets for help on using tickets.