#1353 closed defect (fixed)

Misalignment with spec when setting max filter length in xDeblockCU

Reported by: xinzhao Owned by:
Priority: minor Milestone: VTM-10.1
Component: VTM Version: VTM-10.0
Keywords: Cc: ksuehring, XiangLi, fbossen, jvet@…

Description

In VTM-10, xDeblockCU(..), the setting of max filter length is skipped when luma TU position (x or y) is not multiples of 4. However, for chroma 4:4:4 color format, in single tree, for an 8x8 CU, when luma is coded by ISP and chroma has one single 8x8 TU, VTM-10 skips the setting of max filter length for chroma (i.e., max filter length is initialized value 0), but Spec text tells that the max filter length for chroma should be set as either 1 or 3. Suggest to fix VTM.

Related codes in VTM-10.0, where areaTu points to a luma TU that may be ISP coded such that areaTu.x or areaTu.y may be not multiples of 4:

if( edgeDir == EDGE_HOR && ((areaTu.y % 4) != 0) )
{

continue;

}
if( edgeDir == EDGE_VER && ((areaTu.x % 4) != 0) )
{

continue;

}

Related spec text in JVET-S2001:

8.8.3.3 Derivation process of transform block boundary
...

– The variable gridSize is set as follows:

gridSize = cIdx = = 0 ? 4 : 8 (1205)

– If edgeType is equal to EDGE_VER, the following applies:
– The variable numEdges is set equal to Max( 1, nCbW / gridSize ).
– For xEdge = 0..numEdges − 1 and y = 0..nCbH − 1, the following applies:
...

– Otherwise (cIdx is not equal to 0), the values of maxFilterLengthPs[ x][ y ] and maxFilterLengthQs[ x ][ y ] are derived as follows:
– If the width in chroma samples of the transform block at chroma location ( xCb + x, yCb + y ) and the width at chroma location ( xCb + x − 1, yCb + y ) are both equal to or greater than 8, maxFilterLengthPs[ x ][ y ] and maxFilterLengthQs[ x ][ y ] are set equal to 3.
– Otherwise, maxFilterLengthPs[ x ][ y ] and maxFilterLengthQs[ x ][ y ] are set equal to 1.

Change history (3)

comment:1 Changed 14 months ago by ksuehring

  • Milestone VTM-10.0 deleted
  • Version set to VTM-10.0

comment:2 Changed 13 months ago by fbossen

Suggested fix:
https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/merge_requests/1885

It should be noted that currently available 4:2:2 and 4:4:4 conformance bitstreams may be incorrect because of this bug.

comment:3 Changed 13 months ago by XiangLi

  • Milestone set to VTM-10.1
  • Resolution set to fixed
  • Status changed from new to closed

Fixed as suggested.

Note: See TracTickets for help on using tickets.