Opened 4 years ago

Closed 4 years ago

#1237 closed defect (fixed)

Subblock edges and filtering across picture, subpicture, tile and slice boundaries (8.8.3.4)

Reported by: fbossen Owned by:
Priority: minor Milestone:
Component: spec Version: VVC D10 vE
Keywords: Cc: ksuehring, bbross, XiangLi, fbossen, jvet@…

Description

In 8.8.3.4, edgeFlags is unconditionally set to 2 at subblock boundary locations:

– The value of edgeFlags[ x ][ y ] is derived as follows:
edgeFlags[ x ][ y ] = 2 (1206)

This appears to happen regardless of whether the boundary is also a picture, subpicture, tile or slice boundary, leading to undesired filtering.

The assignment to edgeFlags should probably be conditioned on filterEdgeFlag (similarly to what is done in 8.8.3.3 for transform block edges):

– When x is larger than 0 or filterEdgeFlag is equal to 1, the value of edgeFlags[ x ][ y ] is modified as follows:
edgeFlags[ x ][ y ] = 2

In such case, filterEdgeFlag needs to be added as an input to 8.8.3.4.

Change history (11)

comment:1 Changed 4 years ago by bbross

Thanks for brining that up, that part indeed looks strange since after it is set to 2, there is a comndition "When edgeFlags[ x ][ y ] is equal to 1 or 2,..." which is always true then.

Can somebody confirm that Frank suggested fix is correct?

comment:2 Changed 4 years ago by nanhu

I checked the spec, edgeFlags[][] should be conditionally set as Frank suggested.

comment:3 Changed 4 years ago by jlchen

  • Resolution set to fixed
  • Status changed from new to closed

Thanks Frank for reporting the issue and Han for confirmation. The suggested fix will be included in the vF release.

comment:4 Changed 4 years ago by LiZhang

Hi Frank, Nan,

Is it necessary to have 'filterEdgeFlag is equal to 1' in the suggested text below? I just checked 8.8.3.3 which doesn't have this condition. Also, when x is equal to 0 and filterEdgeFlag is equal to 1 (edgeFlags being 1), I think there is no difference between edgeFlags being 1 or 2.

– When x is larger than 0 or filterEdgeFlag is equal to 1, the value of edgeFlags[ x ][ y ] is modified as follows:
edgeFlags[ x ][ y ] = 2

comment:5 Changed 4 years ago by jlchen

  • Resolution fixed deleted
  • Status changed from closed to reopened

Hi Li,

edgeFlags are used in 8.8.3.5 as follows, not sure whether the behavior is same for edgeFlags being a or 2.

– Otherwise, if cIdx is equal to 0, edgeFlags[ xDi ][ yDj ] is equal to 2, and one or more of the following conditions are true, bS[ xDi ][ yDj ] is set equal to 1:

comment:6 follow-up: Changed 4 years ago by chiaming

It shall not include the condition "x is larger than 0". Otherwise, the edges of the current coding block coded by non-subblock coding mode will not check PB related conditions in deriving bS values. Then, it is not the intention of the current spec.
I suggest the text is fixed to:
– When filterEdgeFlag is equal to 1, the value of edgeFlags[ x ][ y ] is modified as follows:
edgeFlags[ x ][ y ] = 2

comment:7 in reply to: ↑ 6 ; follow-up: Changed 4 years ago by nanhu

In this case, when filterEdgeFlag is 0, does it mean that edgeFlags[][] cannot be set as 2 for the whole coding block? The condition suggested by Frank is only not filtering left/top boundary of current coding block for some undesired cases, while other sub-block (PB) edges of current coding block can still be filtered.

Replying to chiaming:

It shall not include the condition "x is larger than 0". Otherwise, the edges of the current coding block coded by non-subblock coding mode will not check PB related conditions in deriving bS values. Then, it is not the intention of the current spec.
I suggest the text is fixed to:
– When filterEdgeFlag is equal to 1, the value of edgeFlags[ x ][ y ] is modified as follows:
edgeFlags[ x ][ y ] = 2

comment:8 in reply to: ↑ 7 Changed 4 years ago by chiaming

Yes, when filterEdgeFlag is 0, deblocking is disabled for the left/top boundary of the current coding block. It should not set edgeFlags[][] as 2, otherwise it cannot bypass the bS derivation process.

8.8.3.5	Derivation process of boundary filtering strength
...
For xDi with i = 0..xN and yDj with j = 0..yN, the following applies:
 – If edgeFlags[ xDi ][ yDj ] is equal to 0, the variable bS[ xDi ][ yDj ] is set equal to 0.

Replying to nanhu:

In this case, when filterEdgeFlag is 0, does it mean that edgeFlags[][] cannot be set as 2 for the whole coding block? The condition suggested by Frank is only not filtering left/top boundary of current coding block for some undesired cases, while other sub-block (PB) edges of current coding block can still be filtered.

comment:9 Changed 4 years ago by fbossen

The condition "When x is larger than 0 or filterEdgeFlag is equal to 1" works as follows:

If x is equal to 0 and filterEdge is equal to 0, the condition is false, and edgeFlags[x][y] is unchanged.
Otherwise, the condition is true, and edgeFlags[x][y] is changed to 2.

Note that here x = xEdge * sbW takes only values that are aligned with subblock edges.

This is the same as in 8.8.3.3, where edgeFlags[x][y] is set to 1 for transform block edges unless x is equal to 0 and filterEdge is equal to 0:

  • If x is equal to 0, edgeFlags[ x ][ y ] is set equal to filterEdgeFlag.
  • Otherwise, if the location ( xCb + x , yCb + y ) is at a transform block edge, edgeFlags[ x ][ y ] is set equal to 1.

edgeFlags[x][y] being 1 or 2 is important, as noted in several other tickets.

@chiaming I don't understand your concern. The condition "x is larger than 0" is necessary here because we want to filter subblock edges that are not coding block boundaries, regardless of the value of filterEdgeFlag. The value of filterEdgeFlag is relevant only for edges that are also the left (or top) boundary of the current coding block. Having the condition "x is larger than 0" will not cause the left boundary of the current coding block to be filtered if filterEdgeFlag is 0.

comment:10 Changed 4 years ago by chiaming

Sorry, I mistook the condition. The condition suggested by Frank is right.

comment:11 Changed 4 years ago by bbross

  • Resolution set to fixed
  • Status changed from reopened to closed

Thanks everyone for checking and finally confirming the fix suggested by Frank which will be in JVET-S2001-vF.

Note: See TracTickets for help on using tickets.