Motion discontinuities, Geo mode and deblocking (8.8.3.5)

[fbossen]

In 8.8.3.5, there are a number of conditions related to motion discontinuities:

– For the prediction of the subblock containing the sample p0 different reference pictures or a different number of motion vectors are used than for the prediction of the subblock containing the sample q0.
NOTE 1 – The determination of whether the reference pictures used for the two coding sublocks are the same or different is based only on which pictures are referenced, without regard to whether a prediction is formed using an index into RPL 0 or an index into RPL 1, and also without regard to whether the index position within an RPL is different.
NOTE 2 – The number of motion vectors that are used for the prediction of a subblock with top-left sample covering ( xSb, ySb ) is equal to PredFlagL0[ xSb ][ ySb ] + PredFlagL1[ xSb ][ ySb ].
– One motion vector is used to predict the subblock containing the sample p0 and one motion vector is used to predict the subblock containing the sample q0, and the absolute difference between the horizontal or vertical component of the motion vectors used is greater than or equal to 8 in units of 1/16 luma samples.
– Two motion vectors and two different reference pictures are used to predict the subblock containing the sample p0, two motion vectors for the same two reference pictures are used to predict the subblock containing the sample q0 and the absolute difference between the horizontal or vertical component of the two motion vectors used in the prediction of the two subblocks for the same reference picture is greater than or equal to 8 in units of 1/16 luma samples.
– Two motion vectors for the same reference picture are used to predict the subblock containing the sample p0, two motion vectors for the same reference picture are used to predict the subblock containing the sample q0 and both of the following conditions are true:

– The absolute difference between the horizontal or vertical component of list 0 motion vectors used in the prediction of the two subblocks is greater than or equal to 8 in 1/16 luma samples, or the absolute difference between the horizontal or vertical component of the list 1 motion vectors used in the prediction of the two subblocks is greater than or equal to 8 in units of 1/16 luma samples.
– The absolute difference between the horizontal or vertical component of list 0 motion vector used in the prediction of the subblock containing the sample p0 and the list 1 motion vector used in the prediction of the subblock containing the sample q0 is greater than or equal to 8 in units of 1/16 luma samples, or the absolute difference between the horizontal or vertical component of the list 1 motion vector used in the prediction of the subblock containing the sample p0 and list 0 motion vector used in the prediction of the subblock containing the sample q0 is greater than or equal to 8 in units of 1/16 luma samples.

These conditions are somewhat difficult to read and understand. They require 2 clarification notes and use the fuzzy "subblock" term. They may also be incorrect in the context of Geo, since the motion vectors used for prediction can be different from the ones stored in memory and used for deblocking (see 8.5.7.3).

If defining some local variables, this could be expressed as:

– refPicPL0 is not equal to refPicQL0 or refPicPL1 is not equal to refPicQL1.
– refPicPL0 is not equal to refPicPL1, and the absolute difference between the horizontal or vertical component of the motion vectors mvPL0 and mvQL0 is greater than or equal to 8 in units of 1/16 luma samples.
– refPicPL0 is not equal to refPicPL1, and the absolute difference between the horizontal or vertical component of the motion vectors mvPL1 and mvQL1 is greater than or equal to 8 in units of 1/16 luma samples.
– refPicPL0 is equal to refPicPL1 and both of the following conditions are true:

– The absolute difference between the horizontal or vertical component of motion vectors mvPL0 and mvQL0 is greater than or equal to 8 in 1/16 luma samples, or the absolute difference between the horizontal or vertical component of motion vectors mvPL1 and mvQL1 is greater than or equal to 8 in units of 1/16 luma samples.
– The absolute difference between the horizontal or vertical component of motion vectors mvPL0 and mvQL1 is greater than or equal to 8 in units of 1/16 luma samples, or the absolute difference between the horizontal or vertical component of motion vectors mvPL1 and mvQL0 is greater than or equal to 8 in units of 1/16 luma samples.

where

The luma sample locations ( xp0 , yp0 ) and ( xq0 , yq0 ) are derived as follows:
– ( xq0 , yq0 ) is set equal to ( ( xCb + xDi ) * scaleWidth , ( yCb + yDj ) * scaleHeight ).
– If edgeType is equal to EDGE_VER, ( xp0 , yp0 ) is set equal to ( xq0 − 1 , yq0 ).
– Otherwise (edgeType is equal to EDGE_HOR), ( xp0 , yp0 ) is set equal to ( xq0 , yq0 − 1 ).

The reference pictures refPicPLX and refPicQLX for X equal to 0 and 1 are derived as follows:
– If PredFlagLX[ xp0 ][ yp0 ] is equal to 1, refPicPLX is set equal to RefPicList[ X ][ RefIdxLX[ xp0 ][ yp0 ] ]
– Otherwise, refPicPLX is set equal to “No reference picture”.
– If PredFlagLX[ xq0 ][ yq0 ] is equal to 1, refPicQLX is set equal to RefPicList[ X ][ RefIdxLX[ xq0 ][ yq0 ] ]
– Otherwise, refPicQLX is set equal to “No reference picture”.

The motion vectors mvPLX and mvQLX for X equal to 0 and 1 are derived as follows:
– mvPLX is set equal to MvLX[ xp0 ][ yp0 ]
– mvQLX is set equal to MvLX[ xq0 ][ yq0 ]

When refPicPL0 is not equal to refPicQL0, the values of refPicQL0 and refPicQL1 and of mvQL0 and mvQL1 are swapped as follows:

( refPicQL0, refPicQL1 ) = Swap( refPicQL0, refPicQL1 )
( mvQL0, mvQL1 ) = Swap( mvQL0, mvQL1 )

Note that the purpose of the swap is to try to align reference pictures on either side of the edge (i.e. get to refPicPL0 == refPicQL0 and refPicPL1 == refPicQL1). This is to avoid cross-list comparisons later.

[jlchen]

Agree that those conditions are difficult to understand. Your suggested solution is good to me. Of course, the new conditions are also not very easy to read, especially the last swapping operation.

It's quite big change for the last moment. Therefore it's would be good to have more experts to verify that the change is absolutely needed.

Can more deblocking and GEO experts confirm that the current conditions are not precise for GEO mode?

[fbossen]

If you are reluctant to make a lot of edits, you may consider a band-aid solution by adding another note that clarifies the meaning of expressions like "Two motion vectors and two different reference pictures are used to predict the subblock containing the sample p0". Basically reference pictures refer to values of RefPicList[ X ][ RefIdxLX[ xS ][ yS ] ] for a subblock located at ( xS, yS ) and motion vectors refer to values of MvLX[ xS ][ yS ]. These reference pictures / motion vectors may differ from the reference pictures / motion vectors that were actually used for prediction of samples (this would also be true for DMVR in addition to Geo).

[jlchen]

Adding one more note is good to me.

NOTE 3 – Reference pictures for a subblock located at ( xS, yS ) refer to values of RefPicList[ X ][ RefIdxLX[ xS ][ yS ] ] and motion vectors for a subblock located at ( xS, yS ) refer to values of MvLX[ xS ][ yS ]. These reference pictures and motion vectors may differ from the reference pictures and motion vectors that are actually used for the prediction of the subblock.

Will leave Ben to take the action.

[bbross]

Thanks for clarifying that and the band aid should do it with minimum changes. I added the 3rd note as suggested by Jianle for JVET-S2001-vG.