Clipping issue in interpolation process

[wzeng]

In JVET-O2001-vE 8.5.6.3.2,
The luma locations in full-sample units are further modified as follows for i = 0..7:
xInti = Clip3( xSbIntL − 3, xSbIntL + sbWidth + 4, xInti ) (8-775)
yInti = Clip3( ySbIntL − 3, ySbIntL + sbHeight + 4, yInti ) (8-776)

When RPR is enabled, xInti/yInti is scaled while xSbIntL/ySbIntL and sbWidth/sbHeight are not scaled.
The purpose of this clipping operation is to ensure the same amount of reference data is used when DMVR is on. As DMVR is turned off when reference picture has different resolution with current picture, this clipping operation should not be performed when resolution is changed.
In addition, when sps_ref_wraparound_enabled_flag=1, this clipping operation seems not correct either.

The similar issue happens for chroma at 8.5.6.3.4,
The chroma locations in full-sample units ( xInti, yInti ) are further modified as follows for i = 0..3:
xInti = Clip3( xSbIntC − 1, xSbIntC + sbWidth + 2, xInti ) (8-789)
yInti = Clip3( ySbIntC − 1, ySbIntC + sbHeight + 2, yInti ) (8-790)

[zhou]

First of all, I agree that this clipping should not apply when the DMVR is off.

Secondly, this clipping operation seems to be incorrect even if sps_ref_wraparound_enabled_flag = 0.
( xSbIntL, ySbIntL ) is set equal to ( xSb + ( mvLX[ 0 ] >> 4 ), ySb + ( mvLX[ 1 ] >> 4 ) ).
the reference block patch defined by (xSbIntL − 3, ySbIntL − 3) and (xSbIntL + sbWidth + 4, ySbIntL + sbHeight + 4) can be compeletely outside the reference picture in the extream case, while (xInti , yInti) is already clipped into the picture when this additional clipping operation happens.

To fix, the order of those two clippings may need to be swapped. For example,

xInti = Clip3( xSbIntL − 3, xSbIntL + sbWidth + 4, xIntL + i − 3 ) (8 775)
yInti = Clip3( ySbIntL − 3, ySbIntL + sbHeight + 4, yIntL + i − 3 ) (8 776)

then

– If subpic_treated_as_pic_flag[ SubPicIdx ] is equal to 1, the following applies:
xInti = Clip3( SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xInti ) (8 771)
yInti = Clip3( SubPicTopBoundaryPos, SubPicBotBoundaryPos, yInti ) (8 772)
– Otherwise (subpic_treated_as_pic_flag[ SubPicIdx ] is equal to 0), the following applies:
xInti = Clip3( 0, picW − 1, sps_ref_wraparound_enabled_flag ?

ClipH( ( sps_ref_wraparound_offset_minus1 + 1 ) * MinCbSizeY, picW, xInti ) : (8 773)
xIntL + i − 3 )

yInti = Clip3( 0, picH − 1, yInti ) (8 774)

Not sure whether doing so is also sufficient for sps_ref_wraparound_enabled_flag=1

[zhou]

This ticket remains unaddressed. Here is an attempt for the fix:

1) add "– the decoder side motion vector refinement flag dmvrFlag," to the input parameter list of 8.5.6.3.2 Luma sample interpolation filtering process.

2) replace the following

The luma locations in full-sample units ( xInti, yInti ) are derived as follows for i = 0..7:
– If subpic_treated_as_pic_flag[ SubPicIdx ] is equal to 1, the following applies:
xInti = Clip3( SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xIntL + i − 3 ) (8 760)
yInti = Clip3( SubPicTopBoundaryPos, SubPicBotBoundaryPos, yIntL + i − 3 ) (8 761)
– Otherwise (subpic_treated_as_pic_flag[ SubPicIdx ] is equal to 0), the following applies:
xInti = Clip3( 0, picW − 1, sps_ref_wraparound_enabled_flag ?

ClipH( ( sps_ref_wraparound_offset_minus1 + 1 ) * MinCbSizeY, picW, xIntL + i − 3 ) : (8 762)
xIntL + i − 3 )

yInti = Clip3( 0, picH − 1, yIntL + i − 3 ) (8 763)
The luma locations in full-sample units are further modified as follows for i = 0..7:
xInti = Clip3( xSbIntL − 3, xSbIntL + sbWidth + 4, xInti ) (8 764)
yInti = Clip3( ySbIntL − 3, ySbIntL + sbHeight + 4, yInti ) (8 765)

*with

The luma locations in full-sample units ( xInti, yInti ) are derived as follows, for i = 0..7:

• if dmvflag is equal to 1, the following applies:

xInti = Clip3( xSbIntL − 3, xSbIntL + sbWidth + 4, yIntL + i − 3 ) (8 764)
yInti = Clip3( ySbIntL − 3, ySbIntL + sbHeight + 4, yIntL + i − 3 ) (8 765)

• Otherwise (dmvrflag is equal to 0), the following applies:

xInti = yIntL + i − 3 (8 764)
yInti = yIntL + i − 3 (8 765)

The luma locations in full-sample units ( xInti, yInti ) are further modified as follows for i = 0..7:

– If subpic_treated_as_pic_flag[ SubPicIdx ] is equal to 1, the following applies:
xInti = Clip3( SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xInti ) (8 760)
yInti = Clip3( SubPicTopBoundaryPos, SubPicBotBoundaryPos, yInti ) (8 761)

– Otherwise (subpic_treated_as_pic_flag[ SubPicIdx ] is equal to 0), the following applies:
xInti = Clip3( 0, picW − 1, sps_ref_wraparound_enabled_flag ?

ClipH( ( sps_ref_wraparound_offset_minus1 + 1 ) * MinCbSizeY, picW, xInti) : (8 762)
xInti )

yInti = Clip3( 0, picH − 1, yInti )

A similar change should be made to section 8.5.6.3.4 Chroma sample interpolation process

[wzeng]

Thank Minhua. The fix looks ok to me.