Issue on coordinates derivation for TMVP (8.5.2.11, 8.5.5.3, 8.5.5.4, and 8.5.5.6)

[parkmw]

TMVP should be able to use subblock level MVs from colocated picture since MVs are stored and accessed in subblock level. However, the current spec description does not support this. Therefore, the following parts in the spec should be modified.

The subclause 8.5.2.11 should be modified as follows:

1. The bottom right collocated motion vector and the bottom and right boundary sample locations are derived as follows:

...

– If yCb >> CtbLog2SizeY is equal to yColBr >> CtbLog2SizeY, yColBr is less than or equal to botBoundaryPos and xColBr is less than or equal to rightBoundaryPos, the following applies:

– The variable colCb specifies the luma coding block covering the modified location given by ( ( xColBr >> 3 ) << 3, ( yColBr >> 3 ) << 3 ) inside the collocated picture specified by ColPic.
– The luma location ( xColCb, yColCb ) is set equal to the top-left sample of the collocated luma coding block specified by colCb relative to the top-left luma sample of the collocated picture specified by ColPic.
– The derivation process for collocated motion vectors as specified in subclause 8.5.2.12 is invoked with currCb, colCb, ( xColCb, yColCb )( ( xColBr >> 3 ) << 3, ( yColBr >> 3 ) << 3 ), refIdxLX and sbFlag set equal to 0 as inputs, and the output is assigned to mvLXCol and availableFlagLXCol.

...

2. When availableFlagLXCol is equal to 0, the central collocated motion vector is derived as follows:

...

– The variable colCb specifies the luma coding block covering the modified location given by ( ( xColCtr >> 3 ) << 3, ( yColCtr >> 3 ) << 3 ) inside the collocated picture specified by ColPic.
– The luma location ( xColCb, yColCb ) is set equal to the top-left sample of the collocated luma coding block specified by colCb relative to the top-left luma sample of the collocated picture specified by ColPic.
– The derivation process for collocated motion vectors as specified in subclause 8.5.2.12 is invoked with currCb, colCb, ( xColCb, yColCb )( ( xColCtr >> 3 ) << 3, ( yColCtr >> 3 ) << 3, refIdxLX and sbFlag set equal to 0 as inputs, and the output is assigned to mvLXCol and availableFlagLXCol.

The subclause 8.5.5.3 should be modified as follows:

...

– The variable colCb specifies the luma coding block covering the modified location given by ( ( xColSb >> 3 ) << 3, ( yColSb >> 3 ) << 3 ) inside the ColPic.
– The luma location ( xColCb, yColCb ) is set equal to the top-left sample of the collocated luma coding block specified by colCb relative to the top-left luma sample of the collocated picture specified by ColPic.
– The derivation process for collocated motion vectors as specified in subclause 8.5.2.12 is invoked with currCb, colCb, ( xColCb, yColCb )( ( xColSb >> 3 ) << 3, ( yColSb >> 3 ) << 3, refIdxL0 set equal to 0 and sbFlag set equal to 1 as inputs and the output being assigned to the motion vector of the subblock mvL0SbCol[ xSbIdx ][ ySbIdx ] and availableFlagL0SbCol.
– The derivation process for collocated motion vectors as specified in subclause 8.5.2.12 is invoked with currCb, colCb, ( xColCb, yColCb )( ( xColSb >> 3 ) << 3, ( yColSb >> 3 ) << 3, refIdxL1 set equal to 0 and sbFlag set equal to 1 as inputs and the output being assigned to the motion vector of the subblock mvL1SbCol[ xSbIdx ][ ySbIdx ] and availableFlagL1SbCol.

...

The subclause 8.5.5.4 should be modified as follows:

...

– The variable colCb specifies the luma coding block covering the modified location given by ( ( xColCb >> 3 ) << 3, ( yColCb >> 3 ) << 3 ) inside the ColPic.
– The luma location ( xColCb, yColCb ) is set equal to the top-left sample of the collocated luma coding block specified by colCb relative to the top-left luma sample of the collocated picture specified by ColPic.
– The derivation process for collocated motion vectors specified in subclause 8.5.2.12 is invoked with currCb, colCb, (xColCb, yColCb)( ( xColCb >> 3 ) << 3, ( yColCb >> 3 ) << 3, refIdxL0 set equal to 0, and sbFlag set equal to 1 as inputs and the output being assigned to ctrMvL0 and ctrPredFlagL0.
– The derivation process for collocated motion vectors specified in subclause 8.5.2.12 is invoked with currCb, colCb, (xColCb, yColCb)( ( xColCb >> 3 ) << 3, ( yColCb >> 3 ) << 3, refIdxL1 set equal to 0, and sbFlag set equal to 1 as inputs and the output being assigned to ctrMvL1 and ctrPredFlagL1.

...

The subclause 8.5.5.6 should be modified as follows:

...

– If yCb >> CtbLog2SizeY is equal to yColBr >> CtbLog2SizeY, yColBr is less than or equal to botBoundaryPos and xColBr is less than or equal to rightBoundaryPos, the following applies:

– The variable colCb specifies the luma coding block covering the modified location given by ( ( xColBr >> 3 ) << 3, ( yColBr >> 3 ) << 3 ) inside the collocated picture specified by ColPic.
– The luma location ( xColCb, yColCb ) is set equal to the top-left sample of the collocated luma coding block specified by colCb relative to the top-left luma sample of the collocated picture specified by ColPic.
– The derivation process for collocated motion vectors as specified in subclause 8.5.2.12 is invoked with currCb, colCb, ( xColCb, yColCb )( ( xColBr >> 3 ) << 3, ( yColBr >> 3 ) << 3, refIdxLXCorner[ 3 ] and sbFlag set equal to 0 as inputs, and the output is assigned to mvLXCol and availableFlagLXCol.

...

This issue was confirmed by several experts. Please check my suggestion for this issue.

[fbossen]

I confirm there is an issue here, but the proposed fix isn't sufficient.
For example, note that the 3rd input to 8.5.2.12 is
– a luma location ( xColCb, yColCb ) specifying the top-left sample of the collocated luma coding block specified by colCb relative to the top-left luma sample of the collocated picture specified by ColPic,

[parkmw]

Thanks for pointing it out. Right, your comment is correct. We should modify description of the 3rd input to 8.5.2.12. My suggestion is to modify it as follows:
– a luma location ( xColCb, yColCb ) specifying the sample in the collocated luma coding block specified by colCb relative to the top-left luma sample of the collocated picture specified by ColPic,
If you have any other suggestion, please let me know.

And if we need to modify further for this issue, please make any suggestions. Thanks.

[jlchen]

Probably ( xColCb, yColCb ) can be mentioned as follows without mentioning "coding block".

– a luma location ( xColCb, yColCb ) specifying the current luma sample relative to the top-left luma sample of the collocated picture specified by ColPic,

Another issue is the input colCb,

– a variable colCb specifying the collocated coding block inside the collocated picture specified by ColPic,

colCb is used in 8.5.2.12, the input should the "8x8 motion grid block" covers luma location ( xColCb, yColCb ), not the "coding block" covers luma location ( xColCb, yColCb ). It's difficult to define "8x8 motion grid block"

[parkmw]

For your modification of the 3rd input to 8.5.2.12, it looks ok for me.

Regarding the input colCb, I've catched your point. As you mentioned, it seems to be difficult to define "8x8 motion grid block".
But, my opinion is that colCb could be left as it because colCb in subclause 8.5.2.12 is just used to obtain refIdx and coded mode, which should be the same values with ones in "8x8 motion grid block".
From my understanding, even though a CU has multiple MVs, all information except MV should have the same values in the CU. If I missed something, please correct me.

However, I basically agree on your opinion. If we could make a better definition for "8x8 motion grid block", I think it would be much better. I hope someone comes up with it.

[fbossen]

Can't the reference indices change within a CU if SbTMVP or Geo is used?

[parkmw]

In case of SbTMVP, I think all the reference indices in a CU are the same. But, I didn't think about Geo case. I think the reference indcies might change. That means we need to define "8x8 motion grid block".

[jlchen]

Thanks for confirming. As initial try, I would like to use " a variable colCb specifying the 8x8 luma block inside the collocated picture with the top-left sample at the location ( xColCb, yColCb )".

[parkmw]

I have no objection with your idea. Thanks.

[bbross]

A comment on colCb, colCb is used in the following conditions:
– If colCb is coded in an intra, IBC, or palette prediction mode, both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.
– If LongTermRefPic( currPic, currCb, refIdxLX, LX ) is not equal to LongTermRefPic( ColPic, colCb, refIdxCol, listCol ), both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.
– Otherwise, the variable availableFlagLXCol is set equal to 1, refPicList[ listCol ][ refIdxCol ] is set to be the picture with reference index refIdxCol in the RPL listCol of the slice containing the luma block colCb in the collocated picture specified by ColPic, and the following applies:

Further checking the function LongTermRefPic:

The function LongTermRefPic( aPic, aCb, refIdx, LX ), with X being 0 or 1, is defined as follows:
– If the picture with index refIdx from RPL LX of the slice containing the luma block aCb in the picture aPic was marked as "used for long-term reference" at the time when aPic was the current picture, LongTermRefPic( aPic, aCb, refIdx, LX ) is equal to 1.
– Otherwise, LongTermRefPic( aPic, aCb, refIdx, LX ) is equal to 0.

All three conditions would also work if we would keep the original definition of colCb:

– The variable colCb specifies the luma coding block covering the modified location given by ( ( xColSb >> 3 ) << 3, ( yColSb >> 3 ) << 3 ) inside the ColPic.

since:

• the prediction mode of the CU that contains the 8x8 block does not change (covers condition 1.)
• The slice that contains the 8x8 subblock is also not changing within a CU (cover condition 2. and 3.)

The issue with the reference picture index derivation being different for the subblock does not apply to colCb since this is done via the location rather than the variable colCb.

Can somebody confirm this?

[jlchen]

Frank, your comment on colCb seems correct to me. The colCb can be kept it is, or be changed to 8x8 luma block, which becomes editorial only.