Issue with derivation of maxFilterLengthPs on coding subblock boundaries

[emora]

In section 8.8.3.4, maxFilterLengthPs[x][y] is modified if left neighbor (vertical edge) or above neighbor (horizontal edge) is affine or in sub-block coding mode as per equations (8-1028) and (8-1040). The condition that enables this modification is written as follows, for example for vertical edge: "When inter_affine_flag[ xCb − 1 ][ yCb ] is equal to 1 or merge_subblock_flag[ xCb − 1 ][ yCb ] is equal to 1, the following applies ..."

It seems to me that this is incorrect and should be "inter_affine_flag[ xCb − 1 ][ yCb + y ]" and "merge_subblock_flag[ xCb − 1 ][ yCb + y ]" because a block can have multiple (and smaller in height) left neighboring blocks along side its vertical edge, and not necessarily all are coded in affine/sub-block mode.

Same can be said for horizontal edge. Change would be: "inter_affine_flag[ xCb + x ][ yCb − 1 ]" and "merge_subblock_flag[ xCb + x ][ yCb − 1 ]"

VTM software supports currently the proposed modification and is thus unaligned with the current spec.

[emora]

In section 8.8.3.4, maxFilterLengthPs[x][y] is modified if left neighbor (vertical edge) or above neighbor (horizontal edge) is affine or in sub-block coding mode as per equations (1218) and (1230). The condition that enables this modification is written as follows, for example for vertical edge: "When inter_affine_flag[ xCb − 1 ][ yCb ] is equal to 1 or merge_subblock_flag[ xCb − 1 ][ yCb ] is equal to 1, the following applies ..."

It seems to me that this is incorrect and should be "inter_affine_flag[ xCb − 1 ][ yCb + y ]" and "merge_subblock_flag[ xCb − 1 ][ yCb + y ]" because a block can have multiple (and smaller in height) left neighboring blocks along side its vertical edge, and not necessarily all are coded in affine/sub-block mode.

Same can be said for horizontal edge. Change would be: "inter_affine_flag[ xCb + x ][ yCb − 1 ]" and "merge_subblock_flag[ xCb + x ][ yCb − 1 ]"

VTM software supports currently the proposed modification and is thus unaligned with the current spec.

[Kenneth]

Thank you! I agree with your suggestion.

[LiZhang]

I also confirm the suggested changes are correct. Will be fixed in JVET-Q2001.

[bbross]

Thanks for reporting.

This has been fixed in JVET-Q2001-v8.