Decoding process for residual signal in case of SBT is missing

[zhaoyin]

In 8.5.8 (Decoding process for the residual signal of coding blocks coded in inter prediction mode), the residual of a CU is processed. However, in the case of SBT, the spec is not correct since the width and height of the residual TU is set as nTbW and nTbH, and the position of the residual TU is set as (xTbY, yTbY).

To fix this issue, the following text is suggested to be added in 8.5.8:

– Otherwise, if cu_sbt_flag is equal to 1 and cu_sbt_pos_flag is equal to 0, the following applies:

1. The variables wPartIdx and hPartIdx are derived as follows:

wPartIdx = cu_sbt_horizontal_flag ? 4 : ( cu_sbt_quad_flag ? 1 : 2 ) (8-XX)
hPartIdx = !cu_sbt_horizontal_flag ? 4 : ( cu_sbt_quad_flag ? 1 : 2 ) (8-XX)

2. The scaling and transformation process as specified in clause 8.7.2 is invoked with the luma location ( xTbY , yTbY ), the variable cIdx, ( nTbW * wPartIdx / 4 ) and ( nTbH * hPartIdx / 4 ) as inputs, and the output is an ( nTbW * wPartIdx / 4 )x( nTbH * hPartIdx / 4 ) array resSamples.

– Otherwise, if cu_sbt_flag is equal to 1 and cu_sbt_pos_flag is equal to 1, the following applies:

1. The variables wPartIdx, hPartIdx, xPartIdx and yPartIdx are derived as follows:

wPartIdx = cu_sbt_horizontal_flag ? 4 : ( cu_sbt_quad_flag ? 1 : 2 ) (8-XX)
hPartIdx = !cu_sbt_horizontal_flag ? 4 : ( cu_sbt_quad_flag ? 1 : 2 ) (8-XX)
xPartIdx = cu_sbt_horizontal_flag ? 0 : ( cu_sbt_quad_flag ? 3 : 2 ) (8-XX)
yPartIdx = !cu_sbt_horizontal_flag ? 0 : ( cu_sbt_quad_flag ? 3 : 2 ) (8-XX)

2. The scaling and transformation process as specified in clause 8.7.2 is invoked with the luma location ( xTbY + nTbW * xPartIdx / 4, yTbY + nTbH * yPartIdx / 4 ), the variable cIdx, ( nTbW * wPartIdx / 4 ) and ( nTbH * hPartIdx / 4 ) as inputs, and the output is an ( nTbW * wPartIdx / 4 )x( nTbH * hPartIdx / 4 ) array resSamples.

before

– Otherwise, the scaling and transformation process as specified in clause 8.7.2 is invoked with the luma location ( xTbY, yTbY ), the variable cIdx, the transform width nTbW and the transform height nTbH as inputs, and the output is an (nTbW)x(nTbH) array resSamples.

[bbross]

Will be fixed in vD