Scaling process bugs

[jason]

In 8.7.3 (Scaling process for transform coefficients)

(1) The variable rectNonTsFlag is derived as follows:
rectNonTsFlag = ( ( ( Log2( nTbW ) + Log2( nTbH ) ) & 1 ) == 1 && transform_skip_flag[ xTbY ][ yTbY ] == 0 ) (8-955)

TSM is not supported for chroma TBs.
This should be fixed as
rectNonTsFlag = ( ( ( Log2( nTbW ) + Log2( nTbH ) ) & 1 ) == 1 && !( transform_skip_flag[ xTbY ][ yTbY ] && cIdx == 0 ) ) (8-955)

(2) The intermediate scaling factor m[ x ][ y ] is derived as follows:
If one or more of the following conditions are true, m[ x ][ y ] is set equal to 16:
– sps_scaling_list_enabled_flag is equal to 0.
– transform_skip_flag[ xTbY ][ yTbY ] is equal to 1.

This should be fixed as
If one or more of the following conditions are true, m[ x ][ y ] is set equal to 16:
– sps_scaling_list_enabled_flag is equal to 0.
– transform_skip_flag[ xTbY ][ yTbY ] is equal to 1 and cIdx is equal to 0.

(3) The variable rectNorm is not used in 8.7.3. It can be removed.

[bbross]

Thanks for reporting.

This is fixed JVET-P2001-v4 by the adoption of chroma TS.