JVET-Q0089 - TS residual coding disabled with dependent quantization

[bheng]

There appears to be a mismatch between the SW and draft text for the case of:

slice_ts_residual_coding_disabled_flag=1
ph_dep_quant_enabled_flag = 1

It's not immediately clear which aspects of dependent quantization are intended to apply in Q0089 for a transform-skip coded TU. I believe the current behavior is as follows.

• Both the text and the software use dependent quantization state for CABAC context determination. But they don't actually use dependent quantization during the actual dequant process.

• For dequant, the SW does not use dependent quantization at all for transform skip TUs. The DepQuant code is bypassed here:

if ( tu.cs->picHeader->getDepQuantEnabledFlag() && (tu.mtsIdx[compID] != MTS_SKIP) )

• For dequant, the draft text seems to apply the 2x shift for dependent quantization (from the syntax table - shown below), but not any of the other aspects (e.g. in 8.7.3):

TransCoeffLevel[ x0 ][ y0 ][ cIdx ][ xC ][ yC ] =

( 2 * AbsLevel[ xC ][ yC ] − ( QState > 1 ? 1 : 0 ) ) *
( 1 − 2 * coeff_sign_flag[ n ] )

At a minimum, I guess the text needs to remove the 2x shift above for the case of transform_skip_flag = 1.

Additionally, it's not really clear if the residual coding was supposed to use any of the dependent quantization state for these transform skip TUs (e.g. why does dependent quant state affect CABAC context if dependent quantization is not used). Or if all of the dependent quantization aspects were supposed to be disabled for transform skip TUs. So additional changes might be necessary in both text and SW.

[analci]

The SW for Q0089 only affects lossless coding.

The case of slice_ts_residual_coding_disabled_flag=1 only happens for slices that are lossless coded. In this case dependent quantization is already disabled and the example case "slice_ts_residual_coding_disabled_flag=1 ph_dep_quant_enabled_flag = 1" should not happen.

[bheng]

Lossless coding may be the intended usage, but there's nothing stopping that combination of flags in the text, as far as I can tell.

If that combination of flags isn't supposed to be allowed, then the syntax should prevent it. Otherwise, the SW and text need to support any such valid combination.

[analci]

I believe, on the spec text the following change should handle this:

(7.3.10.11 Residual coding syntax)
...

if( ph_dep_quant_enabled_flag && !transform_skip_flag[ x0 ][ y0 ][ cIdx ] ) {

QState = startQStateSb
for( n = numSbCoeff − 1; n >= 0; n− − ) {

...
This would guarantee that in case of TS flag context derivation will not depend on DepQuant states regardless of the lossless cost.

[bbross]

Thanks for bringing that up.

This is fixed in D9 by the adoption of JVET-R0483 Combination 4 and JVET-R0271 variant #3.