Potential bug in transform coefficient zero-out of large TUs

[zhou]

non-zero transform coefficients found on the decoder side in the area outside the top-left 32x32 region of large TUs. For example, in

ParkRunning3_3840x2160_50fps_10bit_420_IBBBP_22_2.vvc

which is the 3rd sub-stream of the sequence parallel coded in RA cfg with QP = 22.

The non-zero coefficient location is:

Frame POC: 20
Location of 64x64 CU inside the frame: x = 1024 y = 704
Location of non-zero COEF inside the TU: x = 32 y = 16
Non-zero COEF value: 14

[mcoban]

TCQ can flip a zero coefficient in the zero-out region to a non-zero coefficient during quantization. Scanning only the non zero-out regions or modifying the TCQ process for handling of the coefficients in the zero-out region could be potential fixes.

[fbossen]

Addressed by JVET-M0257