Spec and VTM s/w mismatch on clipping residual bit-depth of IACT

[xyxiu]

There is one mismatch between JVET-Q2001-v14 and VTM-8.0 on clipping range of the input residuals to IACT. In 8.7.4.6 of JVET-Q2001-v14, the clipping range is [-(1<<BitDepth), (1<<BitDepth)-1], as described as:

rY[ x ][ y ] = Clip3( −( 1 << BitDepth ), ( 1 << BitDepth ) − 1, rY[ x ][ y ] ) (1220)
rCb[ x ][ y ] = Clip3( −( 1 << BitDepth ), ( 1 << BitDepth ) − 1, rCb[ x ][ y ] ) (1221)
rCr[ x ][ y ] = Clip3( −( 1 << BitDepth ), ( 1 << BitDepth ) − 1, rCr[ x ][ y ] ) (1222)

In VTM-8.0, the corresponding clipping range is [-(1<<(BitDepth+1)), (1<<(BitDepth+1))-1] as follows:

int maxAbsclipBD = (1 << (clpRng.bd + 1)) - 1;
y0 = Clip3((-maxAbsclipBD - 1), maxAbsclipBD, y0);
cg = Clip3((-maxAbsclipBD - 1), maxAbsclipBD, cg);
co = Clip3((-maxAbsclipBD - 1), maxAbsclipBD, co);

Because the YCgCo-R transform increases the residual bit-depth by 1-bit, the full dynamic range after ACT should be [-(1<<(BitDepth+1)), (1<<(BitDepth+1))-1]. Therefore, the spec should be aligned with the software.