Mismatch on Inverse JCCR between SW and Spec

[tsukuba.takeshi]

In VTM-8.0, there is a clipping operation in inverse JCCR process as below:

template<int signedMode> void invTransformCbCr( PelBuf &resCb, PelBuf &resCr )
{

Pel* cb = resCb.buf;
Pel* cr = resCr.buf;
for( SizeType y = 0; y < resCb.height; y++, cb += resCb.stride, cr += resCr.stride )
{

for( SizeType x = 0; x < resCb.width; x++ )
{

if ( signedMode == 1 ) { cr[x] = cb[x] >> 1; }
else if ( signedMode == -1 ) { cr[x] = -cb[x] >> 1; }
else if ( signedMode == 2 ) { cr[x] = cb[x]; }
else if ( signedMode == -2 ) { cr[x] = (cb[x] == -32768 && sizeof(Pel) == 2) ? 32767 : -cb[x]; } non-normative clipping to prevent 16-bit overflow
else if ( signedMode == 3 ) { cb[x] = cr[x] >> 1; }
else if ( signedMode == -3 ) { cb[x] = -cr[x] >> 1; }

}

}

}

On the other hand, the spec (JVET-Q2001vE) doesn't have such a clipping operation.

8.7.2 Scaling and transformation process
...

3.The residual samples resSamples[ x ][ y ] with x = 0..nTbW − 1, y = 0..nTbH − 1 are derived as follows:
–If cIdx is equal to codedCIdx, the following applies:

resSamples[ x ][ y ] = res[ x ][ y ] (1151)

–Otherwise, if TuCResMode[ xTbY ][ yTbY ] is equal to 2, the following applies:

resSamples[ x ][ y ] = cSign * res[ x ][ y ] (1152)

–Otherwise, the following applies:

resSamples[ x ][ y ] = ( cSign * res[ x ][ y ] ) >> 1 (1153)

Suggested fixes are 1) or 2):
-1) align SW to Spec, i.e. remove clipping operation.
-2) align Spec to SW, i.e. add clipping operation.

[fbossen]

As the comment in the code says: it's non-normative clipping to avoid overflow. The clip operation is there because cb[x] is stored as a signed 16-bit value (it cannot represent value 32768). The clipping doesn't affect the end result.
If you want the software to not use such clipping, then the data type Pel needs to be defined as a 32-bit integer. In general we don't want to do that (mainly for memory requirements reasons).

There is no need to change either text or software here as far as I can tell.