Opened 4 years ago

Closed 4 years ago

# Order of operations in LMCS equations

Reported by: Owned by: bheng minor spec VVC D10 vG ksuehring, bbross, XiangLi, fbossen, jvet@…

### Description

I believe the LMCS equations below all need an extra parenthesis around the ... >> 11, due to the order of operations of the >> operator.

Section 8.5.6.7 - Equation 989:

predSamplesInter [ x ][ y ] = Clip1( LmcsPivot[ idxY ] + ( ScaleCoeff[ idxY ] * ( predSamplesInter[ x ][ y ] − InputPivot[ idxY ] ) + ( 1 << 10 ) ) >> 11 )

Section 8.7.5.2 - Equation 1191:

predMapSamples[ i ][ j ] = LmcsPivot[ idxY ] + ( ScaleCoeff[ idxY ] * ( predSamples[ i ][ j ] − InputPivot[ idxY ] ) + ( 1 << 10 ) ) >> 11

Section 8.8.2.2 - Equation 1200:

invSample = InputPivot[ idxYInv ] + ( InvScaleCoeff[ idxYInv ] * ( lumaSample − LmcsPivot[ idxYInv ] ) + ( 1 << 10 ) ) >> 11

Specifically, I would suggest the following:

predSamplesInter [ x ][ y ] = Clip1( LmcsPivot[ idxY ] + ( ( ScaleCoeff[ idxY ] * ( predSamplesInter[ x ][ y ] − InputPivot[ idxY ] ) + ( 1 << 10 ) ) >> 11 ) )

predMapSamples[ i ][ j ] = LmcsPivot[ idxY ] + ( ( ScaleCoeff[ idxY ] * ( predSamples[ i ][ j ] − InputPivot[ idxY ] ) + ( 1 << 10 ) ) >> 11 )

invSample = InputPivot[ idxYInv ] + ( ( InvScaleCoeff[ idxYInv ] * ( lumaSample − LmcsPivot[ idxYInv ] ) + ( 1 << 10 ) ) >> 11 )

### comment:1 Changed 4 years ago by yk

• Resolution set to fixed
• Status changed from new to closed

Good catch! Thanks! Will be fixed in JVET-S2001-vH.

Note: See TracTickets for help on using tickets.