Opened 2 months ago

#1494 new defect

Error with brackets and operator precedence in equation 433

Reported by: peterderivaz Owned by:
Priority: minor Milestone:
Component: spec Version: VVC D10 vH
Keywords: Cc: ksuehring, bbross, XiangLi, fbossen, jvet@…

Description

Equation 433 reads:

tmpVal = ( PaletteEscapeVal[ cIdx ][ xCbL + xL ][ yCbL + yL ] * levelScale[ qP%6 ] ) << ( qP / 6 ) + 32 ) >> 6

There are two opening brackets (, and three closing brackets ).

In addition, removing a closing bracket or placing an extra open bracket in any position seems incorrect because + has higher precedence than << according to table 1.

It looks like this might correspond to the following code from DecCu.cpp:544

value = ((((escapeValue.at(posXC, posYC)*g_invQuantScales[0][qpRem]) << qpPer) + add) >> invquantiserRightShift);

suggesting that the correct equation might be:

tmpVal = ( ( ( PaletteEscapeVal[ cIdx ][ xCbL + xL ][ yCbL + yL ] * levelScale[ qP%6 ] ) << ( qP / 6 ) ) + 32 ) >> 6

Change history (0)

Note: See TracTickets for help on using tickets.