Opened 3 years ago

Closed 3 years ago

#1494 closed defect (fixed)

Error with brackets and operator precedence in equation 433

Reported by: peterderivaz Owned by:
Priority: minor Milestone:
Component: spec Version: VVC D10 vH
Keywords: Cc: ksuehring, bbross, XiangLi, fbossen, jvet@…

Description

Equation 433 reads:

tmpVal = ( PaletteEscapeVal[ cIdx ][ xCbL + xL ][ yCbL + yL ] * levelScale[ qP%6 ] ) << ( qP / 6 ) + 32 ) >> 6

There are two opening brackets (, and three closing brackets ).

In addition, removing a closing bracket or placing an extra open bracket in any position seems incorrect because + has higher precedence than << according to table 1.

It looks like this might correspond to the following code from DecCu.cpp:544

value = ((((escapeValue.at(posXC, posYC)*g_invQuantScales[0][qpRem]) << qpPer) + add) >> invquantiserRightShift);

suggesting that the correct equation might be:

tmpVal = ( ( ( PaletteEscapeVal[ cIdx ][ xCbL + xL ][ yCbL + yL ] * levelScale[ qP%6 ] ) << ( qP / 6 ) ) + 32 ) >> 6

Change history (1)

comment:1 Changed 3 years ago by bbross

  • Resolution set to fixed
  • Status changed from new to closed

Good catch, thanks!
Your suggestion is correct and this will be fixed in v2 of JVET-W1004.

Note: See TracTickets for help on using tickets.