Opened 3 years ago
Closed 3 years ago
#1494 closed defect (fixed)
Error with brackets and operator precedence in equation 433
Reported by: | peterderivaz | Owned by: | |
---|---|---|---|
Priority: | minor | Milestone: | |
Component: | spec | Version: | VVC D10 vH |
Keywords: | Cc: | ksuehring, bbross, XiangLi, fbossen, jvet@… |
Description
Equation 433 reads:
tmpVal = ( PaletteEscapeVal[ cIdx ][ xCbL + xL ][ yCbL + yL ] * levelScale[ qP%6 ] ) << ( qP / 6 ) + 32 ) >> 6
There are two opening brackets (, and three closing brackets ).
In addition, removing a closing bracket or placing an extra open bracket in any position seems incorrect because + has higher precedence than << according to table 1.
It looks like this might correspond to the following code from DecCu.cpp:544
value = ((((escapeValue.at(posXC, posYC)*g_invQuantScales[0][qpRem]) << qpPer) + add) >> invquantiserRightShift);
suggesting that the correct equation might be:
tmpVal = ( ( ( PaletteEscapeVal[ cIdx ][ xCbL + xL ][ yCbL + yL ] * levelScale[ qP%6 ] ) << ( qP / 6 ) ) + 32 ) >> 6
Change history (1)
comment:1 Changed 3 years ago by bbross
- Resolution set to fixed
- Status changed from new to closed
Good catch, thanks!
Your suggestion is correct and this will be fixed in v2 of JVET-W1004.