Opened 3 months ago

#408 new defect

typo in gradient calculation formula for PROF in draft 6

Reported by: Yuwen He Owned by:
Priority: minor Milestone:
Component: spec Version: VVC D6 vE
Keywords: Cc: ksuehring, bbross, XiangLi, fbossen, jvet@…

Description

Spec. editors,
In VVC spec. draft 6, the indices of predSamples[][] in gradient calculation (Eq. 8-796 and 8-797) are not correct since the buffer of predSamples is padded in two directions. It should be:
gradientH[ x ][ y ] = ( predSamples[ x + 2 ][ y+1 ] >> shift1 ) − ( predSamples[ x ][ y+1 ] >> shift1 ) (8-796)
gradientV[ x ][ y ] = ( predSamples[ x +1][ y + 2 ] >> shift1 ) − ( predSamples[ x+1 ][ y ] >> shift1 ) (8-797)

Change history (0)

Note: See TracTickets for help on using tickets.