Opened 3 months ago

## #408 new defect

# typo in gradient calculation formula for PROF in draft 6

Reported by: | Yuwen He | Owned by: | |
---|---|---|---|

Priority: | minor | Milestone: | |

Component: | spec | Version: | VVC D6 vE |

Keywords: | Cc: | ksuehring, bbross, XiangLi, fbossen, jvet@… |

### Description

Spec. editors,

In VVC spec. draft 6, the indices of predSamples[][] in gradient calculation (Eq. 8-796 and 8-797) are not correct since the buffer of predSamples is padded in two directions. It should be:

gradientH[ x ][ y ] = ( predSamples[ x + 2 ][ y+1 ] >> shift1 ) − ( predSamples[ x ][ y+1 ] >> shift1 ) (8-796)

gradientV[ x ][ y ] = ( predSamples[ x +1][ y + 2 ] >> shift1 ) − ( predSamples[ x+1 ][ y ] >> shift1 ) (8-797)

**Note:**See TracTickets for help on using tickets.