Opened 3 months ago

Closed 4 weeks ago

#532 closed defect (fixed)

Mismatch in equation 8-640 and 8-641 with SW

Reported by: chhuanb Owned by:
Priority: minor Milestone:
Component: spec Version: VVC D7 vD
Keywords: Cc: ksuehring, bbross, XiangLi, fbossen, jvet@…

Description

Equation 8-640 and 8-641 are not aligned with SW.

Change from:
cpMvLXCorner[ 1 ][ 0 ] = ( cpMvLXCorner[ 0 ][ 0 ] << 7 ) +

( ( cpMvLXCorner[ 2 ][ 1 ] − cpMvLXCorner[ 0 ][ 1 ] ) (8 640)

<< ( 7 + Log2( cbHeight / cbWidth ) ) )

cpMvLXCorner[ 1 ][ 1 ] = ( cpMvLXCorner[ 0 ][ 1 ] << 7 ) +

( ( cpMvLXCorner[ 2 ][ 0 ] − cpMvLXCorner[ 0 ][ 0 ] ) (8 641)

<< ( 7 + Log2( cbHeight / cbWidth ) ) )

To:

cpMvLXCorner[ 1 ][ 0 ] = ( cpMvLXCorner[ 0 ][ 0 ] << 7 ) +

( ( cpMvLXCorner[ 2 ][ 1 ] − cpMvLXCorner[ 0 ][ 1 ] ) (8 640)

<< ( 7 + Log2( cbWidth / cbHeight ) ) )

cpMvLXCorner[ 1 ][ 1 ] = ( cpMvLXCorner[ 0 ][ 1 ] << 7 ) -

( ( cpMvLXCorner[ 2 ][ 0 ] − cpMvLXCorner[ 0 ][ 0 ] ) (8 641)

<< ( 7 + Log2( cbWidth / cbHeight ) ) )

Change history (6)

comment:1 Changed 3 months ago by zhou

This may still not be right.

In the spec, "/" operation is defined as:

/ Integer division with truncation of the result toward zero. For example, 7 / 4 and −7 / −4 are truncated to 1 and −7 / 4 and 7 / −4 are truncated to −1.

Therefore, cbWidth / cbHeight could become zero if cbHeight > cbWidth.

"/" should be replaced by "÷" or ""-", or something like

cpMvLXCorner[ 1 ][ 0 ] = ( cpMvLXCorner[ 0 ][ 0 ] << 7 ) +
( ( cpMvLXCorner[ 2 ][ 1 ] − cpMvLXCorner[ 0 ][ 1 ] ) (8 640)
<< ( 7 + Log2( cbWidth ) - Log2( cbHeight ) ) )
cpMvLXCorner[ 1 ][ 1 ] = ( cpMvLXCorner[ 0 ][ 1 ] << 7 ) -
( ( cpMvLXCorner[ 2 ][ 0 ] − cpMvLXCorner[ 0 ][ 0 ] ) (8 641)
<< ( 7 + Log2( cbWidth ) - Log2( cbHeight ) ) )

There might be several places like this in the spec.

comment:2 Changed 4 weeks ago by bbross

  • Version changed from VVC D6 vE to VVC D7 vC

comment:3 Changed 4 weeks ago by bbross

How is this solved in VTM if a smaller size is divided by a larger one?
Is it really using floating point arithmetic and calculating Log2 on top of that?

comment:4 Changed 4 weeks ago by bbross

  • Version changed from VVC D7 vC to VVC D7 vD

comment:5 Changed 4 weeks ago by chhuanb

VTM is implemented as:

cpMvLXCorner[ 1 ][ 0 ] = ( cpMvLXCorner[ 0 ][ 0 ] << 7 ) +
( ( cpMvLXCorner[ 2 ][ 1 ] − cpMvLXCorner[ 0 ][ 1 ] ) (8 640)
<< ( 7 + Log2( cbWidth ) - Log2( cbHeight ) ) )
cpMvLXCorner[ 1 ][ 1 ] = ( cpMvLXCorner[ 0 ][ 1 ] << 7 ) -
( ( cpMvLXCorner[ 2 ][ 0 ] − cpMvLXCorner[ 0 ][ 0 ] ) (8 641)
<< ( 7 + Log2( cbWidth ) - Log2( cbHeight ) ) )

comment:6 Changed 4 weeks ago by bbross

  • Resolution set to fixed
  • Status changed from new to closed

Thanks! That makes sense and will be fixed in the next version of the draft.

Note: See TracTickets for help on using tickets.