Opened 5 weeks ago

Last modified 5 weeks ago

## #532 new defect

# Mismatch in equation 8-640 and 8-641 with SW

Reported by: | chhuanb | Owned by: | |
---|---|---|---|

Priority: | minor | Milestone: | |

Component: | spec | Version: | VVC D6 vE |

Keywords: | Cc: | ksuehring, bbross, XiangLi, fbossen, jvet@… |

### Description

Equation 8-640 and 8-641 are not aligned with SW.

Change from:

cpMvLXCorner[ 1 ][ 0 ] = ( cpMvLXCorner[ 0 ][ 0 ] << 7 ) +

( ( cpMvLXCorner[ 2 ][ 1 ] − cpMvLXCorner[ 0 ][ 1 ] ) (8 640)

<< ( 7 + Log2( cbHeight / cbWidth ) ) )

cpMvLXCorner[ 1 ][ 1 ] = ( cpMvLXCorner[ 0 ][ 1 ] << 7 ) +

( ( cpMvLXCorner[ 2 ][ 0 ] − cpMvLXCorner[ 0 ][ 0 ] ) (8 641)

<< ( 7 + Log2( cbHeight / cbWidth ) ) )

To:

cpMvLXCorner[ 1 ][ 0 ] = ( cpMvLXCorner[ 0 ][ 0 ] << 7 ) +

( ( cpMvLXCorner[ 2 ][ 1 ] − cpMvLXCorner[ 0 ][ 1 ] ) (8 640)

<< ( 7 + Log2(

cbWidth / cbHeight) ) )

cpMvLXCorner[ 1 ][ 1 ] = ( cpMvLXCorner[ 0 ][ 1 ] << 7 ) **-**

( ( cpMvLXCorner[ 2 ][ 0 ] − cpMvLXCorner[ 0 ][ 0 ] ) (8 641)

<< ( 7 + Log2(

cbWidth / cbHeight) ) )

**Note:**See TracTickets for help on using tickets.

This may still not be right.

In the spec, "/" operation is defined as:

/ Integer division with truncation of the result toward zero. For example, 7 / 4 and −7 / −4 are truncated to 1 and −7 / 4 and 7 / −4 are truncated to −1.

Therefore, cbWidth / cbHeight could become zero if cbHeight > cbWidth.

"/" should be replaced by "÷" or ""-", or something like

cpMvLXCorner[ 1 ][ 0 ] = ( cpMvLXCorner[ 0 ][ 0 ] << 7 ) +

( ( cpMvLXCorner[ 2 ][ 1 ] − cpMvLXCorner[ 0 ][ 1 ] ) (8 640)

<< ( 7 + Log2( cbWidth ) - Log2( cbHeight ) ) )

cpMvLXCorner[ 1 ][ 1 ] = ( cpMvLXCorner[ 0 ][ 1 ] << 7 ) -

( ( cpMvLXCorner[ 2 ][ 0 ] − cpMvLXCorner[ 0 ][ 0 ] ) (8 641)

<< ( 7 + Log2( cbWidth ) - Log2( cbHeight ) ) )

There might be several places like this in the spec.