Opened 4 years ago

Closed 4 years ago

#1409 closed defect (fixed)

Overflow processing 16 bit sequences if JVET_R0351_HIGH_BIT_DEPTH_ENABLED is not set

Reported by: adybrowne Owned by:
Priority: minor Milestone:
Component: VTM Version:
Keywords: Cc: ksuehring, XiangLi, fbossen, jvet@…


At present VTM can process 16 bit input sources using 10 (or other) bits. The input reader automatically shifts right input data by the correct number of bits. However if JVET_R0351_HIGH_BIT_DEPTH_ENABLED is not set, the 16 bit unsigned data is read into a 16 bit signed array and the conversion to 10 bits is therefore incorrect.

The simplest solution is to prevent the use of 16 bit sources when JVET_R0351_HIGH_BIT_DEPTH_ENABLED is not set. This must be extended to preclude 15 bit sources due to the rounding present in the conversion.

Change history (1)

comment:1 Changed 4 years ago by fbossen

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.