Opened 4 years ago

Closed 4 years ago

#820 closed defect (fixed)

RPR: sampleRateConv() function takes wrong input bitdepth

Reported by: taoranlu Owned by:
Priority: minor Milestone: VTM-8.0
Component: VTM Version: VTM-7.1
Keywords: RPR Cc: ksuehring, XiangLi, fbossen, jvet@…

Description

In Picture.cpp, inside function rescalePicture() when calling the function sampleRateConv(), the input parameter regarding bitdepth is currently

bitDepths.recon[comp]

This is wrong as bitDepth.recon[] should take channel type instead of component type. It will cause wrong bitdepth for comp=2 (Cr component) and the result is that Cr component may not be properly clipped after interpolation. Suggest to use

bitDepths.recon[toChannelType(compID)]

to fix this issue.

Change history (2)

comment:2 Changed 4 years ago by XiangLi

  • Milestone set to VTM-8.0
  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.