id	summary	reporter	owner	description	type	status	priority	milestone	component	version	resolution	keywords	cc
562	RPR - Calculation of refHeight	bheng		"In xPredInterBlkRPR( ), the calculation of refHeight (the number of rows of  intermediate horizontal filtered results required for vertical interpolation), is calculated as follows:

    int refHeight = height * scalingRatio.second >> SCALE_RATIO_BITS;

However, during the actual interpolation process, the actual rows used are determined incrementally using (row *stepY), as follows:

      int posY = (int32_t)y0Int + row * stepY;
      yInt = ( posY + offY ) >> posShift;


Given that there is some rounding error in stepY, the accumulated error from (row * stepY) can cause the interpolation use rows beyond the the rows generated by using (refHeight + vFilterSize - 1 + extSize).  This can cause the decoder to use uninitialized memory and/or crash.

I believe something like the following could be used to determine which rows the interpolation process will actually require.

int refHeight = ((((int32_t)y0Int + (height-1) * stepY) + offY ) >> posShift) - ((((int32_t)y0Int) + offY ) >> posShift) + 1;





"	defect	closed	minor	VTM-7.0	VTM	VTM-6.1	fixed		ksuehring XiangLi fbossen jvet@…