id	summary	reporter	owner	description	type	status	priority	milestone	component	version	resolution	keywords	cc
1453	Overflow in RPR scaling calculation	audrey.turquin		"In source/Lib/CommonLib/UnitTools.cpp, in getRprScaling function, there are the following calculations :


{{{
xScale = ( ( refPicWidth << SCALE_RATIO_BITS ) + ( curPicWidth >> 1 ) ) / curPicWidth;
yScale = ( ( refPicHeight << SCALE_RATIO_BITS ) + ( curPicHeight >> 1 ) ) / curPicHeight;
}}}


All the variables are int.
However, in case of 8K stream, with high values of scaling window, the calculation ""(refPicWidth << SCALE_RATIO_BITS) + (curPicWidth >> 1)"" can exceed the max int value, and thus, lead to an overflow."	defect	closed	minor	VTM-12.0	VTM	VTM-11.2	fixed		ksuehring XiangLi fbossen jvet@…