id	summary	reporter	owner	description	type	status	priority	milestone	component	version	resolution	keywords	cc
168	Interaction between M0140 (SBT) and M0297 (MTS Zero Out)	bheng		"M0297 (MTS Zero Out) intends to prevent coefficients outside the low-frequency 16x16 region for MTS transform types (!= DCT2) by modifying the coefficient scanning syntax to make it impossible to send coefficients outside this area. 

Part of M0140 assigns implicitly signaled MTS transform types (DCT8/DST7) to sub-block transform blocks.  

Should the M0297 syntax modification also be applied to M0140 SBT blocks?


Currently the syntax modification is not applied to SBT blocks, so they can generate coefficients outside the 16x16 region for MTS transform types.  

M0297 adds some logic later in an attempt to zero these coefficients out (see below), but the skipHeight doesn't have any effect (it’s not actually used in the inverse transform, for example fastInverseDST7_B32).  So, the following code only zeros out the columns, not the rows.

  const int      skipWidth  = ( trTypeHor != DCT2 && width  == 32 ) ? 16 : ...;
  const int      skipHeight = ( trTypeVer != DCT2 && height == 32 ) ? 16 : ...;
"	defect	closed	minor	VTM-4.0	VTM		fixed		ksuehring XiangLi fbossen jvet@…