Issues with ISP transform splitting

[bheng]

In 8.4.5.1, the IntraSubPartitionsSplitType seems to be used without checking if the cIdx is luma or chroma. For example, in Equations (242) and (243).

nW = IntraSubPartitionsSplitType = = ISP_VER_SPLIT ? nTbW / NumIntraSubPartitions : nTbW
nH = IntraSubPartitionsSplitType = = ISP_HOR_SPLIT ? nTbH / NumIntraSubPartitions : nTbH

Also, in Equations (244) and (245), ISP_VER_SPLIT and ISP_HOR_SPLIT are constants, not variables. The "IntraSubPartitionsSplitType == " is missing.

xPartInc = ISP_VER_SPLIT ? 1 : 0
yPartInc = ISP_HOR_SPLIT ? 1 : 0

[deluxan]

Hi. Agree with the bug report. Here is a possible fix:

– The variables nW, nH, nPbW, pbFactor, xPartInc, yPartInc and NumPartitions are derived as follows:
nW = cIdx == 0 && IntraSubPartitionsSplitType = = ISP_VER_SPLIT ? nTbW / NumIntraSubPartitions : nTbW (242)
nH = cIdx == 0 && IntraSubPartitionsSplitType = = ISP_HOR_SPLIT ? nTbH / NumIntraSubPartitions : nTbH (243)
xPartInc = cIdx == 0 && IntraSubPartitionsSplitType == ISP_VER_SPLIT ? 1 : 0 (244)
yPartInc = cIdx == 0 && IntraSubPartitionsSplitType == ISP_HOR_SPLIT ? 1 : 0 (245)
nPbW = Max( 4 , nW ) (246)
pbFactor = nPbW / nW (247)
NumPartitions = cIdx == 0? NumIntraSubPartitions : 1 (248)
– For i = 0..NumPartitions − 1, the following applies:

[bbross]

Thanks for reporting Brian and Santi for providing a fix. This will be fixed in JVET-S2001-vH as suggested but the newly introduced variable numPartitions will start with a lower case "n" since it is not global.