Deblocking of cu_cbf_flag=0 CUs with implicit transform splitting.

[bheng]

When a CU has cu_cbf_flag=0, VTM adds a single TU with size equal to the CU size.

For example:

if( cu.skip )
{

cs.addTU ( cu, partitioner.chType );

}

However, large CUs should instead be implicitly split into multiple empty TUs using according to the maximum transform size.

The TU edges of these empty TUs can affect the deblocking filter behavior. For example, the filtering lengths in subblock motion CUs.

[ksuehring]

I'm not very familiar with this part of the spec. Could you please point me to where this is specified?

[bheng]

No problem. My understanding is as follows:

Implicit Transform Splitting
JVET-Q2001-vB, Section 8.5.8.

" Depending on maxTbSize, the following applies:

– If nTbW is greater than maxTbWidth or nTbH is greater than maxTbHeight, the following ordered steps apply

...

2. The decoding process process for the residual signal of coding units coded in inter prediction mode as specified in this clause is invoked with the location ( xTb0, yTb0 ), the transform block width nTbW set equal to newTbW and the height nTbH set equal to newTbH,

etc..."

I believe this process is applied to all inter CUs, regardless of cu_cbf_flag = 0 or 1.

For example, I see nothing here to suggest that a 128x128 CU with cu_cbf_flag=0 is treated any differently than a CU with all tu_cbf_luma / tu_cbf_cb / tu_cbf_cr equal to 0. The current software however treats these two cases differently and uses an empty TU with size 128x128, (rather than multiple 64x64 or 32x32 TUs) which has implications for deblocking.

Deblocking TU Edges and Sub-Block Motion
JVET-Q2001-vB, Section 8.8.3.4

"– Otherwise, if one or more of the following conditions are true:

– xEdge is equal to 1,
– xEdge is equal to ( nCbW / 8 ) − 1,
– edgeTbFlags[ x − sbW ][ y ] is equal to 1,
– edgeTbFlags[ x + sbW ][ y ] is equal to 1,

the following applies:

maxFilterLengthPs[ x ][ y ] = 2
maxFilterLengthQs[ x ][ y ] = 2"

The distance from a subblock motion edge to the nearest TU edge (edgeTbFlags) can affect the filter length. This is regardless of whether the TU has CBF=1 or not.

Therefore, if the implicit split TUs are ignored, like in the current SW, the results will be different than the text.