Mismatch with spec on CuPredMode inference when modeType = MODE_TYPE_INTER

[forayr]

In specification:

When pred_mode_flag is not present, it is inferred as follows:
    • If cbWidth is equal to 4 and cbHeight is equal to 4, pred_mode_flag is inferred to be equal to 1.
    • Otherwise, if modeType is equal to MODE_TYPE_INTRA, pred_mode_flag is inferred to be equal to 1.
    • Otherwise, if modeType is equal to MODE_TYPE_INTER, pred_mode_flag is inferred to be equal to 0.
    • Otherwise, pred_mode_flag is inferred to be equal to 1 when decoding an I slice, and equal to 0 when decoding a P or B slice, respectively.

In vtm, predMode is always inferred to MODE_INTER if modeType is equal to MODE_TYPE_INTER, even if cbWidth is equal to 4 and cbHeight is equal to 4.

[zhaoyin]

Thank you for pointing out this problem.

A week ago, we identified a bug in the BT/TT split allowing conditions in JVET-O0050 spec and software. When a region is an inter coding region (i.e., modeType of the region is MODE_TYPE_INTER), the luma and chroma can continue to split in a single tree. In the spec and software, it is possible that a coding node (e.g., 8x4) in an inter coding region further splits into 4x4 child nodes.

However, the split that results in 4x4 child nodes in an inter coding region shall not be allowed, since 4x4 CU cannot use inter mode. It becomes an un-defined status for these 4x4 CUs, because no coding modes are allowed for them. Therefore, this BT/TT split restriction is necessary to the spec and the software.

The software does not trigger problem because the encoder RDO always decides not to split the 8x4 node into 4x4 child nodes in an inter coding region. The 4x4 child nodes will have MAX_DOUBLE cost due to no mode being tried in RDO, and thus 8x4 node not splitting defeats splitting into 4x4 in RDO. It is like that the encoder unintentionally performs a conformance check and signal a redundant split_cu_flag (as 0) in this case. So, there is no 4x4 CUs inside an inter coded region in the VTM-produced bitstream. This problem needs to be fixed, either by encoder conformance check or normative BT/TT split restrictions (i.e., adding one line in BT allowing condition and one line in TT allowing condition in the spec).

With this fix on BT/TT split allowing condition, 4x4 CUs are not possible to be produced in an Inter coding region, thus solving this software-spec mismatch.

Because this split restriction was not reported in the JVET-O0050 spec and software, we consider a new proposal addressing this bugfix might be better than directly fixing it during the software/spec integration.