[IBC] neighbouring block availability issue

[fbarbier]

in 8.6.2.2 Derivation process for IBC luma block vector prediction

The derivation process for spatial block vector candidates from neighbouring coding units as specified in clause 8.6.2.3 is invoked with the luma coding block location ( xCb, yCb ) set equal to ( xSmr, ySmr )
may be problematic when the shared region located at xSmr and ySmr is not using IBC whereas the current block (located at xCb, yCb) is IBC.

Then when calling 6.4.4 with checkPredModeY equal to true, the availability flags for neighbors will be 0 (not because of the neighbor mode but because of the mode at the Smr block position).

Is is intentional in the spec or is it only a VTM bug ?

[fbarbier]

Please find attached a patch on VTM that is showing the issue.
Below the command lines to reproduce:

git checkout bb993c3618871bb69f123dd6b2d99d2693fb3204
patch -p1 -i ./IBC_avail.diff

cd build
readonly sanitizer="-fsanitize=address"
cmake .. -DCMAKE_BUILD_TYPE=Debug -DCMAKE_CXX_FLAGS=$sanitizer -DCMAKE_EXE_LINKER_FLAGS=$sanitizer
make -j 8
cd ..

./bin/DecoderAppStaticd -b encoded.vvc -o /dev/null

The output:

Shared position PredMode 1 is different from current PredMode 2
Shared position PredMode 1 is different from neighbor PredMode 2
isAvailableA1 should be 0 accordingly to specification
DecoderAppStaticd: /home/fredb/projects/vtm/source/Lib/CommonLib/UnitTools.cpp:1141: void PU::getIBCMergeCandidates(const PredictionUnit&, MergeCtx&, const int&): Assertion `0' failed.

The output shows that the current PredMode (IBC) may differ from the PredMode at the shared position.
Thus if the description of 8.6.2.2 / 8.6.2.3 and 6.4.4 is correct, a mismatch with VTM exists.

[yucheng]

Hi Fred,

According to my understanding, there is one minor bug related to SMR in current VVC Draft6 vE. For one corner case example, for a SMR region with area size equal to 4x16, and in this example, both the left 4x4 and right 4x4 CU are intra-coded and the center 4x8 CU is IBC-coded. In this case, when deriving the spatial candidates for the center 4x8 IBC-coded CU in section 8.6.2.3, the variable “checkPredModeY” will be set equal to true when invoking clause 6.4.4. However, the current luma location in this case will be (xSmr, ySmr) which is equal to the left 4x4 intra-coded luma location in SMR. Thus, availableN will be accidentally set equal to false because of the wrong luma location getting the CuPredMode.

The implementation and the intention in VTM6.0 is correct. That is, the original SMR concept and intention in proposal M0170 is the same as VTM6, and VVC Draft5 is correct also. However, in the VVC Draft6 vE it is wrong due to some integration change in neighboring availability function. In the VTM6, when deriving spatial merging candidates in SMR (in UnitTools.cpp line 1153 and line 1178), it will check if the “SMR’s neighborings” are IBC-coded or not.

It is suggested to have editorial changes to modify the VVC draft 6 vE. In section 6.4.4, it is suggested to import current block prediction mode, as shown below.

6.4.4 Derivation process for neighbouring block availability
Inputs to this process are:
– the luma location ( xCurr, yCurr ) of the top-left sample of the current block relative to the top-left luma sample of the current picture,
– the luma location ( xNbY, yNbY ) covered by a neighbouring block relative to the top-left luma sample of the current picture,
– the variable checkPredModeY specifying whether availability depends on the prediction mode.
– the variable cIdx specifying the colour component of the current block.
– the variable currPredMode specifying the coding mode of current block.
Output of this process is the availability of the neighbouring block covering the location ( xNbY, yNbY ), denoted as availableN.
…
When all of the following conditions are true, availableN is set equal to FALSE.
– checkPredModeY is equal to TRUE.
– availableN is set equal to TRUE.
– CuPredMode[ 0][ xNbY ][ yNbY ] is not equal to currPredMode CuPredMode[ 0][ xCurr ][ yCurr ].

When invoking section 6.4.4, it is suggested to import the current block’s prediction mode, as shown one example below. Suggested modification is similar in inter cases and at other sections invoking section 6.4.4.

8.6.2.3 Derivation process for IBC spatial block vector candidates
…
– The derivation process for neighbouring block availability as specified in clause 6.4.4 is invoked with the current luma location ( xCurr, yCurr ) set equal to ( xCb, yCb ), the neighbouring luma location ( xNbA1, yNbA1 ), checkPredModeY set equal to TRUE, CurrPredMode set equal to MODE_IBC and cIdx set equal to 0 as inputs, and the output is assigned to the block availability flag availableA1.

[fbarbier]

Thanks for answering,

Indeed, your proposal would fix the issue. I got the same conclusion.
We just make sure that it matches the initial design intent.

[bbross]

Thanks for spotting and confirming this issues.

I though about it and think there is a simpler solution that makes my life easier.
Instead of changing all invocations of 6.4.4 to add currPredMode, I would prefer to:

• not set (xCb, yCb) to (xSmr, ySmr) when invoking 8.6.2.3 but add it as an additional input parameter
• In 8.6.2.3 keep the invocation of 6.4.4 with (xCb, yCb) and change all other occurrences of xCb, yCb to ySmr and ySmr.

This way we just need to modify 8.6.2.3 and not all invocations of 6.4.4.
Can you confirm this fix?

[LiZhang]

I could confirm Ben's suggestion works.

Meanwhile, if we add a new input parameter (xCurrCb, yCurrCb) set to (xCb, yCb) when invoking 8.6.2.3, then we just need to change the invocation of 6.4.4 with (xCurrCb, yCurrCb). It could also solve the problem as well. Either way is fine.

Thanks.

[bbross]

Thanks Li for confirming, either way is fine and simple.