Mismatch spec/VTM in wide angle intra prediction mode mapping process

[audrey.turquin]

In section 8.4.5.2.6, there is the following condition:

Otherwise, if all of the following conditions are true, predModeIntra is set equal to ( predModeIntra − 67 ).
– nH is greater than nW
– predModeIntra is less than or equal to 66
– predModeIntra is greater than ( whRatio > 1 ) ? ( 60 − 2 * whRatio ) : 60

However, in VTM, in function getWideAngle, what is done is :

else if (height > width && predMode > VDIA_IDX - modeShift[deltaSize])                                                                                                                                                                   
{                                                                                                                                                                                                                                        
    predMode -= (VDIA_IDX - 1);                                                                                                                                                                                                            
}

As VDIA_IDX = 66, predMode is set to predMode - 65, and not predMode - 67.

[deluxan]

In my opinion, there is no mismatch here. The software can implement what the spec says in many ways. In the spec, wide angles cannot take the values 0 and 1, given that they are reserved for planar and dc. Therefore, you have that 'weird jump' between 2 and -1. However, this 'jump' is of course not present when you are calculating the actual distances between angles in the intra prediction process in the software. For this reason, getWideAngle can map angles to 0 and 1.

I agree, however, that it is confusing, given that getWideAngle does not, in fact, return a wide angle the way they are defined in the spec. Perhaps renaming getWideAngle to something like getModifiedWideAngle would be more appropriate.

But the bottom line is, in my opinion, that the behaviour of both software and spec is aligned and correct.

[audrey.turquin]

Ok, I see what you mean about the 'jump'.
Thanks for the clarification.