Compilation can fail for TEncSearch.cpp

[gordon]

Gcc compilation can fail in TEncSearch.cpp, in TEncSearch::estIntraPredQT (line 3144).

The indication is "array bounds exceeded". The compiler has partially followed the logic of

Int numModesAvailable = 67;
...
if (doFastSearch)
...
else
... problem here, with

for( Int i=0; i < numModesForFullRD; i++)

uiRdModeList[i] = i;

The compiler assumes that the array is being initialized up to numModesAvailable (deduced from previous tests), which is bigger than the array size.

From what I can see, doFastSearch is always true and the else case is never executed.
If that is true, the else case can be deleted as in the supplied patch.

If the else-case is to be left in for future-proofing any up coming logic, the array size will have to be increased.

[gordon]

asserts doFastSearch is true, and deletes else case.

[XiangLi]

Thanks for the report. Yes, it is a bug with additional intra prediction modes (under macro QC_USE_65ANG_MODES). FAST_UDI_MAX_RDMODE_NUM should be set to 67. Please check whether the attached fix solve the problem.

[XiangLi]

bug fix

[gordon]

Yes, the patch works and everything compiles OK.

[XiangLi]

Thanks for the confirmation. The patch has been applied in /branches/HM-14.0-KTA-1.0-dev.