A patch has been provided by Tomohiro Ikai:

I'd like to share a patch to fix the trace functionality. #define FIX_TRACE 1

With this, TraceEnc.txt basically matches with TraceDec.txt, which is very useful for debugging. After KTA1, some trace-related fix seems have been done with the macro "DEBUG". The changes are straight forward, so we may remove the macro while integration.

Note: Since the amount of the trace text in ecodeBin()/encodeBin() is too much, I suggest to disable them (included in my patch)

I'm filing it here for later application to the HM-16.6 based branch.

When ATVMP macro (QC_SUB_PU_TMVP) is off, we may get encoding/decoding MD5 mismatch sometimes.

Gcc compilation can fail in TEncSearch.cpp, in TEncSearch::estIntraPredQT (line 3144).

The indication is "array bounds exceeded". The compiler has partially followed the logic of

Int numModesAvailable = 67; ... if (doFastSearch) ... else ... problem here, with

for( Int i=0; i < numModesForFullRD; i++)

uiRdModeList[i] = i;

The compiler assumes that the array is being initialized up to numModesAvailable (deduced from previous tests), which is bigger than the array size.

From what I can see, doFastSearch is always true and the else case is never executed. If that is true, the else case can be deleted as in the supplied patch.

If the else-case is to be left in for future-proofing any up coming logic, the array size will have to be increased.