Present in jem4.0 rc1. In TEncCu::xCheckRDCostInter obmc is tried 'on', then 'off'. When it's tried 'off' it uses a different prediction buffer that is not saved in rpcTempCU. If this becomes the winning candidate after xCheckBestMode, this "wrong" prediction buffer is used in a later JVET_C0024_PBINTRA_FAST test in xCompressCU.

The attached patch seems to correct the problem, but needs to be verified. Compression gain differences on 17-frame RA tests we've done show very small average differences, but with variations up to +/- 0.3%

Version: HM-16.6-JEM-4.0

I noticed encoding failed with floating point QP in most cases. The encoder was terminated by accessing an array with index -1.

It would seem that NUM_QP_PROB is not enough and TComStats::aaQPUsed runs out of space during encoding.

The issue seems to be fixed by increasing JVET_B0039_INC_NUM_QP_PROB from 7 to 14 as its comment says, but I'm not sure what value is appropriate.

Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00000000004b5b36 in TEncSbac::loadContextsFromPrev (this=0x7ffe173f3980, apcStats=0x83b8670, eSliceType=B_SLICE, iQPIdx=-1, bFromGloble=false, iQPIdxRst=-1, bAfterLastISlice=false)

at /.../source/Lib/TLibEncoder/TEncSbac.cpp:3273

#1 0x0000000000456a2e in TEncSlice::encodeSlice (this=0x7ffe173f66e0, pcPic=0x2ba23b668010, pcSubstreams=0x11902a80, numBinsCoded=@0x7ffe173d0680: 0, alfParam=...)

at /.../source/Lib/TLibEncoder/TEncSlice.cpp:1290

In function xCopyYuv2Tmp(), the follwoing code "m_pppcPredYuvBest[uiNextWIdx][uiNextHIdx]->copyToPartYuv( m_pppcPredYuvBest[uiWIdx][uiHIdx], uiPartUnitIdx );"

should be modified as "m_pppcPredYuvBest[uiNextWIdx][uiNextHIdx]->copyToPartYuv( m_pppcRecoYuvTemp[uiWIdx][uiHIdx], uiPartUnitIdx );"

That doesn't impact the performance (in PSNR and rate), but keeps the m_pppcPredYuvBest buffer as the correct prediction pixel.