NULL-pointer crash of VTM 4.0.1rc1 in HDR config after JVET_M0428_ENC_DB_OPT integration

[crhelmrich]

In getDistortionDb() introduced through JVET_M0428_ENC_DB_OPT, m_tmpStorageLCU is NULL in line

PelBuf tmpRecLuma = m_tmpStorageLCU->getBuf(tmpArea);

when activating option --LumaLevelToDeltaQPMode=1 for HDR coding during the encoder configuration.

The reason seems to be that the in-loop reshaping introduced through JVET_M0427_INLOOP_RESHAPER
is disabled when LumaLevelToDeltaQPMode is nonzero, see EncAppCfg.cpp:xCheckParameter(). Also,
the xCalDebCost() function seems to be called even if the M0428 related code is disabled via encoder
option --EncDbOpt=0 (which is the default).

I don't know how to fix this properly. Can the relevant developers please advise?

Sample commandline (you can use any input sequence, it seems):
-c D:\encoder_randomaccess_vtm4.cfg --LumaLevelToDeltaQPMode=1 -wdt 416 -hgt 240 -fr 60 -f 1 --InputBitDepth=8 -i D:\sequences\BQSquare_416x240_60.yuv -o ..\..\bqs.yuv -b ..\..\bqs.bin

Best regards,

Christian Helmrich

[taoranlu]

It seems the reshaping implementation made by JVET_M0428_ENC_DB_OPT need some change on conditioning. I'm looking into this part of code and check how to deal with this.

[taoranlu]

The null pointer issue can be easily fixed by replacing line

if ( compID == COMPONENT_Y && !afterDb)

by

    if ( compID == COMPONENT_Y && !afterDb && !m_pcEncCfg->getLumaLevelToDeltaQPMapping().isEnabled())

However, the JVET_M0428_ENC_DB_OPT still have other issues with HDR setting (WCG_EXT=1), if compile the code with JVET_M0427_INLOOP_RESHAPER=0 and WCG_EXT=1, in the case that reshaping implementation is completely turned off, the encoding still crashes when using HDR test setting with --LumaLevelToDeltaQPMode=1

[nanhu]

merge request has been submitted to fix the problem: ​https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/merge_requests/331

[ksuehring]

The MR was merged.