Opened 5 years ago

Closed 5 years ago

#200 closed defect (fixed)

NULL-pointer crash of VTM 4.0.1rc1 in HDR config after JVET_M0428_ENC_DB_OPT integration

Reported by: crhelmrich Owned by:
Priority: critical Milestone: VTM-4.0.1
Component: VTM Version:
Keywords: deblocking, reshaping Cc: ksuehring, XiangLi, fbossen, jvet@…


In getDistortionDb() introduced through JVET_M0428_ENC_DB_OPT, m_tmpStorageLCU is NULL in line

PelBuf tmpRecLuma = m_tmpStorageLCU->getBuf(tmpArea);

when activating option --LumaLevelToDeltaQPMode=1 for HDR coding during the encoder configuration.

The reason seems to be that the in-loop reshaping introduced through JVET_M0427_INLOOP_RESHAPER
is disabled when LumaLevelToDeltaQPMode is nonzero, see EncAppCfg.cpp:xCheckParameter(). Also,
the xCalDebCost() function seems to be called even if the M0428 related code is disabled via encoder
option --EncDbOpt=0 (which is the default).

I don't know how to fix this properly. Can the relevant developers please advise?

Sample commandline (you can use any input sequence, it seems):
-c D:\encoder_randomaccess_vtm4.cfg --LumaLevelToDeltaQPMode=1 -wdt 416 -hgt 240 -fr 60 -f 1 --InputBitDepth=8 -i D:\sequences\BQSquare_416x240_60.yuv -o ..\..\bqs.yuv -b ..\..\bqs.bin

Best regards,

Christian Helmrich

Change history (4)

comment:1 Changed 5 years ago by taoranlu

It seems the reshaping implementation made by JVET_M0428_ENC_DB_OPT need some change on conditioning. I'm looking into this part of code and check how to deal with this.

comment:2 Changed 5 years ago by taoranlu

The null pointer issue can be easily fixed by replacing line

if ( compID == COMPONENT_Y && !afterDb)


    if ( compID == COMPONENT_Y && !afterDb && !m_pcEncCfg->getLumaLevelToDeltaQPMapping().isEnabled())

However, the JVET_M0428_ENC_DB_OPT still have other issues with HDR setting (WCG_EXT=1), if compile the code with JVET_M0427_INLOOP_RESHAPER=0 and WCG_EXT=1, in the case that reshaping implementation is completely turned off, the encoding still crashes when using HDR test setting with --LumaLevelToDeltaQPMode=1

comment:3 Changed 5 years ago by nanhu

merge request has been submitted to fix the problem:

comment:4 Changed 5 years ago by ksuehring

  • Resolution set to fixed
  • Status changed from new to closed

The MR was merged.

Note: See TracTickets for help on using tickets.