#722 closed defect (fixed)

VTM master branch encoder crash for YUV444 coding

Reported by: xyxiu Owned by:
Priority: major Milestone: VTM-7.1
Component: VTM Version:
Keywords: 444 coding Cc: ksuehring, XiangLi, fbossen, jvet@…


Encoder crashed for the VTM master branch s/w for YUV444 coding. The attached cfg files and command line can be used to generate the crash point where happened at the first frame

EncoderApp -c Map_444.cfg -c encoder_randomaccess_vtm.cfg -c classF.cfg -c yuv444.cfg

It seems after disabling the macro JVET_N0278_FIXES the encoder can run but decoder still crash.

Attachments (4)

cfg.zip (3.9 KB) - added by xyxiu 18 months ago.
test.cfg (84 bytes) - added by delagrangep 18 months ago.
Test config with identity tables (crash)
ticket722-407dba6.patch (5.7 KB) - added by delagrangep 18 months ago.
Fix on top of MR1149
ticket722-ea41699.patch (5.5 KB) - added by delagrangep 18 months ago.
Fix while keeping UseIdentityTableForNon420Chroma

Download all attachments as: .zip

Change history (9)

Changed 18 months ago by xyxiu

comment:1 Changed 18 months ago by Vadim

There is currently a problem with JVET_P0410_CHROMA_QP_MAPPING integration for non 4:2:0 formats in VTM master branch.

Encoder sets
cfg_qpOutValCb.values = { 0 };
but later
m_chromaQpMappingTableParams.m_numPtsInCQPTableMinus1[0] = (int)cfg_qpOutValCb.values.size() - 2;

It gives negative value coded with UVLC, encoder crashes.

Probably the fix should be
m_chromaQpMappingTableParams.m_numPtsInCQPTableMinus1[0] = std::max<int>(0, (int)cfg_qpOutValCb.values.size() - 2);

comment:2 Changed 18 months ago by XiangLi

Yes, chroma table is reset in the following code, where m_useIdentityTableForNon420Chroma (i.e., encoder cfg option UseIdentityTableForNon420Chroma) was introduced in JVET-O0650. Tests showed that the crash can be avoided by "--UseIdentityTableForNon420Chroma=0". It looks this encoder only parameter is not used anymore. An email was sent to the proponent on whether the parameter and related code could be removed.

if (m_useIdentityTableForNon420Chroma && m_chromaFormatIDC != CHROMA_420) {

m_chromaQpMappingTableParams.m_sameCQPTableForAllChromaFlag = true;
cfg_qpInValCb.values = { 0 };
cfg_qpInValCr.values = { 0 };
cfg_qpInValCbCr.values = { 0 };
cfg_qpOutValCb.values = { 0 };
cfg_qpOutValCr.values = { 0 };
cfg_qpOutValCbCr.values = { 0 };


comment:4 Changed 18 months ago by delagrangep

Thanks for raising the issue. Root cause is bad handling of single-value (=identity) tables.
Attached are two fixes (choose):

  • one on top of MR1149 (ticket722-407dba6.patch)
  • one two commits before, without removal of UseIdentityTableForNon420Chroma (ticket722-ea41699.patch)

Quickly tested with attached test.cfg, which still crashes in MR1149.
Please review and test.

About removal of UseIdentityTableForNon420Chroma:
I think this should be decided by AHG15. I remember discussions about bypass of mapping tables in 4:4:4 which was the way it worked before adoption of progammable tables (O0650). We decided to keep that behaviour for our tests. UseIdentityTableForNon420Chroma, which is 1 by default, mimics that behaviour without touching the cfg files. If we remove it, then we have to update cfg files for non-4:2:0 cases (no big deal).

Changed 18 months ago by delagrangep

Test config with identity tables (crash)

Changed 18 months ago by delagrangep

Fix on top of MR1149

Changed 18 months ago by delagrangep

Fix while keeping UseIdentityTableForNon420Chroma

comment:5 Changed 18 months ago by fbossen

  • Milestone set to VTM-7.1
  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.