Opened 8 months ago

Closed 7 months ago

#722 closed defect (fixed)

VTM master branch encoder crash for YUV444 coding

Reported by: xyxiu Owned by:
Priority: major Milestone: VTM-7.1
Component: VTM Version:
Keywords: 444 coding Cc: ksuehring, XiangLi, fbossen, jvet@…


Encoder crashed for the VTM master branch s/w for YUV444 coding. The attached cfg files and command line can be used to generate the crash point where happened at the first frame

EncoderApp -c Map_444.cfg -c encoder_randomaccess_vtm.cfg -c classF.cfg -c yuv444.cfg

It seems after disabling the macro JVET_N0278_FIXES the encoder can run but decoder still crash.

Attachments (4) (3.9 KB) - added by xyxiu 8 months ago.
test.cfg (84 bytes) - added by delagrangep 8 months ago.
Test config with identity tables (crash)
ticket722-407dba6.patch (5.7 KB) - added by delagrangep 8 months ago.
Fix on top of MR1149
ticket722-ea41699.patch (5.5 KB) - added by delagrangep 8 months ago.
Fix while keeping UseIdentityTableForNon420Chroma

Download all attachments as: .zip

Change history (9)

Changed 8 months ago by xyxiu

comment:1 Changed 8 months ago by Vadim

There is currently a problem with JVET_P0410_CHROMA_QP_MAPPING integration for non 4:2:0 formats in VTM master branch.

Encoder sets
cfg_qpOutValCb.values = { 0 };
but later
m_chromaQpMappingTableParams.m_numPtsInCQPTableMinus1[0] = (int)cfg_qpOutValCb.values.size() - 2;

It gives negative value coded with UVLC, encoder crashes.

Probably the fix should be
m_chromaQpMappingTableParams.m_numPtsInCQPTableMinus1[0] = std::max<int>(0, (int)cfg_qpOutValCb.values.size() - 2);

comment:2 Changed 8 months ago by XiangLi

Yes, chroma table is reset in the following code, where m_useIdentityTableForNon420Chroma (i.e., encoder cfg option UseIdentityTableForNon420Chroma) was introduced in JVET-O0650. Tests showed that the crash can be avoided by "--UseIdentityTableForNon420Chroma=0". It looks this encoder only parameter is not used anymore. An email was sent to the proponent on whether the parameter and related code could be removed.

if (m_useIdentityTableForNon420Chroma && m_chromaFormatIDC != CHROMA_420) {

m_chromaQpMappingTableParams.m_sameCQPTableForAllChromaFlag = true;
cfg_qpInValCb.values = { 0 };
cfg_qpInValCr.values = { 0 };
cfg_qpInValCbCr.values = { 0 };
cfg_qpOutValCb.values = { 0 };
cfg_qpOutValCr.values = { 0 };
cfg_qpOutValCbCr.values = { 0 };


comment:4 Changed 8 months ago by delagrangep

Thanks for raising the issue. Root cause is bad handling of single-value (=identity) tables.
Attached are two fixes (choose):

  • one on top of MR1149 (ticket722-407dba6.patch)
  • one two commits before, without removal of UseIdentityTableForNon420Chroma (ticket722-ea41699.patch)

Quickly tested with attached test.cfg, which still crashes in MR1149.
Please review and test.

About removal of UseIdentityTableForNon420Chroma:
I think this should be decided by AHG15. I remember discussions about bypass of mapping tables in 4:4:4 which was the way it worked before adoption of progammable tables (O0650). We decided to keep that behaviour for our tests. UseIdentityTableForNon420Chroma, which is 1 by default, mimics that behaviour without touching the cfg files. If we remove it, then we have to update cfg files for non-4:2:0 cases (no big deal).

Changed 8 months ago by delagrangep

Test config with identity tables (crash)

Changed 8 months ago by delagrangep

Fix on top of MR1149

Changed 8 months ago by delagrangep

Fix while keeping UseIdentityTableForNon420Chroma

comment:5 Changed 7 months ago by fbossen

  • Milestone set to VTM-7.1
  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.