Opened 2 years ago

Closed 2 years ago

#179 closed defect (fixed)

HDR Anchor encoding crashes with VTM4.0

Reported by: taoranlu Owned by:
Priority: major Milestone:
Component: VTM Version:
Keywords: Cc: ksuehring, XiangLi, fbossen, jvet@…

Description

Current VTM will crash during encoding for HDR Anchor test condition. Part of reason is caused by bug of reshaper implementation regarding RDCost. The fix is provided in merge request: https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/merge_requests/269/diffs

However, lumaDQP still have issue with (possibly) JVET_M0102_INTRA_SUBPARTITIONS. The encoder for HDR Anchor setting will crash even if we turn reshaper macro completely off ( JVET_M0427_INLOOP_RESHAPER=0) and enable WCG_EXT = 1 in typedef.h

The code will crash in EncCU.cpp, xCompressCU(): it enters the condition when DQP is used, and then the getTU() function will crash. Inside getTU(), it crashes in the while loop introduced by JVET_M0102_INTRA_SUBPARTITIONS.

if (tempCS->pps->getUseDQP() && CS::isDualITree(*tempCS) && isChroma(partitioner.chType))

{

const Position chromaCentral(tempCS->area.Cb().chromaPos().offset(tempCS->area.Cb().chromaSize().width >> 1, tempCS->area.Cb().chromaSize().height >> 1));
const Position lumaRefPos(chromaCentral.x << getComponentScaleX(COMPONENT_Cb, tempCS->area.chromaFormat), chromaCentral.y << getComponentScaleY(COMPONENT_Cb, tempCS->area.chromaFormat));
const CodingStructure* baseCS = bestCS->picture->cs;
const CodingUnit* colLumaCu = baseCS->getCU(lumaRefPos, CHANNEL_TYPE_LUMA);

const TransformUnit* tu = baseCS->getTU(lumaRefPos, CHANNEL_TYPE_LUMA);

if (colLumaCu && tu)
{

currTestMode.qp = colLumaCu->qp;

}

}

In CodingStructure.cpp, getTU()

#if JVET_M0102_INTRA_SUBPARTITIONS

if( idx != 0 )
{

unsigned extraIdx = 0;
if( isLuma( effChType ) )
{

const TransformUnit& tu = *tus[idx - 1];
if( tu.cu->ispMode ) Intra SubPartitions mode
{

we obtain the offset to index the corresponding sub-partition
if( subTuIdx != -1 )
{

extraIdx = subTuIdx;

}
else
{

while( pos != tus[idx - 1 + extraIdx]->blocks[effChType].pos() )

{

extraIdx++;

}

}

}

}
return tus[idx - 1 + extraIdx];

}

#else

Change history (1)

comment:1 Changed 2 years ago by fbossen

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.