Opened 11 days ago

Closed 7 days ago

#676 closed defect (fixed)

Fix rate control with WCG_EXT

Reported by: lzz8246 Owned by:
Priority: minor Milestone: VTM-7.1
Component: VTM Version: VTM-7.0rc1
Keywords: rate control Cc: ksuehring, XiangLi, fbossen, jvet@…

Description

In the WCG_EXT macro, the RD cost in the RDO process will be calculated by the unadjusted lambda and lambda-derived distortion scale factor as shown in the code:

#if WCG_EXT
double RdCost::calcRdCost( uint64_t fracBits, Distortion distortion, bool useUnadjustedLambda )
#else
double RdCost::calcRdCost( uint64_t fracBits, Distortion distortion )
#endif
{
#if JVET_AHG14_LOSSLESS
  if( m_costMode == COST_LOSSLESS_CODING && 0 != distortion )
  {
    return MAX_DOUBLE;
  }
#endif
#if WCG_EXT
  return ( useUnadjustedLambda ? m_DistScaleUnadjusted : m_DistScale ) * double( distortion ) + double( fracBits );
#else
  return m_DistScale * double( distortion ) + double( fracBits );
#endif
}

These unadjusted parameters are saved before encoding a slice. But when rate control is applied, the frame level lambda will be determined by the allocated bits after "m_pcRdCost->saveUnadjustedLambda();". The lambda used in the RDO process is not the calculated lambda but the unadjusted lambda saved before bit allocation, leading to a not working of frame level rate control. Fix it by adding a "m_pcRdCost->saveUnadjustedLambda();" in "EncSlice::resetQP()". This function "EncSlice::resetQP()" will be executed only when rate control is enabled.

Another modification is change the order of "pRdCost->setLambda()" and "pRdCost->saveUnadjustedLambda();" in the CTU level rate control.

Following is the encoding results of RaceHorses before/after fixation.
Configuration: randomaccess, RateControl=1, TargetBitrate=400000, LCULevelRateControl=0
Before fixation:
Total Frames | Bitrate Y-PSNR U-PSNR V-PSNR YUV-PSNR

300 a 279.3752 29.2691 34.6837 35.7230 30.4033

After fixation:
Total Frames | Bitrate Y-PSNR U-PSNR V-PSNR YUV-PSNR

300 a 403.0296 35.2344 38.8569 40.0848 35.8540

Change history (5)

comment:1 follow-up: Changed 9 days ago by crhelmrich

Hi, this bug report also seems relevant to my perceptual QP adaptation (QPA) code. If I understand your report correctly, whenever rate control or QPA code calls pcRdCost->setLambda(some lambda, bit depth), the following code should be added right after that call:

#if WCG_EXT
pcRdCost->saveUnadjustedLambda();
#endif

Is that correct? If so, then the QPA code needs to be fixed in a similar fashion (two QPA-related calls to pcRdCost->setLambda() are affected, one in EncCu::updateLambda() and one at the beginning of EncSlice::encodeCtus()).

Christian

P.S.: What about the rate control related call to pcRdCost->setLambda(oldLambda, bit depth) at the end of EncSlice::encodeCtus()? Doesn't that need a pcRdCost->saveUnadjustedLambda() as well?

Last edited 9 days ago by crhelmrich (previous) (diff)

comment:2 in reply to: ↑ 1 Changed 8 days ago by lzz8246

Replying to crhelmrich:
Yes, I have verified that this marco WCG_EXT will have some influence on the encoding results of QPA. Actually, I think the WCG_EXT will affect all the cases where the QP or lambda varies among CTUs, or the QP or lambda used in encoding process differs from the one obtained from the pre-defined QP. But I have no idea whether the influence is positive or negative on the cases except rate control. So, to avoid BD-rate change in other cases, I just fixed the rate control by modifying the functions which will only be executed when rate control is applied. Meanwhile, I think it should be further study that calling saveUnadjustedLambda() in pcRdCost->setLambda() since many other places including cu-level coding have called pcRdCost->setLambda().
Thanks for your kindly reminder in the P.S., I have added saveUnadjustedLambda() before compressCTU() to fix the not working of CTU-level rate control. While at the end of EncSlice::encodeCtus(), the current CTU has been encoded with the correct lambda. So I think there may not be influence on rate control, I will conduct some more experiments to verify it.

Hi, this bug report also seems relevant to my perceptual QP adaptation (QPA) code. If I understand your report correctly, whenever rate control or QPA code calls pcRdCost->setLambda(some lambda, bit depth), the following code should be added right after that call:

#if WCG_EXT
pcRdCost->saveUnadjustedLambda();
#endif

Is that correct? If so, then the QPA code needs to be fixed in a similar fashion (two QPA-related calls to pcRdCost->setLambda() are affected, one in EncCu::updateLambda() and one at the beginning of EncSlice::encodeCtus()).

Christian

P.S.: What about the rate control related call to pcRdCost->setLambda(oldLambda, bit depth) at the end of EncSlice::encodeCtus()? Doesn't that need a pcRdCost->saveUnadjustedLambda() as well?

comment:3 follow-up: Changed 8 days ago by crhelmrich

Hi, thanks very much for your reply. Meanwhile I did some testing with the QPA, and your suggested call of pcRdCost->saveUnadjustedLambda() definitely improves the subjective QPA performance (at the same time, it also leads to a slight rate reduction when QPA is active). Could you please add the following corresponding fix to your merge request !676:

In EncCu::updateLambda (...):

   if (updateRdCostLambda)
   {
      m_pcRdCost->setLambda (newLambda, slice->getSPS()->getBitDepths());
+ #if WCG_EXT
+     m_pcRdCost->saveUnadjustedLambda();
+ #endif
   }
 }
 #endif // SHARP_LUMA_DELTA_QP || ENABLE_QPA_SUB_CTU

With this additional change, your merge request fixes not only the rate control but also the QPA code (and possibly other lambda adjusting code, as you mentioned). By the way, the WCG related code, executed when getWCGChromaQPControl().isEnabled(), is not affected by this additional fix.

Christian

comment:4 in reply to: ↑ 3 Changed 8 days ago by lzz8246

Replying to crhelmrich:
Got it, I have updated the merge request.

Hi, thanks very much for your reply. Meanwhile I did some testing with the QPA, and your suggested call of pcRdCost->saveUnadjustedLambda() definitely improves the subjective QPA performance (at the same time, it also leads to a slight rate reduction when QPA is active). Could you please add the following corresponding fix to your merge request !676:

In EncCu::updateLambda (...):

   if (updateRdCostLambda)
   {
      m_pcRdCost->setLambda (newLambda, slice->getSPS()->getBitDepths());
+ #if WCG_EXT
+     m_pcRdCost->saveUnadjustedLambda();
+ #endif
   }
 }
 #endif // SHARP_LUMA_DELTA_QP || ENABLE_QPA_SUB_CTU

With this additional change, your merge request fixes not only the rate control but also the QPA code (and possibly other lambda adjusting code, as you mentioned). By the way, the WCG related code, executed when getWCGChromaQPControl().isEnabled(), is not affected by this additional fix.

Christian

comment:5 Changed 7 days ago by ksuehring

  • Milestone set to VTM-7.1
  • Resolution set to fixed
  • Status changed from new to closed

The MR was merged.

Note: See TracTickets for help on using tickets.