Rate control in current VTM does not work.

[liufeiyang]

Rate control in current VTM for intra frame does not work. Issues are pointed out below. We also provide MR to fix them.

1. Alpha and beta of intra frame will be clipped into range of non-intra frame. In current R-D model, beta for intra frame is positive while that for non-intra frame is negative. Information from log file is shown below.

enc-BasketballDrive-intra-37-913449.log:

POC 0 LId: 0 TId: 0 ( IDR_N_LP, I-SLICE, QP 36 ) 145576 bits [Y 37.7743 dB U 42.5253 dB V 42.7773 dB] [ET 117 ] [L0] [L1]
POC 1 LId: 0 TId: 0 ( CRA, I-SLICE, QP 31 ) 255656 bits [Y 39.1747 dB U 43.8172 dB V 44.5325 dB] [ET 212 ] [L0] [L1]
POC 2 LId: 0 TId: 0 ( CRA, I-SLICE, QP 26 ) 550880 bits [Y 40.7797 dB U 45.1589 dB V 46.7530 dB] [ET 457 ] [L0] [L1]
POC 3 LId: 0 TId: 0 ( CRA, I-SLICE, QP 21 ) 2546016 bits [Y 46.0984 dB U 46.1833 dB V 48.4812 dB] [ET 435 ] [L0] [L1]
POC 4 LId: 0 TId: 0 ( CRA, I-SLICE, QP 16 ) 4362840 bits [Y 50.7490 dB U 49.4621 dB V 50.4992 dB] [ET 593 ] [L0] [L1]
POC 5 LId: 0 TId: 0 ( CRA, I-SLICE, QP 11 ) 7067032 bits [Y 55.0679 dB U 55.2035 dB V 55.1767 dB] [ET 697 ] [L0] [L1]
POC 6 LId: 0 TId: 0 ( CRA, I-SLICE, QP 6 ) 9985280 bits [Y 59.7519 dB U 60.3722 dB V 61.1108 dB] [ET 732 ] [L0] [L1]
WARNING: Encoded stream does not meet MinCr requirements numBytesInVclNalUnits (1540685) must be <= 1253376. Try increasing Qp, tier or level
POC 7 LId: 0 TId: 0 ( CRA, I-SLICE, QP 1 ) 12326160 bits [Y 63.6726 dB U 63.9420 dB V 64.9370 dB] [ET 765 ] [L0] [L1]
WARNING: Encoded stream does not meet MinCr requirements numBytesInVclNalUnits (1809230) must be <= 1253376. Try increasing Qp, tier or level
POC 8 LId: 0 TId: 0 ( CRA, I-SLICE, QP -3 ) 14474496 bits [Y 67.6419 dB U 67.6652 dB V 68.8952 dB] [ET 845 ] [L0] [L1]
WARNING: Encoded stream does not meet MinCr requirements numBytesInVclNalUnits (2064611) must be <= 1253376. Try increasing Qp, tier or level

2. In rate allocation of CTUs of intra frame, variable noOfLCUsLeft is m_numberOfLCU - LCUIdx + 1, which is always 1 more than number of CTUs left. Thus it is able to be replaced by m_LCULeft.

     if (isIRAP)
     {
       int noOfLCUsLeft = m_numberOfLCU - LCUIdx + 1;
       int bitrateWindow = min(4,noOfLCUsLeft);
       double MAD      = getLCU(LCUIdx).m_costIntra;
   
       if (m_remainingCostIntra > 0.1 )
       {
         double weightedBitsLeft = (m_bitsLeft*bitrateWindow+(m_bitsLeft-getLCU(LCUIdx).m_targetBitsLeft)*noOfLCUsLeft)/(double)bitrateWindow;
         avgBits = int( MAD*weightedBitsLeft/m_remainingCostIntra );
       }
       else
       {
         avgBits = int( m_bitsLeft / m_LCULeft );
       }
       m_remainingCostIntra -= MAD;
     }
   

3. Updating alpha and beta of intra frames. According to R-D model of intra frames which is proposed in JCTVC-M0257, variable lnbpp should be ln(C/bpp). However, current equation for calculating lnbpp only consider ln(C), bpp is missing in this equation. We have checked the code, even in HM11.0, this equation is incorrect. (in the source code provided by M0257, this equation exactly matched with the document)

void EncRCPic::updateAlphaBetaIntra(double *alpha, double *beta)
{
  double lnbpp = log(pow(m_totalCostIntra / (double)m_numberOfPixel, BETA1));
  double diffLambda = (*beta)*(log((double)m_picActualBits)-log((double)m_targetBits));

  diffLambda = Clip3(-0.125, 0.125, 0.25*diffLambda);
  *alpha    =  (*alpha) * exp(diffLambda);
  *beta     =  (*beta) + diffLambda / lnbpp;
}

4. Chroma distortion weight adjustment is missing for CTU level RDO.

5. Source code of parameters updating is confusing. Many codes are duplicated and useless. It can be cleaned up for better reading.

[liufeiyang]

MR number is 1661.
Fixed in ​https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/merge_requests/1661