Incorrect use of chroma distortion weights if Cb and Cr QP offsets differ (affects perceptual QPA, may affect SBT in HDR-PQ CTCs)

[crhelmrich]

VTM 6.x, in initEncSlice(), calls setUpLambda() to initialize the Cb and Cr distortion weights. These weights are computed, and multiplied onto the respective component distortions in RdCost::getDistPart(), independently for Cb and Cr. However, there are some places where an average chroma weight is calculated via RdCost::getChromaWeight() and used for both Cb and Cr. This does not seem to be an issue with the SDR or HDR-HLG CTCs, but if I understand correctly, different Cb and Cr QP offsets (and, thus, different Cb and Cr lambdas) may occur in the HDR-PQ CTCs.

Therefore, in the case of unequal Cb and Cr QP offsets, the following encoder functions may, according to my understanding, work suboptimally:

• rate control (if pCfg->getUseRateCtrl() in encodeCtus(), not sure if this is used)
• perceptual QPA with chroma QP adaptation (e.g., if pCfg->getUsePerceptQPA() && pcSlice->getPPS()->getUseDQP() in encodeCtus())
• SBT fast algorithm (InterSearch::calcMinDistSbt() called in xEncodeInterResidual()).

Before I submit a merge request with a proposed fix (used of a RdCost::getDistortionWeight(compID) instead of the RdCost::getChromaWeight() at the respective places), it would be great if someone could confirm that my understanding is correct.

Thanks and best,

Christian

[crhelmrich]

check of the effect of this fix on HDR CTCs

[crhelmrich]

Since I did not receive any comments on this, I went ahead and created merge request
​https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/merge_requests/1058
to fix the issue for the unequal Cb/Cr QP offsets and a related combination-with-JCCR case.

Note that I did NOT fix the SBT case since the fix would affect the HDR class 1 CTCs. See the attached Excel file for details, the results don't seem to show a benefit by fixing the InterSearch::calcMinDistSbt() function used by SBT.

Best,

Christian