Incorrect bit cost count of force 0 ALF coefficients in ALF encoder RD estimation

[ezhizng]

There is a variable in ALF encoder RD estimation to count the bit cost of one ALF coefficient set when force set all ALF coefficients to 0.
The variable was defined as "static int". Therefore, the variable was updated through encoding the entire sequence.
The incorrect bit cost impact the RD cost estimation when comparing the RD cost of one derived ALF coefficient set to an all-zero ALF coefficient set (unfiltered ALF).
To fix this problem, there are two options:
opt1: re-initialize the variable to 0, before bit counts
opt2: define the variable as int (not static int)

One example of the fix code at: source/Lib/EncoderLib/EncAdaptiveLoopFilter.cpp
EncAdaptiveLoopFilter::getDistForce0()
{
1727 #if JVET_P0164_ALF_SYNTAX_SIMP
1728 static int zeroBitsVarBin = 0;
1729 zeroBitsVarBin = 0;
1730 for (int i = 0; i < alfShape.numCoeff - 1; i++)
1731 {
1732 zeroBitsVarBin += lengthGolomb(0, 3);
1733 }
1734 #endif
}

In the attachment, there is a encode log file of first two POCs. The test sequence was RaceHorses_416x240_30.yuv, encode setting was random access at qp37.

[ezhizng]

encodeLog

[jennylai]

yes, you are right.
The definition of this variable (zeroBitsVarBin) shall be int.
Thanks for reporting.

[fbossen]

Fixed here:
​https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/merge_requests/1488