Opened 8 months ago

Last modified 7 months ago

#728 new defect

Incorrect bit cost count of force 0 ALF coefficients in ALF encoder RD estimation

Reported by: ezhizng Owned by:
Priority: minor Milestone:
Component: VTM Version: VTM-7.0
Keywords: Cc: ksuehring, XiangLi, fbossen, jvet@…


There is a variable in ALF encoder RD estimation to count the bit cost of one ALF coefficient set when force set all ALF coefficients to 0.
The variable was defined as "static int". Therefore, the variable was updated through encoding the entire sequence.
The incorrect bit cost impact the RD cost estimation when comparing the RD cost of one derived ALF coefficient set to an all-zero ALF coefficient set (unfiltered ALF).
To fix this problem, there are two options:
opt1: re-initialize the variable to 0, before bit counts
opt2: define the variable as int (not static int)

One example of the fix code at: source/Lib/EncoderLib/EncAdaptiveLoopFilter.cpp
1728 static int zeroBitsVarBin = 0;
1729 zeroBitsVarBin = 0;
1730 for (int i = 0; i < alfShape.numCoeff - 1; i++)
1731 {
1732 zeroBitsVarBin += lengthGolomb(0, 3);
1733 }
1734 #endif

In the attachment, there is a encode log file of first two POCs. The test sequence was RaceHorses_416x240_30.yuv, encode setting was random access at qp37.

Attachments (1)

RA_POC_0_RaceHorses_416x240_30.txt (23.9 KB) - added by ezhizng 8 months ago.

Download all attachments as: .zip

Change history (2)

Changed 8 months ago by ezhizng


comment:1 Changed 7 months ago by jennylai

yes, you are right.
The definition of this variable (zeroBitsVarBin) shall be int.
Thanks for reporting.

Note: See TracTickets for help on using tickets.