Mismatch between spec text and vtm on long-tap filter

[wzeng]

In JVET-O2001-vE 8.8.3.6.7,
– Otherwise, if one of the following conditions are true,

– maxFilterLengthQ is equal to 7 and maxFilterLengthP is equal to 5,
– maxFilterLengthQ is equal to 5 and maxFilterLengthP is equal to 7,

the following applies:
refMiddle = ( p4 + p3 + 2* ( p2 + p1 + p0 + q0 + q1 + q2 ) + q3 + q4 + 8 ) >> 4 (8-1178)

While in VTM6.0 function LoopFilter::xFilteringPandQ(),

if (newNumberPSide == 7 && newNumberQSide == 5)
{

refMiddle = (2 * (srcP[0] + srcQ[0] + srcP[-offset] + srcQ[offset]) + srcP[-2 * offset] + srcQ[2 * offset] + srcP[-3 * offset] + srcQ[3 * offset] + srcP[-4 * offset] + srcQ[4 * offset] + srcP[-5 * offset] + srcQ[5 * offset] + 8) >> 4;

}

I think VTM6.0 is correct. The spec text should be corrected to:

refMiddle=(2 ∗ (q0 + p0 + p1 + q1) + q2 + p2 + q3 + p3 + q4 + p4 + q5 + p5 + 8) >> 4 (8-1178)

[Kenneth]

Thanks for spotting the mismatch between text and software. You are correct, the spec should be corrected to the software. I suggest the following text:

refMiddle = ( p5 + p4 + p3 + p2 + 2*( p1 + p0 + q0 + q1 ) + q2 + q3 + q4 + q5 + 8 ) >> 4 (8-1178)

[bbross]

Good catch! Thanks!
Fixed in a document we are preparing to be submitted as an editorial input
to the 16th JVET meeting.