Issue in equation 1269 and 1270 of deblocking filter in the specification text

[chhuanb]

There seems to be a bug in the derivaion of the number of deblocking processing unit for chroma.

According to my understanding, for YUV420, the processing unit size for an edge is 2-pixel for chroma.
For example, if edgeType is EDGE_VER and nCbH is 8, the edge is divided into 4 units, with each unit containing 2-pixel rows.
However, according to equation 1270, "yN = edgeType = = EDGE_VER ? ( nCbH / 4 / subH ) − 1 : Max( 0, ( nCbH / gridSize ) − 1 )" below, if ((edgeType == EDGE_VER) && (nCbH == 8) && (subH == 2)) , yN will be 0, instead of 3.

8.8.3.6 Edge filtering process for one direction
– The variable gridSize is set as follows:
gridSize = cIdx = = 0 ? 4 : 8 (1266)
– The variables subW, subH, xN, yN are derived as follows:
subW = cIdx = = 0 ? 1 : SubWidthC (1267)
subH = cIdx = = 0 ? 1 : SubHeightC (1268)
xN = edgeType = = EDGE_VER ? Max( 0, ( nCbW / gridSize ) − 1 ) : ( nCbW / 4 / subW ) − 1 (1269)
yN = edgeType = = EDGE_VER ? ( nCbH / 4 / subH ) − 1 : Max( 0, ( nCbH / gridSize ) − 1 ) (1270)

Should it be:
xN = edgeType = = EDGE_VER ? Max( 0, ( nCbW / gridSize ) − 1 ) : ( nCbW / (4 / subW) ) − 1 (1269)
yN = edgeType = = EDGE_VER ? ( nCbH / (4 / subH) ) − 1 : Max( 0, ( nCbH / gridSize ) − 1 ) (1270)

[Kenneth]

Good catch. You are correct parentheses are missing around 4/subW and 4/subH in equations 1269 and 1270 respectively.

[bbross]

Thanks, will be fixed in JVET-S2001-v7.