Transform block edge not clearly defined (8.8.3.3)

[fbossen]

In 8.8.3.3 the following appears:

Otherwise, if the location ( xCb + x , yCb + y ) is at a transform block edge, edgeFlags[ x ][ y ] is set equal to 1.

It is unclear what a transform block edge is here. In particular there is no reference to the orientation of the edge or which colour component is referred to.

It is suggested to make the text more precise to avoid confusion. For example:

Otherwise, if the transform block covering the location ( xCb + x , yCb + y ) does not cover the location (xCb + x − 1, yCb + y ) for the colour component cIdx, edgeFlags[ x ][ y ] is set equal to 1.

[bbross]

Thanks, I agree that it does not hurt to further specify that. The suggested solution looks good to me and I will incorporate that into JVET-S2001-vF. However the other occurrence in this subclause need to be changed as well but for hor. edges with y-1 instead of x-1.

The third occurrence in 8.8.5.3 should be clear enough so I will not change that:
"Otherwise, if the block edge is also a transform block edge and the sample p0 or q0 is in a transform block which contains one or more non-zero transform coefficient levels, bS[ xDi ][ yDj ] is set equal to 1."

[fbossen]

Yes, there is second occurrence where you would want something like:

Otherwise, if the transform block covering the location ( xCb + x , yCb + y ) does not cover the location ( xCb + x , yCb + y − 1) for the colour component cIdx, edgeFlags[ x ][ y ] is set equal to 1.

As for the third occurrence in 8.8.3.5, the current use of edgeFlags is a bit strange:
value 0 means no transform block edge and no subblock edge
value 1 means transform block edge and no subblock edge
value 2 means subblock edge (but doesn't say anything about transform block edge)

If the use is changed to allow for 4 values to distinguish between the 4 possible combinations, say:
value 0 means no transform block edge and no subblock edge
value 1 means transform block edge and no subblock edge
value 2 means no transform block edge and subblock edge
value 3 means transform block edge and subblock edge

Then the checks in 8.8.3.5 can be
edgeFlags[ xDi ][ yDj ] % 2 is equal to 1 when checking for transform block edge, and
edgeFlags[ xDi ][ yDj ] / 2 is equal to 1 when checking for subblock edge

instead of currently
"the block edge is also a transform block edge", and
edgeFlags[ xDi ][ yDj ] is equal to 2