Opened 4 years ago
Closed 4 years ago
#789 closed defect (fixed)
Log2 issue in CCLM
Reported by: | zhangkai | Owned by: | |
---|---|---|---|
Priority: | minor | Milestone: | |
Component: | spec | Version: | |
Keywords: | Cc: | ksuehring, bbross, XiangLi, fbossen, jvet@… |
Description
In 8.4.5.2.13 Specification of INTRA_LT_CCLM, INTRA_L_CCLM and INTRA_T_CCLM intra prediction mode
...
– If diff is not equal to 0, the following applies:
diffC = maxC − minC (410)
x = Floor( Log2( diff ) ) (411)
normDiff = ( ( diff << 4 ) >> x ) & 15 (412)
x += ( normDiff != 0 ) ? 1 : 0 (413)
y = Floor( Log2( Abs ( diffC ) ) ) + 1 (414)
a = ( diffC * ( divSigTable[ normDiff ] | 8 ) + 2y − 1 ) >> y (415)
...
Here, diffC may be equal to zero. Log2(0) is meaningless.
eq (414) should be changed to
y = diffC > 0 ? Floor( Log2( Abs ( diffC ) ) ) + 1 : 0
Change history (3)
comment:1 Changed 4 years ago by fbossen
comment:2 Changed 4 years ago by zhangkai
Yes, I agree. My initial intention is to change it to
y = Abs(diffC) > 0 ? Floor( Log2( Abs ( diffC ) ) ) + 1 : 0
Abs is missing in the ticket
comment:3 Changed 4 years ago by bbross
- Resolution set to fixed
- Status changed from new to closed
Thanks for reporting!
This has been fixed in JVET-Q2001-v5.
The suggested fix is incorrect as it will produce an incorrect result for negative values of diffC.
Alternate fixes to avoid Log2(0) would be:
y = Ceil( Log2( Abs( diffC ) + 1 ) ) )
or
y = diffC == 0 ? 0 : Floor( Log2( Abs ( diffC ) ) ) + 1
Also note that in (415) 2^{y-1} should probably be replaced by ( ( 1 << y ) >> 1 ) such that the result is 0 when y is 0.