Opened 3 years ago
Closed 3 years ago
#1515 closed defect (fixed)
Overflow in ALF derivation process when bitdepth > 10
Reported by: | marie-pierre.gallasso | Owned by: | |
---|---|---|---|
Priority: | minor | Milestone: | VTM-15.0 |
Component: | VTM | Version: | VTM-14.2 |
Keywords: | Cc: | ksuehring, XiangLi, fbossen, jvet@… |
Description
In the following condition (deriveClassificationBlk function in AdaptiveLoopFilter.cpp),
if( (uint32_t)d1 * (uint32_t)hv0 > (uint32_t)hv1 * (uint32_t)d0 )
the result of (uint32_t)d1 * (uint32_t)hv0 or (uint32_t)hv1 * (uint32_t)d0 may need more than 32 bits when bitdepth > 10.
The multiplication may overflow and the comparison of the two results may be wrong.
Change history (2)
comment:1 Changed 3 years ago by dmytro.rusanovskyy
comment:2 Changed 3 years ago by XiangLi
- Milestone changed from VTM-14.1 to VTM-15.0
- Resolution set to fixed
- Status changed from new to closed
- Version set to VTM-14.2
Fixed as suggested.
Note: See TracTickets for help on using tickets.
Software fix to address bug-report #1515 was produced. Suggested changes are the following:
#if RExtHIGH_BIT_DEPTH_SUPPORT
#else
#endif