Opened 6 weeks ago

#1642 new defect

SIMD code for ALF classification does not match C code

Reported by: emora Owned by:
Priority: minor Milestone:
Component: VTM Version: VTM-23.0
Keywords: Cc: ksuehring, XiangLi, fbossen, jvet@…


The simdDeriveClassificationBlk_HBD() function does not match the deriveClassificationBlk() function. This condition:

if( (uint32_t)d1 * (uint32_t)hv0 > (uint32_t)hv1 * (uint32_t)d0 )

has been modified to:

if( (uint64_t)d1 * (uint64_t)hv0 > (uint64_t)hv1 * (uint64_t)d0 )

when RExt_HIGH_BIT_DEPTH_SUPPORT is enabled. However the corresponding SIMD code does not reflect this modification as _mm_mullo_epi32(d1, hv0) or _mm_mullo_epi32(hv1, d0) will only retain the first 32 bits of the multiplication result.

__m128i a = _mm_xor_si128(_mm_mullo_epi32(d1, hv0), _mm_set1_epi32(0x80000000));
__m128i b = _mm_xor_si128(_mm_mullo_epi32(hv1, d0), _mm_set1_epi32(0x80000000));

Change history (0)

Note: See TracTickets for help on using tickets.