Opened 5 years ago
Last modified 5 years ago
#468 new defect
ERROR: AddressSanitizer: unknown-crash
| Reported by: | fbarbier | Owned by: | |
|---|---|---|---|
| Priority: | minor | Milestone: | |
| Component: | VTM | Version: | VTM-6.1 |
| Keywords: | Cc: | ksuehring, XiangLi, fbossen, jvet@… |
Description
When building with ASAN the VTM 6.0, a crash may be found using the following command lines :
git checkout cb45f4386cf08d5b209333be61965a4beb4ff305 cd build readonly sanitizer="-fsanitize=address" cmake .. -DCMAKE_BUILD_TYPE=Debug -DCMAKE_CXX_FLAGS=$sanitizer -DCMAKE_EXE_LINKER_FLAGS=$sanitizer make -j 8 cd .. ./bin/EncoderAppStaticd --SourceWidth=192 --SourceHeight=128 --InputChromaFormat=420 --InputBitDepth=8 --Profile=next --FrameRate=60 --FramesToBeEncoded=1 --GOPSize=4 --Frame1="B 1 5 -6.5 0.2590 0 0 1.0 0 0 0 4 4 1 5 9 13 0 0" --Frame2="B 2 4 -6.5 0.2590 0 0 1.0 0 0 0 4 4 1 2 6 10 0 0" --Frame3="B 3 5 -6.5 0.2590 0 0 1.0 0 0 0 4 4 1 3 7 11 0 0" --Frame4="B 4 1 0.0 0.0 0 0 1.0 0 0 0 4 4 1 4 8 12 0 0" --FastSearch=2 --LCTUFast=1 --PBIntraFast=1 --CTUSize=64 --MaxPartitionDepth=5 --PCMEnabledFlag=1 --SAO=1 --ALF=1 --DepQuant=1 --DualITree=1 --MaxDeltaQP=2 --MTS=3 --IMV=1 --Affine=1 --AffineType=1 --Triangle=1 --SBT=1 -i source.yuv -b encoded.vvc
Output is :
VVCSoftware: VTM Encoder Version 6.0 [Linux][GCC 8.2.0][64 bit] [SIMD=AVX]
TOOL CFG: IBD:0 HAD:1 RDQ:1 RDQTS:1 RDpenalty:0 LQP:0 SQP:0 ASR:0 MinSearchWindow:8 RestrictMESampling:0 FEN:0 ECU:0 FDM:1 CFM:0 ESD:0 TransformSkip:0 TransformSkipFast:0 TransformSkipLog2MaxSize:5 BDPCM:0 Slice: M=0 Tiles:1x1 MCTS:0 CIP:0 SAO:1 ALF:1 PCM:1 TransQuantBypassEnabled:0 WPP:0 WPB:0 PME:2 WaveFrontSynchro:0 WaveFrontSubstreams:1 ScalingList:0 TMVPMode:1 DQ:1 SignBitHidingFlag:0 RecalQP:0
NEXT TOOL CFG: LFNST:0 MMVD:1 Affine:1 AffineType:1 PROF:0 SubPuMvp:0+0 DualITree:1 IMV:1 BIO:0 LMChroma:1 CclmCollocatedChroma:0 MTS: 1(intra) 1(inter) SBT:1 ISP:0 SMVD:0 CompositeLTReference:0 GBi:0 GBiFast:0 LADF:0 MHIntra:0 Triangle:1 AllowDisFracMMVD:0 AffineAmvr:0 AffineAmvrEncOpt:0 DMVR:0 MmvdDisNum:8 JointCbCr:0 PLT:0 IBC:0 HashME:0 WrapAround:0 LoopFilterAcrossVirtualBoundaries:0 Reshape:0 MIP:1 EncDbOpt:0
FAST TOOL CFG: LCTUFast:1 FastMrg:0 PBIntraFast:1 IMV4PelFast:1 MTSMaxCand: 3(intra) 4(inter) AMaxBT:0 E0023FastEnc:1 ContentBasedFastQtbt:0 UseNonLinearAlfLuma:1 UseNonLinearAlfChroma:1 MaxNumAlfAlternativesChroma:8 FastMIP:0 FastLocalDualTree:0 NumSplitThreads:1 NumWppThreads:1+0 EnsureWppBitEqual:0
started @ Mon Aug 19 17:12:37 2019
=================================================================
==28801==ERROR: AddressSanitizer: unknown-crash on address 0x62f00002933c at pc 0x55c7f6d1982a bp 0x7ffdbd761fc0 sp 0x7ffdbd761fb8
READ of size 16 at 0x62f00002933c thread T0
#0 0x55c7f6d19829 in _mm_loadu_si128(long long __vector(2) const*) /usr/lib/gcc/x86_64-linux-gnu/8/include/emmintrin.h:703
#1 0x55c7f6d19829 in simdDeriveClassificationBlk<(X86_VEXT)3> /home/fredb/projects/vtm/source/Lib/CommonLib/x86/avx/../AdaptiveLoopFilterX86.h:100
#2 0x55c7f721ced4 in AdaptiveLoopFilter::deriveClassification(AlfClassifier**, AreaBuf<short const> const&, Area const&, Area const&) /home/fredb/projects/vtm/source/Lib/CommonLib/AdaptiveLoopFilter.cpp:750
#3 0x55c7f70bc929 in EncAdaptiveLoopFilter::ALFProcess(CodingStructure&, double const*, double) /home/fredb/projects/vtm/source/Lib/EncoderLib/EncAdaptiveLoopFilter.cpp:770
#4 0x55c7f6e86a95 in EncGOP::compressGOP(int, int, std::__cxx11::list<Picture*, std::allocator<Picture*> >&, std::__cxx11::list<UnitBuf<short>*, std::allocator<UnitBuf<short>*> >&, bool, bool, InputColourSpaceConversion, bool, bool) /home/fredb/projects/vtm/source/Lib/EncoderLib/EncGOP.cpp:2518
#5 0x55c7f6ec2140 in EncLib::encode(bool, PelStorage*, PelStorage*, InputColourSpaceConversion, std::__cxx11::list<UnitBuf<short>*, std::allocator<UnitBuf<short>*> >&, int&) /home/fredb/projects/vtm/source/Lib/EncoderLib/EncLib.cpp:637
#6 0x55c7f6a17233 in EncApp::encode() /home/fredb/projects/vtm/source/App/EncoderApp/EncApp.cpp:779
#7 0x55c7f6ad29f9 in main /home/fredb/projects/vtm/source/App/EncoderApp/encmain.cpp:153
#8 0x7ffa43a1a09a in __libc_start_main ../csu/libc-start.c:308
#9 0x55c7f6a0adb9 in _start (/home/fredb/projects/vtm/bin/EncoderAppStaticd+0x339db9)
0x62f000029348 is located 0 bytes to the right of 53064-byte region [0x62f00001c400,0x62f000029348)
allocated by thread T0 here:
#0 0x7ffa44016038 in __interceptor_posix_memalign (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xea038)
#1 0x55c7f6aec604 in short* detail::aligned_malloc<short>(unsigned long, unsigned long) /home/fredb/projects/vtm/source/Lib/CommonLib/CommonDef.h:604
#2 0x55c7f6ae35ed in PelStorage::create(ChromaFormat const&, Area const&, unsigned int, unsigned int, unsigned int, bool) /home/fredb/projects/vtm/source/Lib/CommonLib/Buffer.cpp:799
#3 0x55c7f721b5cd in AdaptiveLoopFilter::create(int, int, ChromaFormat, int, int, int, int const*) /home/fredb/projects/vtm/source/Lib/CommonLib/AdaptiveLoopFilter.cpp:652
#4 0x55c7f70b71f9 in EncAdaptiveLoopFilter::create(EncCfg const*, int, int, ChromaFormat, int, int, int, int const*, int const*) /home/fredb/projects/vtm/source/Lib/EncoderLib/EncAdaptiveLoopFilter.cpp:423
#5 0x55c7f6ebe72f in EncLib::create() /home/fredb/projects/vtm/source/Lib/EncoderLib/EncLib.cpp:148
#6 0x55c7f6a16276 in EncApp::xCreateLib(std::__cxx11::list<UnitBuf<short>*, std::allocator<UnitBuf<short>*> >&) /home/fredb/projects/vtm/source/App/EncoderApp/EncApp.cpp:666
#7 0x55c7f6a1698f in EncApp::encode() /home/fredb/projects/vtm/source/App/EncoderApp/EncApp.cpp:713
#8 0x55c7f6ad29f9 in main /home/fredb/projects/vtm/source/App/EncoderApp/encmain.cpp:153
#9 0x7ffa43a1a09a in __libc_start_main ../csu/libc-start.c:308
SUMMARY: AddressSanitizer: unknown-crash /usr/lib/gcc/x86_64-linux-gnu/8/include/emmintrin.h:703 in _mm_loadu_si128(long long __vector(2) const*)
Shadow bytes around the buggy address:
0x0c5e7fffd210: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c5e7fffd220: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c5e7fffd230: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c5e7fffd240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c5e7fffd250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c5e7fffd260: 00 00 00 00 00 00 00[00]00 fa fa fa fa fa fa fa
0x0c5e7fffd270: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c5e7fffd280: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c5e7fffd290: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c5e7fffd2a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c5e7fffd2b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==28801==ABORTING
Attachments (1)
Change history (5)
Changed 5 years ago by fbarbier
comment:1 Changed 5 years ago by fbarbier
- Version changed from VTM-6.0rc1 to VTM-6.1
comment:2 Changed 5 years ago by ksuehring
I tried reproducing with clang/Xcode. I had to remove EnablePCM from the command line. Now I'm getting a heap overflow in ALF SIMD code:
VVCSoftware: VTM Encoder Version 6.1 [Mac OS X][clang 10.0.1][64 bit] [SIMD=AVX2]
TOOL CFG: IBD:0 HAD:1 RDQ:1 RDQTS:1 RDpenalty:0 LQP:0 SQP:0 ASR:0 MinSearchWindow:8 RestrictMESampling:0 FEN:0 ECU:0 FDM:1 CFM:0 ESD:0 TransformSkip:0 TransformSkipFast:0 TransformSkipLog2MaxSize:5 BDPCM:0 Slice: M=0 Tiles:1x1 MCTS:0 CIP:0 SAO:1 ALF:1 TransQuantBypassEnabled:0 WPP:0 WPB:0 PME:2 WaveFrontSynchro:0 WaveFrontSubstreams:1 ScalingList:0 TMVPMode:1 DQ:1 SignBitHidingFlag:0 RecalQP:0
NEXT TOOL CFG: LFNST:0 MMVD:1 Affine:1 AffineType:1 PROF:0 SubPuMvp:0+0 DualITree:1 IMV:1 BIO:0 LMChroma:1 CclmCollocatedChroma:0 MTS: 1(intra) 1(inter) SBT:1 ISP:0 SMVD:0 CompositeLTReference:0 GBi:0 GBiFast:0 LADF:0 MHIntra:0 Triangle:1 AllowDisFracMMVD:0 AffineAmvr:0 AffineAmvrEncOpt:0 DMVR:0 MmvdDisNum:8 JointCbCr:0 PLT:0 IBC:0 HashME:0 WrapAround:0 LoopFilterAcrossVirtualBoundaries:0 Reshape:0 MIP:1 EncDbOpt:0
FAST TOOL CFG: LCTUFast:1 FastMrg:0 PBIntraFast:1 IMV4PelFast:1 MTSMaxCand: 3(intra) 4(inter) AMaxBT:0 E0023FastEnc:1 ContentBasedFastQtbt:0 UseNonLinearAlfLuma:1 UseNonLinearAlfChroma:1 MaxNumAlfAlternativesChroma:8 FastMIP:0 FastLocalDualTree:0 NumSplitThreads:1 NumWppThreads:1+0 EnsureWppBitEqual:0 RPR:0
started @ Thu Sep 12 16:43:44 2019
=================================================================
==45197==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x62f0000c334b at pc 0x000100746b73 bp 0x7ffeefbf94d0 sp 0x7ffeefbf94c8
READ of size 16 at 0x62f0000c334b thread T0
2019-09-12 16:46:51.583453+0200 atos[45222:3599789] examining /Users/USER/*/EncoderApp [45197]
#0 0x100746b72 in void simdDeriveClassificationBlk<(X86_VEXT)4>(AlfClassifier**, int***, AreaBuf<short const> const&, Area const&, Area const&, int, int, int) AdaptiveLoopFilterX86.h:94
#1 0x100104f33 in AdaptiveLoopFilter::deriveClassification(AlfClassifier**, AreaBuf<short const> const&, Area const&, Area const&) AdaptiveLoopFilter.cpp:733
#2 0x1009d545d in EncAdaptiveLoopFilter::ALFProcess(CodingStructure&, double const*, double) EncAdaptiveLoopFilter.cpp:780
#3 0x100adc660 in EncGOP::compressGOP(int, int, std::__1::list<Picture*, std::__1::allocator<Picture*> >&, std::__1::list<UnitBuf<short>*, std::__1::allocator<UnitBuf<short>*> >&, bool, bool, InputColourSpaceConversion, bool, bool) EncGOP.cpp:2757
#4 0x100b568fe in EncLib::encode(bool, PelStorage*, PelStorage*, InputColourSpaceConversion, std::__1::list<UnitBuf<short>*, std::__1::allocator<UnitBuf<short>*> >&, int&) EncLib.cpp:807
#5 0x10001ce7e in EncApp::encode() EncApp.cpp:818
#6 0x1000e1ccd in main encmain.cpp:153
#7 0x7fff6c9ef3d4 in start (libdyld.dylib:x86_64+0x163d4)
0x62f0000c334b is located 3 bytes to the right of 53064-byte region [0x62f0000b6400,0x62f0000c3348)
allocated by thread T0 here:
#0 0x103b909c4 in wrap_posix_memalign (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x5c9c4)
#1 0x10014a2a9 in short* detail::aligned_malloc<short>(unsigned long, unsigned long) CommonDef.h:592
#2 0x1001496f1 in PelStorage::create(ChromaFormat const&, Area const&, unsigned int, unsigned int, unsigned int, bool) Buffer.cpp:799
#3 0x100107a4e in AdaptiveLoopFilter::create(int, int, ChromaFormat, int, int, int, int const*) AdaptiveLoopFilter.cpp:663
#4 0x1009ceaff in EncAdaptiveLoopFilter::create(EncCfg const*, int, int, ChromaFormat, int, int, int, int const*, int const*) EncAdaptiveLoopFilter.cpp:428
#5 0x100adc0e0 in EncGOP::compressGOP(int, int, std::__1::list<Picture*, std::__1::allocator<Picture*> >&, std::__1::list<UnitBuf<short>*, std::__1::allocator<UnitBuf<short>*> >&, bool, bool, InputColourSpaceConversion, bool, bool) EncGOP.cpp:2749
#6 0x100b568fe in EncLib::encode(bool, PelStorage*, PelStorage*, InputColourSpaceConversion, std::__1::list<UnitBuf<short>*, std::__1::allocator<UnitBuf<short>*> >&, int&) EncLib.cpp:807
#7 0x10001ce7e in EncApp::encode() EncApp.cpp:818
#8 0x1000e1ccd in main encmain.cpp:153
#9 0x7fff6c9ef3d4 in start (libdyld.dylib:x86_64+0x163d4)
SUMMARY: AddressSanitizer: heap-buffer-overflow AdaptiveLoopFilterX86.h:94 in void simdDeriveClassificationBlk<(X86_VEXT)4>(AlfClassifier**, int***, AreaBuf<short const> const&, Area const&, Area const&, int, int, int)
Shadow bytes around the buggy address:
0x1c5e00018610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x1c5e00018620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x1c5e00018630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x1c5e00018640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x1c5e00018650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x1c5e00018660: 00 00 00 00 00 00 00 00 00[fa]fa fa fa fa fa fa
0x1c5e00018670: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x1c5e00018680: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x1c5e00018690: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x1c5e000186a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x1c5e000186b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
2019-09-12 16:46:52.340189+0200 EncoderApp[45197:3593337] =================================================================
2019-09-12 16:46:52.340284+0200 EncoderApp[45197:3593337] ==45197==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x62f0000c334b at pc 0x000100746b73 bp 0x7ffeefbf94d0 sp 0x7ffeefbf94c8
2019-09-12 16:46:52.340298+0200 EncoderApp[45197:3593337] READ of size 16 at 0x62f0000c334b thread T0
2019-09-12 16:46:52.340308+0200 EncoderApp[45197:3593337] #0 0x100746b72 in void simdDeriveClassificationBlk<(X86_VEXT)4>(AlfClassifier**, int***, AreaBuf<short const> const&, Area const&, Area const&, int, int, int) AdaptiveLoopFilterX86.h:94
2019-09-12 16:46:52.340321+0200 EncoderApp[45197:3593337] #1 0x100104f33 in AdaptiveLoopFilter::deriveClassification(AlfClassifier**, AreaBuf<short const> const&, Area const&, Area const&) AdaptiveLoopFilter.cpp:733
2019-09-12 16:46:52.340339+0200 EncoderApp[45197:3593337] #2 0x1009d545d in EncAdaptiveLoopFilter::ALFProcess(CodingStructure&, double const*, double) EncAdaptiveLoopFilter.cpp:780
2019-09-12 16:46:52.340407+0200 EncoderApp[45197:3593337] #3 0x100adc660 in EncGOP::compressGOP(int, int, std::__1::list<Picture*, std::__1::allocator<Picture*> >&, std::__1::list<UnitBuf<short>*, std::__1::allocator<UnitBuf<short>*> >&, bool, bool, InputColourSpaceConversion, bool, bool) EncGOP.cpp:2757
2019-09-12 16:46:52.340552+0200 EncoderApp[45197:3593337] #4 0x100b568fe in EncLib::encode(bool, PelStorage*, PelStorage*, InputColourSpaceConversion, std::__1::list<UnitBuf<short>*, std::__1::allocator<UnitBuf<short>*> >&, int&) EncLib.cpp:807
2019-09-12 16:46:52.340568+0200 EncoderApp[45197:3593337] #5 0x10001ce7e in EncApp::encode() EncApp.cpp:818
2019-09-12 16:46:52.340578+0200 EncoderApp[45197:3593337] #6 0x1000e1ccd in main encmain.cpp:153
2019-09-12 16:46:52.340616+0200 EncoderApp[45197:3593337] #7 0x7fff6c9ef3d4 in start (libdyld.dylib:x86_64+0x163d4)
2019-09-12 16:46:52.340627+0200 EncoderApp[45197:3593337]
2019-09-12 16:46:52.340633+0200 EncoderApp[45197:3593337] 0x62f0000c334b is located 3 bytes to the right of 53064-byte region [0x62f0000b6400,0x62f0000c3348)
2019-09-12 16:46:52.340640+0200 EncoderApp[45197:3593337] allocated by thread T0 here:
2019-09-12 16:46:52.340646+0200 EncoderApp[45197:3593337] #0 0x103b909c4 in wrap_posix_memalign (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x5c9c4)
2019-09-12 16:46:52.340724+0200 EncoderApp[45197:3593337] #1 0x10014a2a9 in short* detail::aligned_malloc<short>(unsigned long, unsigned long) CommonDef.h:592
2019-09-12 16:46:52.340740+0200 EncoderApp[45197:3593337] #2 0x1001496f1 in PelStorage::create(ChromaFormat const&, Area const&, unsigned int, unsigned int, unsigned int, bool) Buffer.cpp:799
2019-09-12 16:46:52.340748+0200 EncoderApp[45197:3593337] #3 0x100107a4e in AdaptiveLoopFilter::create(int, int, ChromaFormat, int, int, int, int const*) AdaptiveLoopFilter.cpp:663
2019-09-12 16:46:52.340784+0200 EncoderApp[45197:3593337] #4 0x1009ceaff in EncAdaptiveLoopFilter::create(EncCfg const*, int, int, ChromaFormat, int, int, int, int const*, int const*) EncAdaptiveLoopFilter.cpp:428
2019-09-12 16:46:52.340801+0200 EncoderApp[45197:3593337] #5 0x100adc0e0 in EncGOP::compressGOP(int, int, std::__1::list<Picture*, std::__1::allocator<Picture*> >&, std::__1::list<UnitBuf<short>*, std::__1::allocator<UnitBuf<short>*> >&, bool, bool, InputColourSpaceConversion, bool, bool) EncGOP.cpp:2749
2019-09-12 16:46:52.340922+0200 EncoderApp[45197:3593337] #6 0x100b568fe in EncLib::encode(bool, PelStorage*, PelStorage*, InputColourSpaceConversion, std::__1::list<UnitBuf<short>*, std::__1::allocator<UnitBuf<short>*> >&, int&) EncLib.cpp:807
2019-09-12 16:46:52.340943+0200 EncoderApp[45197:3593337] #7 0x10001ce7e in EncApp::encode() EncApp.cpp:818
2019-09-12 16:46:52.340964+0200 EncoderApp[45197:3593337] #8 0x1000e1ccd in main encmain.cpp:153
2019-09-12 16:46:52.340977+0200 EncoderApp[45197:3593337] #9 0x7fff6c9ef3d4 in start (libdyld.dylib:x86_64+0x163d4)
2019-09-12 16:46:52.340995+0200 EncoderApp[45197:3593337]
2019-09-12 16:46:52.341009+0200 EncoderApp[45197:3593337] SUMMARY: AddressSanitizer: heap-buffer-overflow AdaptiveLoopFilterX86.h:94 in void simdDeriveClassificationBlk<(X86_VEXT)4>(AlfClassifier**, int***, AreaBuf<short const> const&, Area const&, Area const&, int, int, int)
2019-09-12 16:46:52.341023+0200 EncoderApp[45197:3593337] Shadow bytes around the buggy address:
2019-09-12 16:46:52.341096+0200 EncoderApp[45197:3593337] 0x1c5e00018610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2019-09-12 16:46:52.341110+0200 EncoderApp[45197:3593337] 0x1c5e00018620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2019-09-12 16:46:52.341124+0200 EncoderApp[45197:3593337] 0x1c5e00018630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2019-09-12 16:46:52.341153+0200 EncoderApp[45197:3593337] 0x1c5e00018640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2019-09-12 16:46:52.341163+0200 EncoderApp[45197:3593337] 0x1c5e00018650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2019-09-12 16:46:52.341196+0200 EncoderApp[45197:3593337] =>0x1c5e00018660: 00 00 00 00 00 00 00 00 00[fa]fa fa fa fa fa fa
2019-09-12 16:46:52.341205+0200 EncoderApp[45197:3593337] 0x1c5e00018670: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
2019-09-12 16:46:52.341213+0200 EncoderApp[45197:3593337] 0x1c5e00018680: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
2019-09-12 16:46:52.341339+0200 EncoderApp[45197:3593337] 0x1c5e00018690: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
2019-09-12 16:46:52.341358+0200 EncoderApp[45197:3593337] 0x1c5e000186a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
2019-09-12 16:46:52.341372+0200 EncoderApp[45197:3593337] 0x1c5e000186b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
2019-09-12 16:46:52.341384+0200 EncoderApp[45197:3593337] Shadow byte legend (one shadow byte represents 8 application bytes):
2019-09-12 16:46:52.341394+0200 EncoderApp[45197:3593337] Addressable: 00
2019-09-12 16:46:52.341405+0200 EncoderApp[45197:3593337] Partially addressable: 01 02 03 04 05 06 07
2019-09-12 16:46:52.341416+0200 EncoderApp[45197:3593337] Heap left redzone: fa
2019-09-12 16:46:52.341428+0200 EncoderApp[45197:3593337] Freed heap region: fd
2019-09-12 16:46:52.341440+0200 EncoderApp[45197:3593337] Stack left redzone: f1
2019-09-12 16:46:52.341452+0200 EncoderApp[45197:3593337] Stack mid redzone: f2
2019-09-12 16:46:52.341517+0200 EncoderApp[45197:3593337] Stack right redzone: f3
2019-09-12 16:46:52.341538+0200 EncoderApp[45197:3593337] Stack after return: f5
2019-09-12 16:46:52.341566+0200 EncoderApp[45197:3593337] Stack use after scope: f8
2019-09-12 16:46:52.341580+0200 EncoderApp[45197:3593337] Global redzone: f9
2019-09-12 16:46:52.341587+0200 EncoderApp[45197:3593337] Global init order: f6
2019-09-12 16:46:52.341593+0200 EncoderApp[45197:3593337] Poisoned by user: f7
2019-09-12 16:46:52.341599+0200 EncoderApp[45197:3593337] Container overflow: fc
2019-09-12 16:46:52.341604+0200 EncoderApp[45197:3593337] Array cookie: ac
2019-09-12 16:46:52.341624+0200 EncoderApp[45197:3593337] Intra object redzone: bb
2019-09-12 16:46:52.341638+0200 EncoderApp[45197:3593337] ASan internal: fe
2019-09-12 16:46:52.341645+0200 EncoderApp[45197:3593337] Left alloca redzone: ca
2019-09-12 16:46:52.341683+0200 EncoderApp[45197:3593337] Right alloca redzone: cb
2019-09-12 16:46:52.341702+0200 EncoderApp[45197:3593337] Shadow gap: cc
2019-09-12 16:46:52.341716+0200 EncoderApp[45197:3593337]
==45197==ABORTING
AddressSanitizer report breakpoint hit. Use 'thread info -s' to get extended information about the report.
(lldb)
comment:3 Changed 5 years ago by ksuehring
Apparently this can be fixed by giving the ALF temporary buffer a bit bigger margin, e.g.:
AdaptiveLoopFilter::create()
change
m_tempBuf.create( format, Area( 0, 0, picWidth, picHeight ), maxCUWidth, MAX_ALF_FILTER_LENGTH >> 1, 0, false );
to
m_tempBuf.create( format, Area( 0, 0, picWidth, picHeight ), maxCUWidth, MAX_ALF_FILTER_LENGTH, 0, false );
comment:4 Changed 5 years ago by fbossen
The core issue is that the SIMD code processes data in bunches of 8 samples. However the number of elements that are processed is a multiple of 4. The SIMD code may thus attempt to read an extra 4 elements in some cases. Allocating an additional 4 samples for m_tempBuf resolves the ASAN issue.
The allocation of m_tempBuf is not super clean to start with. The classifier assumes a margin of 3 which is independent of the max ALF filter length, so the margin shouldn't simply depend on the max filter length. Increasing the margin from MAX_ALF_FILTER_LENGTH >> 1 to MAX_ALF_FILTER_LENGTH works, but it somewhat a hack.
I would suggest modifying the SIMD code to not read more data than required and modify the margin in the allocation to min(3, MAX_ALF_FILTER_LENGTH >> 1).
Note: See TracTickets for help on using tickets.
still present on 6.1