Opened 5 years ago
Closed 5 years ago
#512 closed defect (fixed)
Palette crash with kimono-444, singletree, low-delay, lowqp (qp=2), PLT=1
Reported by: | gordon | Owned by: | |
---|---|---|---|
Priority: | minor | Milestone: | VTM-6.2 |
Component: | VTM | Version: | VTM-6.1 |
Keywords: | Cc: | ksuehring, XiangLi, fbossen, jvet@… |
Description
A 64-bit 'max' limit in IntraSearch::PLTSearch is initialized to a 32-bit value which is too small. When coding the summary line a crash occurs the first time palette is tested because all of the attempts searched have a bits value greater than this max, leaving things uninitialized for later treatment.
Initializing to MAX_UINT64 solves the problem.
diff --git a/source/Lib/EncoderLib/IntraSearch.cpp b/source/Lib/EncoderLib/IntraSearch.cpp index d9c2b82..b3380a9 100644 --- a/source/Lib/EncoderLib/IntraSearch.cpp +++ b/source/Lib/EncoderLib/IntraSearch.cpp @@ -1789,7 +1789,7 @@ void IntraSearch::PLTSearch(CodingStructure &cs, Partitioner& partitioner, Compo //calculate palette index preCalcPLTIndex(cs, partitioner, compBegin, numComp); //derive run - uint64_t bits = MAX_UINT; + uint64_t bits = MAX_UINT64; deriveRunAndCalcBits(cs, partitioner, compBegin, numComp, PLT_SCAN_HORTRAV, bits); if ((cu.curPLTSize[compBegin] + cu.useEscape[compBegin]) > 1) {
Change history (1)
comment:1 Changed 5 years ago by XiangLi
- Milestone set to VTM-6.2
- Resolution set to fixed
- Status changed from new to closed
Note: See TracTickets for help on using tickets.
Fixed in https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/merge_requests/921.