Custom query (1557 matches)
Results (43 - 45 of 1557)
Ticket | Resolution | Summary | Owner | Reporter |
---|---|---|---|---|
#1566 | fixed | VTM encoder hits assert with --ISPFast=1 and --EncDbOpt =1 | Vadim | |
Description |
VTM encoder hits assert with --ISPFast=1 and --EncDbOpt=1 in debug mode (for example in AI), it seems ISPFast implementation has issues in mode cost comparison. If CHECKD is changed to CHECK, the encoder will crash in release mode as well. 1848: CHECKD( bestCostDct2NoIsp <= bestIspCost, "wrong cost!" ); ERROR: In function "EncCu::xCheckRDCostIntra" in H:\VVCSoftware_VTM\source\Lib\EncoderLib\EncCu.cpp:1848: wrong cost! Is this CHECKD correct? |
|||
#1565 | fixed | Invalid encoder memory access for EncDbOpt=1 and CHROMA_400 | regensky | |
Description |
In case the deblocking filter is applied during RDO ('EncDbOpt=1') and the chroma format is CHROMA_400, the encoder accesses invalid memory in RdCost::getDistPart (L. 429, L. 433)`. This leads to a segmentation fault (L. 438). The issue originates in EncCu::xCalDebCost (L. 4613, L. 4629): ComponentID compEnd = ( cu->isSepTree() && isLuma( partitioner.chType ) ) ? COMPONENT_Y : COMPONENT_Cr; COMPONENT_Cr is selected although CHROMA_400 features no COMPONENT_Cr. This can be solved by replacing the corresponding lines with ComponentID compEnd = isChromaEnabled(cu->chromaFormat) && !(cu->isSepTree() && isLuma(partitioner.chType)) ? COMPONENT_Cr : COMPONENT_Y; Command line to reproduce the issue: EncoderAppStatic -c cfg/encoder_randomaccess_vtm.cfg -i Tarsier_1920x1080_30fps_8bit_400.yuv -wdt 1920 -hgt 1080 --InputBitDepth=8 --InputChromaFormat=400 -fr 30 -f 1 -q 37 |
|||
#1564 | fixed | Intra prediction ref pixel array bounds too small for wide angle | swarrington | |
Description |
It appears that the range of the ref[] array specified in eq.332 contains insufficient samples for generating the prediction plane in the case of some rectangular regions. It seems it may be an equation error in the max bounds specified in eq.332. Consider: nCbW = 64 nCbH = 4 nTbW = 32 nTbH = 4 cIdx = 0 IntraSubPartitionsSplitType = ISP_NO_SPLIT refIdx = 0 Unmapped predModeIntra = 13
However, per eq.332, ref is only populated up to max index of 65. So it seems this process is accessing an unitialized value of ref[66]. Examining the VTM code for equivalent bounds handling to eq.332 in xPredIntraAng(): const int log2Ratio = floorLog2(width) - floorLog2(height); const int s = std::max<int>(0, bIsModeVer ? log2Ratio : -log2Ratio); const int maxIndex = (multiRefIdx << s) + 2; const Pel val = refMain[refLength + multiRefIdx]; for (int z = 1; z <= maxIndex; z++) {
} Per the spec eq.332, max index when refIdx==0 is refW+refIdx+x=64+0+(...*0) + 1 == 65, while for VTM maxIndex it is refLength+multiRefIdx+z = 64+0+(0<<...) + 2 == 66. |