Opened 5 years ago

Closed 4 years ago

#205 closed defect (fixed)

Encoder fails encoding with --TransquantBypassEnable=1

Reported by: fbarbier Owned by:
Priority: minor Milestone:
Component: VTM Version: VTM-5.0rc1
Keywords: Cc: ksuehring, XiangLi, fbossen, jvet@…

Description

$ git checkout 6c6ea18debabe74ce264b4e4a70d8cb9370fc5d0
$ cd build
$ cmake ..
$ make -j 8
$ cd ..

$ bin/EncoderAppStatic --SourceWidth=240 --SourceHeight=64 --InputChromaFormat=420 --InputBitDepth=10 --Profile=next --FrameRate=60 --FramesToBeEncoded=1 --GOPSize=4 '--Frame1=B 1 5 -6.5 0.2590 0 0 1.0 0 0 0 4 4 -1 -5 -9 -13 0' '--Frame2=B 2 4 -6.5 0.2590 0 0 1.0 0 0 0 4 4 -1 -2 -6 -10 1 -1 5 1 1 1 0 1' '--Frame3=B 3 5 -6.5 0.2590 0 0 1.0 0 0 0 4 4 -1 -3 -7 -11 1 -1 5 0 1 1 1 1' '--Frame4=B 4 1 0.0 0.0 0 0 1.0 0 0 0 4 4 -1 -4 -8 -12 1 -1 5 0 1 1 1 1' --MaxCUWidth=8 --MaxCUHeight=8 --MaxPartitionDepth=5 --TULog2MaxSize=6 --SAO=0 --LoopFilterDisable=0 --TransquantBypassEnable=1 -i source.yuv -b encoded.vvc

terminate called after throwing an instance of 'std::out_of_range'

what(): bitset::set: position (which is 1041) >= _Nb (which is 1024)

Attachments (4)

source.yuv (45.0 KB) - added by fbarbier 5 years ago.
source.2.yuv (54.0 KB) - added by fbarbier 5 years ago.
source3.yuv (42.0 KB) - added by fbarbier 5 years ago.
source4.yuv (25.3 KB) - added by fbarbier 5 years ago.

Download all attachments as: .zip

Change history (10)

Changed 5 years ago by fbarbier

Changed 5 years ago by fbarbier

comment:1 Changed 5 years ago by fbarbier

source.2.yuv​ is a better input for this issue as the other one is not really a legal yuv (FRAME prefixing raw data)

comment:2 Changed 5 years ago by ksuehring

  • Version set to VTM-4.0.1

comment:3 Changed 5 years ago by fbarbier

Please find here the way to reproduce the issue with an up-to-date version :

git checkout ccf7079dbef3cc0b1f43dd8f8b3e2c4f63fe3254
cd build
cmake ..
make -j 8
cd ..

bin/EncoderAppStatic --SourceWidth=256 --SourceHeight=112 --InputChromaFormat=420 --InputBitDepth=8 --Profile=next --FrameRate=60 --FramesToBeEncoded=1 --GOPSize=1 --IntraPeriod=1 --MaxCUWidth=8 --MaxCUHeight=8 --MaxPartitionDepth=5 --TransquantBypassEnable=1 --CUTransquantBypassFlagForce=1 -i source3.yuv -b encoded.vvc

terminate called after throwing an instance of 'std::out_of_range'

what(): bitset::set: position (which is 1041) >= _Nb (which is 1024)

Changed 5 years ago by fbarbier

comment:4 Changed 5 years ago by fbarbier

  • Version changed from VTM-4.0.1 to VTM-5.0rc1

comment:5 Changed 5 years ago by fbarbier

git checkout 87322351f6739930c0fe81353a7911fc7e2f77f8
cd build
cmake ..
make -j 8
cd ..

./bin/EncoderAppStaticd --SourceWidth=216 --SourceHeight=8 --InputChromaFormat=420 --InputBitDepth=10 --Profile=next --FrameRate=60 --FramesToBeEncoded=5 --GOPSize=4 '--Frame1=B 1 5 -6.5 0.2590 0 0 1.0 0 0 0 4 4 -1 -5 -9 -13 0' '--Frame2=B 2 4 -6.5 0.2590 0 0 1.0 0 0 0 4 4 -1 -2 -6 -10 1 -1 5 1 1 1 0 1' '--Frame3=B 3 5 -6.5 0.2590 0 0 1.0 0 0 0 4 4 -1 -3 -7 -11 1 -1 5 0 1 1 1 1' '--Frame4=B 4 1 0.0 0.0 0 0 1.0 0 0 0 4 4 -1 -4 -8 -12 1 -1 5 0 1 1 1 1' --FastSearch=2 --LumaLevelToDeltaQPMode=0 --MaxCUWidth=16 --MaxCUHeight=24 --MaxPartitionDepth=5 --PCMEnabledFlag=0 --LCTUFast=0 --DepQuant=1 --TransquantBypassEnable=1 -i source4.yuv -b encoded.vvc

terminate called after throwing an instance of 'std::out_of_range'

what(): bitset::set: position (which is 838882304) >= _Nb (which is 1024)

Changed 5 years ago by fbarbier

comment:6 Changed 4 years ago by fbarbier

  • Resolution set to fixed
  • Status changed from new to closed

Obsolete option. I close it.

Note: See TracTickets for help on using tickets.