Opened 5 years ago
Closed 4 years ago
#987 closed defect (fixed)
Assertion on min_qp_prime_ts_minus4 failed at decoder when internalBitDepth is set to 8 and inputBitDepth is set to 10
Reported by: | tsukuba.takeshi | Owned by: | |
---|---|---|---|
Priority: | minor | Milestone: | VTM-8.1 |
Component: | VTM | Version: | VTM-8.0 |
Keywords: | Cc: | ksuehring, XiangLi, fbossen, jvet@… |
Description
In VTM-8.0, min_qp_prime_ts_minus4 is set to (6*(internalBitDepth - inputBitDepth)) at encoder as below:
void EncLib::xInitSPS( SPS& sps ) { ... sps.setMinQpPrimeTsMinus4(ChannelType(channelType), (6 * (m_bitDepth[channelType] - m_inputBitDepth[channelType]))); ... }
When internalBitDepth is set to 8 and inputBitDepth is set to 10, the value of min_qp_prime_ts_minus4 becomes -12 (=6*(8-10)), and the signed value of -12 is coded by UVLC as below:
WRITE_UVLC(pcSPS->getMinQpPrimeTsMinus4(CHANNEL_TYPE_LUMA), "min_qp_prime_ts_minus4");
The decoded value of min_qp_prime_ts_minus4 by UVLC becomes the unsigned value of 4294967284 at decoder.
Thus, following assertion on min_qp_prime_ts_minus4 failed at decoder:
CHECK(uiCode > 48, "Invalid min_qp_prime_ts_minus4 signalled");
Suggested fix is to change inputBitDepth in min_qp_prime_ts_minus4 derivation to 8bit as below:
sps.setMinQpPrimeTsMinus4(ChannelType(channelType), (6 * (m_bitDepth[channelType] - 8)));
i.e., min_qp_prime_ts_minus4 is set to (6*(internalBitDepth - 8)) at encoder.
Change history (7)
comment:1 Changed 5 years ago by tsukuba.takeshi
comment:2 Changed 5 years ago by XiangLi
Thanks for the report. It might be better to clarify why internalBitDepth is set to 8 given inputBitDepth is 10? We are losing quality before encoding. In this case, should we print out an error message and stop the encoder to avoid confusion?
comment:3 Changed 5 years ago by yhchao
Hi Xiang,
If the derivation is changed to (6*(internalBitDepth - 8)), it will cause quality loss at low QP if the inputBitDepth is 10. The detail can be found in our proposal JVET-O0919. I think the solution you provide, i.e., printing error message, makes sense. Another solution is to clip min_qp_prime_ts_minus4 to 0 when it goes to negative.
comment:4 Changed 5 years ago by fbossen
It should be possible to encode a 10-bit source with 8-bit precision (or alternatively a 12-bit source with 10-bit precision). We shouldn't disallow such a combination. Printing a warning message may be helpful. Clipping min_qp_prime_ts_minus4 to 0 at the low end seems to make sense.
comment:5 Changed 5 years ago by tsukuba.takeshi
Thanks a lot for your feedback.
I agree that clipping min_qp_prime_ts_minus4 to 0 at the low end is a better solution since it has no impacts to other conditions "inputBitDepth>=internalBitDepth".
So, I will withdraw MR!1481 and create a new MR including:
- Clipping min_qp_prime_ts_minus4 to 0 at the low end
- Print a warning message when intenalBitDepth is lower than inputBitDepth at xCheckParameter()
comment:6 Changed 5 years ago by tsukuba.takeshi
Please take a loot at new MR!1483:
https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/merge_requests/1483
comment:7 Changed 4 years ago by tsukuba.takeshi
- Resolution set to fixed
- Status changed from new to closed
MR!1483 was already merged.
Suggested MR is https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/merge_requests/1481