Opened 6 years ago
Closed 6 years ago
#305 closed defect (fixed)
MTS candidate ordering if LFNST is used
Reported by: | chollmann | Owned by: | |
---|---|---|---|
Priority: | minor | Milestone: | VTM-5.1 |
Component: | VTM | Version: | VTM-5.0 |
Keywords: | LFNST, MTS | Cc: | ksuehring, XiangLi, fbossen, jvet@… |
Description
When using LFNST, the encoder evaluates the MTS candidates (DST7, DCT8) and (DCT8, DST7) in an order based on the intra mode:
IntraSearch.cpp line 2919
| first | second intraMode < 34 | (DST7, DCT8) | (DCT8, DST7) intraMode >= 34 | (DCT8, DST7) | (DST7, DCT8)
This is done if a bool variable moreProbMTSIdxFirst is true. This bool is set based on whether the transform index is larger than 1 (see EncCu.cpp line 1459). However, transform index = 1 (which is (DCT8, DST7)) is one of the modes that are supposed to be sorted.
The fix corrects the comparison to transform index > 0.
Change history (2)
comment:1 Changed 6 years ago by ksuehring
comment:2 Changed 6 years ago by XiangLi
- Milestone set to VTM-5.1
- Resolution set to fixed
- Status changed from new to closed
Fixed as suggested.
Note: See TracTickets for help on using tickets.
Merge request 589 was submitted to address the issue.
https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/merge_requests/589