First, scaling list is not properly initialized for 128×128 transform. Second, error scale coefficients are not properly initialized for 2M×2N blocks when ((M+N)&0x01)≠0. Consequently, encoder sometimes makes incorrect rate-distortion decisions, which degrades overall coding performance. With the attached patch, 0.5+% luma BD-rate reduction is observed for RA, LD-B and LD-P without encoding time increase. (The issue is also reported in JVET-D0025.)

In JEM3.0 when this macro is off the encoder compresses the first I-frame then quits.

In the code paths of RDOQ off and SignHiding off, COM16_C983_RSAF is not properly implemented, which leads to encoding/decoding mismatch.