Commit Graph

427 Commits

Author SHA1 Message Date
Geza Lore
b04ea832a4 Minor cleanup of inter mode search.
Change-Id: I523a3b30eb80fc6c6ed83638fdb82cf65c22b2e5
2016-07-01 09:00:05 +01:00
Debargha Mukherjee
e5e37e310b Merge "Reject ext-inter compound modes based on modelled RD." into nextgenv2 2016-06-30 18:18:53 +00:00
Geza Lore
532304e468 Reject ext-inter compound modes based on modelled RD.
Reject ext-inter compound modes before doing full rate distortion
evaluation, if the corresponding single reference modes had a lower
modelled RD.

ext-inter speedup up to TBD.

Coding performance: TBD

Change-Id: I358bfb879c5ebe5e7afbf6f540cc784f8de14857
2016-06-30 09:56:17 +01:00
Jingning Han
49222c3718 Fix shift value in dist_block with hbd
This offset value related to the bit depth has been taken care of
inside the function vp10_highbd_block_error.

Change-Id: I58dd8a53380ba4529d59837e56a951bc81a2962e
2016-06-29 16:42:23 -07:00
Geza Lore
92922be83c Remove skip_txfm optimization.
Commit 0d6980d7a1 removed some use
of the skip_txfm optimization, and the rest are not productive.

The current use of this optimization is only used with --good
and --cpu-used >= 3, however the overhead of this is higher than the
speedup it yields.

Removing this, and subsequently simplifying model_rd_for_sb yields
a net encoder speedup:
--cpu-used=0    ~1.5% faster
--cpu-used=3    ~2.0% faster

The code simplification is also significant.

Change-Id: I1dd668c32de15a2e912c59c42379d0f9e1032ff8
2016-06-28 10:03:03 +01:00
Debargha Mukherjee
f3dfa0c36a Quantization fix for new-quant/var-tx
Also use the fp quantizer consistently

lowres: -0.07 BDRATE improvement

Change-Id: I9174f6ad54a74d38541004b99cb3689d0c09be55
2016-06-27 17:22:09 -07:00
Jingning Han
813201e174 Disable trellis optimized quantization in the first-pass
This resolves the use of uninitialized value in the first-pass
encoding.

Change-Id: I78bc19214a1bfde5c5641424550cbbe4e52cae99
2016-06-27 12:46:07 -07:00
Sarah Parker
cc16c5d805 Merge "Add multiple quantization profiles to new_quant experiment" into nextgenv2 2016-06-27 18:46:25 +00:00
Sarah Parker
fbe6fb2773 Add multiple quantization profiles to new_quant experiment
Add the ability to pick between 3 quantization profiles.
The profile is chosen based on the entropy context at the
block level.

Change-Id: Iaea0485798441b7d635962c2563f3a477f582dac
2016-06-24 16:16:13 -07:00
Jingning Han
bf89ee7109 Merge "Use uniform quantizer for sub8x8 block coding" into nextgenv2 2016-06-24 17:07:46 +00:00
Jingning Han
0353e8d6b8 Merge "Refactor sub8x8 block transform and quantization process" into nextgenv2 2016-06-24 17:07:35 +00:00
Jingning Han
45c9add28c Merge "Make recursive txfm partitioning use uniform quantizer" into nextgenv2 2016-06-24 03:18:11 +00:00
Jingning Han
d100948ca2 Use uniform quantizer for sub8x8 block coding
Use the trellis optimization based uniform quantizer to encode the
sub8x8 block coding.

Change-Id: Ibbf7791b0aa430b7c67ef38eac3af6379578f56d
2016-06-23 17:01:00 -07:00
Jingning Han
bbe1b2217b Refactor sub8x8 block transform and quantization process
This commit refactors the transform and quantization process for
sub8x8 blocks and unifies the related functions.

Change-Id: I005f61f3eb49eec44f947b906c4e308cab9935a2
2016-06-23 16:56:05 -07:00
Jingning Han
db78fa9abb Make recursive txfm partitioning use uniform quantizer
Replace the expanded zero-bin quantizer with uniform quantizer in
the recursive transform block partitioning scheme. This improves
the compression performance by 0.4% for lowres.

Change-Id: I1c32ce9ebba0f0760e36a2c5bd20f2f5887ea5b4
2016-06-23 15:23:43 -07:00
Geza Lore
c9e7675c1a Force SIMPLE_TRANSLATION motion for SEG_LVL_SKIP blocks.
Change-Id: Ib8ac19f25d06351b8aabed742aa0be66e28ec4d4
2016-06-23 14:09:16 +01:00
Yue Chen
883e3b840f Merge "(Cosmetics) Remove unnecessary new parameters in obmc experiment" into nextgenv2 2016-06-22 17:22:47 +00:00
Debargha Mukherjee
78842b2870 Merge "Reinstate "Optimize wedge partition selection." without tests." into nextgenv2 2016-06-22 16:59:40 +00:00
Yue Chen
02596589e7 (Cosmetics) Remove unnecessary new parameters in obmc experiment
pred_variance in obmc experiment is equivalant to recon_variance in
baseline

Change-Id: Iba8fb9bd973898be5a0d87a507ceaf65c75bdc51
2016-06-22 06:24:32 +00:00
Alex Converse
27d3905cae Merge "Cleanup dist_block()" into nextgenv2 2016-06-22 00:16:10 +00:00
Geza Lore
135d663159 Reinstate "Optimize wedge partition selection." without tests.
This reinstates commit efda2831e5
without the tests and with fixes for 32 bit x86 builds.

Change-Id: I34be4fe1e8a67686d26ba256fd7efe0eb6a569e8
2016-06-21 20:31:50 +01:00
Debargha Mukherjee
7f929d292d Merge "Always respect tile bounds in calc_target_weighted_pred." into nextgenv2 2016-06-21 18:33:40 +00:00
Geza Lore
78bd14b38d Always respect tile bounds in calc_target_weighted_pred.
The tile boundaries should now be respected even between tile rows.
regardless of whether ext-tile is used or not.

Change-Id: I5a39fd274451114a4264215f97f12be2c908016d
2016-06-21 17:56:29 +01:00
Yue Chen
1273c39c03 Fix RDO issue of obmc + speed feature fast_inter_tx_type_search
Change-Id: I86a967ad2d824ca7877626eed9eb11f0e057b22d
2016-06-20 16:38:12 +00:00
Jingning Han
887f020691 Fix unit test failure in obmc exp
Properly restore the rate cost in the inner search loop of obmc
prediction. This avoids unexpected encoding behavior. It fixes
the unit test failure in obmc experiment:

AltRefForcedKeyTestLarge.Frame1IsKey/2

Change-Id: I667b219dfcf2f2c63d9d984900ed3cfd10c354bd
2016-06-17 17:44:03 -07:00
Zoe Liu
5805a14ca6 Merge bi-predictive frames to EXT_REFS
This patch removed the experiment of BIDIR_PRED and merged the feature
into the experiment of EXT_REFS:

(1) Each frame now has up to 6 reference frames, namely
    LAST_FRAME, LAST2_FRAME, LAST3_FRAME, GOLDEN_FRAME, (forward) and
    BWDREF_FRAME, ALTREF_FRAME (backward);
    LAST4_FRAME has been removed;
(2) First pass still keeps the 8 updates:
    KF_UPDATE, LF_UPDATE, GF_UPDATE, ARF_UPDATE, OVERLAY_UPDATE, and
    BRF_UPDATE, LAST_BIPRED_UPDATE, BI_PRED_UPDATE;
(3) show_existing_frame==1 is supported in the experiment of EXT_REFS;
(4) New encoding modes are added for both single-ref and compound cases,
    through the use of the 2 extra forward references (LAST2 & LAST3)
    and the 1 extra backward reference (BWDREF).

RD performance wise, using Overall PSNR: Avg/BDRate
        Bipred only      Prev EXT_REFS    Current EXT_REFS with bipred
lowres: -3.474/-3.324    -1.748/-1.586    -4.613/-4.387
derflr: -2.097/-1.353    -1.439/-1.215    -3.120/-2.252
midres: -2.129/-1.901    -1.345/-1.185    -2.898/-2.636

If in vp10/encoder/firstpass.h, change BFG_INTERVAL from 2 to 3, i.e. to
use 2 bi-predictive frames than 1, a further improvement may be
obtained:
                 Current EXT_REFS with bipred
        1 bi-predictive frame    2 bi-predictive frames
lowres: -4.613/-4.387            -4.675/-4.465
derflr: -3.120/-2.252            -3.333/-2.516
midres: -2.898/-2.636            -3.406/-3.095

Change-Id: Ib06fe9ea0a5cfd7418a1d79b978ee9d80bf191cb
2016-06-17 12:43:39 -07:00
Alex Converse
cf7968dcea Cleanup dist_block()
Change-Id: Iff0c0548924efd5a01c3a301cc5b4cdfda42e87e
2016-06-16 15:27:22 -07:00
Debargha Mukherjee
095c88e470 Merge "Fix estimate_wedge_sign with high bit-depth." into nextgenv2 2016-06-15 16:43:51 +00:00
Jingning Han
c457fc3553 Merge "Rework transform quantization pipeline" into nextgenv2 2016-06-15 16:07:10 +00:00
Jingning Han
08bf788ebd Merge "Refactor the trellis optimization process" into nextgenv2 2016-06-15 16:07:03 +00:00
Geza Lore
d3df694fa8 Fix estimate_wedge_sign with high bit-depth.
This fixes some crashes in
VP10/EndToEndTestLarge.EndtoEndPSNRTest/ with high bit-depth and
ext-inter.

Change-Id: I10f0f08e1be4bd5c388616074d4aa3f91a2fda7a
2016-06-15 10:52:29 +01:00
Hui Su
1f493d1ff8 Merge "Speed up ext-intra inter frame encoding" into nextgenv2 2016-06-15 02:46:06 +00:00
Hui Su
e7fb03c8ae Merge "ext-intra: refactor rd loop in interframe" into nextgenv2 2016-06-15 02:46:00 +00:00
Jingning Han
1faf288798 Rework transform quantization pipeline
This commit reworks the transform and quantization unit. It enables
the use of adaptive quantization for intra modes. This further
improves the compression performance:
lowres 0.36%
midres 0.79%
hdres  0.73%

The key frame coding performance is improved:
lowres 1.7%
midres 1.9%
hdres  3.3%

The overall coding gains are:
lowres 1.1%
midres 1.8%
hdres  2.3%

Change-Id: Iaec1a3a4c1d5eac883ab526ed076d957060479dd
2016-06-14 16:32:04 -07:00
Hui Su
69f6fd2134 Merge "Fix rate cost calculation for ext-intra" into nextgenv2 2016-06-14 23:11:25 +00:00
hui su
8c3b3d3686 Handle intra modes when tx type speed feature is enabled
Change-Id: I9dc156214f3b3ded33ab30d558124b3151548161
2016-06-14 13:46:53 -07:00
hui su
8f9c9b28a8 Speed up ext-intra inter frame encoding
Skip filter intra mode search when regular intra modes have large
rd cost.

Encoding speed improvement:  8%.

Compression performance drop: 0.02%  / 0.09%  / 0.03% on
                              lowres / midres / hdres

Change-Id: I94d3e48781bff6ae6895a54f271dd65c959bb976
2016-06-14 13:46:17 -07:00
hui su
70566f0563 ext-intra: refactor rd loop in interframe
Move filter intra modes search to the end, after regular
mode search.

On average no performance changes.

Change-Id: I9293c8fdf706ebf831fbd61c6bb81959790f4848
2016-06-14 13:46:17 -07:00
hui su
7fa61d7d51 Fix rate cost calculation for ext-intra
It was broken by commit 8ee640f979.

Change-Id: I26b9eba810c74849b0805e64da2d269ab0685cb9
2016-06-14 13:46:17 -07:00
Debargha Mukherjee
902ee5060c A crash fix for supertx / ext-inter combination.
Change-Id: I9860376c98aa3b25f5bf86ed13d4a7631fa6b153
2016-06-13 13:57:30 -07:00
Jingning Han
a9a8c5993b Refactor the trellis optimization process
Speed up the trellis optimization unit by 10%.

Change-Id: If055f6c0589a405c008d2900bb8fbc11b1246f66
2016-06-13 12:19:57 -07:00
Jingning Han
4588676cfb Merge "Trellis based adaptive quantization" into nextgenv2 2016-06-13 17:36:19 +00:00
Debargha Mukherjee
81f8b3f31c Merge "Some refactoring to support warped motion mode" into nextgenv2 2016-06-10 23:18:39 +00:00
Jingning Han
25ca322957 Trellis based adaptive quantization
This commit combines uniform quantizer with trellis based coefficient
level optimization. It improves the codebase compression performance:

lowres 0.8%
midres 1.0%
hdres  1.6%

Note that the current trellis optimization unit is using C code. This
will make the cost of the overall quantization process slower. A number
of optimizations will come up next.

Change-Id: Id441dd238e4844409d0f08f82604be777f3f5282
2016-06-10 12:56:14 -07:00
Debargha Mukherjee
03be30ba3e Some refactoring to support warped motion mode
Change-Id: I15d54a3ae48b2b33082668116792c6595bdb3ddb
2016-06-10 12:04:18 -07:00
Sarah Parker
a21afd421b Move new quant experiment from nextgen
This experiment implements non-uniform quantization where
the width of the bins increases gradually to more closely
match a laplacian distribution of the coeficcients.

Performance Gain:
derflr: 0.15%
hevcmr: 0.675%

Change-Id: I25234244e3bcd94b87c1f77cf682190b61c8ef94
2016-06-10 08:06:22 -07:00
Angie Chiang
95340fccb3 Revert "Optimize wedge partition selection."
This reverts commit efda2831e5.

This commit causes segmentation fault at SSE2/SumSquares2DTest.RandomValues/0

Change-Id: I171937e4daf6f15323e8206418773deb03bd8c53
2016-06-09 19:17:37 -07:00
Jingning Han
cedf90a9d6 Merge "Remove swap buffer speed feature" into nextgenv2 2016-06-08 19:45:54 +00:00
Jingning Han
0d6980d7a1 Remove swap buffer speed feature
The inter prediction residual can undergo different transform types
during the rate-distortion optimization search. The assumption used
in this speed feature no longer holds true. This commit removes the
related code to clean up the codebase and clear out unit test
failure in higher speed setting.

Change-Id: I7f7cd4df2345ed3e607c9fae75b38cd2dbde0cac
2016-06-08 11:27:00 -07:00
Jingning Han
b48eb90023 Merge "Add tx type speed feature to recursive transform block partitioning" into nextgenv2 2016-06-07 23:44:01 +00:00