122 Commits

Author SHA1 Message Date
Debargha Mukherjee
e5848dea5a Rectangular transforms 4x8 & 8x4
Added a new expt rect-tx to be used in conjunction with ext-tx.
[rect-tx is a temporary config flag and will eventually be
merged into ext-tx once it works correctly with all other
experiments].

Added 4x8 and 8x4 tranforms for use initially with rectangular
sub8x8 y blocks as part of this experiment.

There is about a -0.2% BDRATE improvement on lowres, others pending.

When var-tx is on rectangular transforms are currently not used.
That will be enabled in a subsequent patch.

Change-Id: Iaf3f88ede2740ffe6a0ffb1ef5fc01a16cd0283a
2016-07-21 10:46:41 -07:00
Zoe Liu
5805a14ca6 Merge bi-predictive frames to EXT_REFS
This patch removed the experiment of BIDIR_PRED and merged the feature
into the experiment of EXT_REFS:

(1) Each frame now has up to 6 reference frames, namely
    LAST_FRAME, LAST2_FRAME, LAST3_FRAME, GOLDEN_FRAME, (forward) and
    BWDREF_FRAME, ALTREF_FRAME (backward);
    LAST4_FRAME has been removed;
(2) First pass still keeps the 8 updates:
    KF_UPDATE, LF_UPDATE, GF_UPDATE, ARF_UPDATE, OVERLAY_UPDATE, and
    BRF_UPDATE, LAST_BIPRED_UPDATE, BI_PRED_UPDATE;
(3) show_existing_frame==1 is supported in the experiment of EXT_REFS;
(4) New encoding modes are added for both single-ref and compound cases,
    through the use of the 2 extra forward references (LAST2 & LAST3)
    and the 1 extra backward reference (BWDREF).

RD performance wise, using Overall PSNR: Avg/BDRate
        Bipred only      Prev EXT_REFS    Current EXT_REFS with bipred
lowres: -3.474/-3.324    -1.748/-1.586    -4.613/-4.387
derflr: -2.097/-1.353    -1.439/-1.215    -3.120/-2.252
midres: -2.129/-1.901    -1.345/-1.185    -2.898/-2.636

If in vp10/encoder/firstpass.h, change BFG_INTERVAL from 2 to 3, i.e. to
use 2 bi-predictive frames than 1, a further improvement may be
obtained:
                 Current EXT_REFS with bipred
        1 bi-predictive frame    2 bi-predictive frames
lowres: -4.613/-4.387            -4.675/-4.465
derflr: -3.120/-2.252            -3.333/-2.516
midres: -2.898/-2.636            -3.406/-3.095

Change-Id: Ib06fe9ea0a5cfd7418a1d79b978ee9d80bf191cb
2016-06-17 12:43:39 -07:00
Debargha Mukherjee
03be30ba3e Some refactoring to support warped motion mode
Change-Id: I15d54a3ae48b2b33082668116792c6595bdb3ddb
2016-06-10 12:04:18 -07:00
Geza Lore
6c4306c27d Fix decoder crash with supertx
xd->plane[0].n4_h and xd->plane[0].n4_w are not set at that point
when using supertx.

While this fixes the immediate crash described in the referenced
bug report, there are still issues in the ref-mv experiment that
causes these tests to fail, so they are kept disabled.

BUG=https://bugs.chromium.org/p/webm/issues/detail?id=1230

Change-Id: Ibf8ef02847a903f8d10e6be28e16694db10c75af
2016-06-06 09:58:11 +01:00
Alex Converse
aee0091161 Replace some vpxbool calls with entropy coder agnostic calls.
Change-Id: Ifbcd0714fcf994c43b69255185456c7a255df66c
2016-05-31 15:42:19 -07:00
hui su
e5f47d4334 ext-intra: refactor mode info. writing and reading
No performance changes.

Change-Id: I001068330ea217a993aee9b79d7ffead0d23100e
2016-05-26 14:56:40 -07:00
Zoe Liu
cf5083d4cd Added an experiment "bidir_pred" for backward prediction
Major parts have been implemented as follows:
(1) Added BRF_UPDATE, LASTNRF_UPDATE, and NRF_UPDATE in firstpass.c;
(2) Added the handling for the scenario of
"cpi->common.show_existing_frame == 1" at the encoder;
(3) Added a new reference frame of BWDREF_FRAME;
(4) Have bwd-ref work with upsampled references.

Note that when the experiment of "ext_refs" turned on, this experiment
will be turned off automatically currently.

RD performance in Overall PSNR has been improved, compared against the
VP10 baseline:

lowres: Avg -3.312; BDRate -3.154
derflr: Avg -1.927; BDRate -1.176
midres: Avg -2.149; BDRate -2.001
hdres : Avg -0.567; BDRate -0.588

Change-Id: I4c06ff51cc20194bffbd4d2346e57ba3dcf6b62c
2016-05-24 13:55:57 -07:00
Debargha Mukherjee
fa5022978d Merge "Wedge refactoring to handle signs better" into nextgenv2 2016-05-20 23:19:39 +00:00
Debargha Mukherjee
e5de2ad632 Wedge refactoring to handle signs better
Mostly refactoring. Handles signs better though results are
more or less neutral.

Change-Id: If499537c8f8da4f34d104ebfda072eb4c85fb12f
2016-05-20 14:12:52 -07:00
Jingning Han
9161464f6c Account sub8x8 block reference filter type for prob context
If a reference block is coded with sub8x8 block size, and if it
has sub-pixel level motion vectors, its prediction filter type
should be used as context information.

The coding performance gains of dual filter type coding scheme are
lowres  0.57%
hdres   0.88%

Change-Id: I68b98f2518d02f11c29d0256aeb45b2580fe5cac
2016-05-18 12:35:31 -07:00
Jingning Han
4677e1a718 Unify the per directional filter type system for compound modes
For the current stage, we assume a single prediction filter type
per direction in the settings of compound inter prediction modes.

Change-Id: I12a1afdd364b93fcee870bd11ad01fc40ab48cff
2016-05-16 14:41:08 -07:00
Geza Lore
9ab9438fbb Break tile row dependencies.
When not using ext-tile, there were still dependencies between tile
rows due to various tools (eg intra predictors) relying on the above
row or above mode info, which can be in the above tile. This is now
broken (the same way as it was when ext-tile is enabled) by fixing
the appropriate predicates.

Change-Id: I107dd0d8481775a792f14e05cfbbd761f16cdc1e
2016-05-10 13:09:47 +01:00
Alex Converse
8c9da4e943 Fix ans+ref_mv build.
Use vp10_read/write instead of vpx_read/write.

Change-Id: I2b7f17e9cdbea14ff48f4bd9776dd3e6aff17a2b
2016-05-09 15:35:58 -07:00
Yaowu Xu
1a6ec3c756 Merge "Change initializations to be compatible with MSVC" into nextgenv2 2016-05-07 20:33:12 +00:00
Jingning Han
bd33326372 Dual prediction filter type for motion compensated reference
Make the bit-stream level support per direction filter type coding
for motion compensated reference.

Change-Id: I61a2360b301075f6734cfd9711b7ae68f214174d
2016-05-07 03:03:04 +00:00
Yaowu Xu
33993798e1 Change initializations to be compatible with MSVC
Change-Id: If5473dadc40d3caea61953fbd112a01939dc1183
2016-05-06 14:20:15 -07:00
Alex Converse
130cccba8d Rename pick_filter_intra.
The word 'pick' is usually used in functions that make decisions where
the bitstream allows multiple legal choices, and not to limit the
bitstream format itself.

Change-Id: Ia60709c29e004475e1aa8861aefded27ebaf4712
2016-05-05 17:06:54 -07:00
Geza Lore
1502d9e44a Fix mismatch with ext-interp.
The encoder signals the interp filter type in the frame header if all
blocks use the same filter (see bitstream.c:fix_interp_filter). This
decision is made based on the counts, but with ext-interp, the counts
are actually only incremented for blocks that fail vp10_is_interp_needed
(see for example encodeframe.c:update_state), otherwise a default value
is used (EIGHTTAP_REGULAR). The decoder however first checks if the
interp filter is signaled at the frame level, and uses that filter type
for all blocks, even if the default value should have been used.

This patch makes the decoder first check with vp10_is_interp_needed
to see if the default value should be used and then checks the frame
level signaling, which reconciles the difference between encoder and
decoder.

Change-Id: I87857ade42dea06b0d5ec2a029e9219268334dbb
2016-05-05 18:24:21 +01:00
Debargha Mukherjee
3407785536 Refactoring and uv fix for wedge
lowres: -1.72%

Change-Id: I4c883097caac72fab8e01945454579891617145e
2016-05-03 08:02:08 -07:00
Debargha Mukherjee
88fe7871be Refactor wedge generation
Change-Id: I2ec4f562e28a4673477e20186f9d6167b24b76b8
2016-04-28 17:51:21 -07:00
Debargha Mukherjee
7ff7943455 Brings back near-near compound mode into ext-inter
lowres: improves by 0.1%

Change-Id: I245019916bf47c6e24bc8c3953b86715ab0193c9
2016-04-28 11:34:13 -07:00
Jingning Han
ec2ffda599 Handle zero motion vector residual
This commit handles the zero motion vector residuals for single
and compound reference modes, respectively. It improves the coding
performance by 0.13% with no additional encoding complexity.

Change-Id: I16075a836025bd2746da2ff4698fb9261e4b08c1
2016-04-18 18:14:01 -07:00
Yue Chen
16a99e967c Merge "Optimization for EXT_INTER + OBMC combination" into nextgenv2 2016-04-17 18:54:33 +00:00
Yue Chen
321794c4d5 Optimization for EXT_INTER + OBMC combination
In the rd loop, check the perf of obmc, whose mv is copied from regular
inter predictor, when wedge interinter is better than regular inter
(previously it will force allow_obmc = 0). The condition of the early
termination before this step is relaxed to avoid skipping too many obmc
predictions. The rates of the overhead are properly calculated for these tools.

The logic of the bitstream syntax:
(a single ref) the interintra flag is sent first, only if it is 0, we
send the obmc flag;
(compound refs) the obmc flag is sent first, only if it is 0, we send
the wedge interinter flag

Coding gain
lowres: 0.428% (2.287%->2.715%)

Change-Id: I5f3a34640b398e313cbf84235c9fe2073eb2173f
2016-04-15 17:03:20 -07:00
Jingning Han
525995a3d9 Apply motion vector precision check to candidate mv
This avoids repeatedly checking the candidate motion vector
precision level at the decoder end. The compression performance
varies at 0.01% level.

Change-Id: I4a88e95decd900d0cac9a0c2e70ba43ef7ecac38
2016-04-14 09:44:41 -07:00
Jingning Han
885a81f468 Merge "Fix a few mis-use cases of MAX_MV_REF_CANDIDATES" into nextgenv2 2016-04-13 23:44:25 +00:00
Jingning Han
f33a0a8215 Fix a few mis-use cases of MAX_MV_REF_CANDIDATES
Fix several use cases where MAX_MV_REF_CANDIDATES is mixed up with
is_compound flag to avoid potential coding interruption.

Change-Id: Ifdee1ef8a81ef6d1c155315c6c6a3074aa7a8b5e
2016-04-13 15:16:55 -07:00
Alex Converse
e7224b7866 Convert some vpx boolcoder calls back to vp10 generic calls.
Change-Id: I362f753ff42d4c4fb94df2419cdaad423d7a4229
2016-04-12 11:00:52 -07:00
Geza Lore
454989ff32 Make superblock size variable at the frame level.
The uncompressed frame header contains a bit to signal whether the
frame is encoded using 64x64 or 128x128 superblocks. This can vary
between any 2 frames.

vpxenc gained the --sb-size={64,128,dynamic} option, which allows the
configuration of the superblock size used (default is dynamic). 64/128
will force the encoder to always use the specified superblock size.
Dynamic would enable the encoder to choose the sb size for each
frame, but this is not implemented yet (dynamic does the same as 128
for now).

Constraints on tile sizes depend on the superblock size, the following
is a summary of the current bitstream syntax and semantics:

If both --enable-ext-tile is OFF and --enable-ext-partition is OFF:
     The tile coding in this case is the same as VP9. In particular,
     tiles have a minimum width of 256 pixels and a maximum width of
     4096 pixels. The tile width must be multiples of 64 pixels
     (except for the rightmost tile column). There can be a maximum
     of 64 tile columns and 4 tile rows.

If --enable-ext-tile is OFF and --enable-ext-partition is ON:
     Same constraints as above, except that tile width must be
     multiples of 128 pixels (except for the rightmost tile column).

There is no change in the bitstream syntax used for coding the tile
configuration if --enable-ext-tile is OFF.

If --enable-ext-tile is ON and --enable-ext-partition is ON:
     This is the new large scale tile coding configuration. The
     minimum/maximum tile width and height are 64/4096 pixels. Tile
     width and height must be multiples of 64 pixels. The uncompressed
     header contains two 6 bit fields that hold the tile width/heigh
     in units of 64 pixels. The maximum number of tile rows/columns
     is only limited by the maximum frame size of 65536x65536 pixels
     that can be coded in the bitstream. This yields a maximum of
     1024x1024 tile rows and columns (of 64x64 tiles in a 65536x65536
     frame).

If both --enable-ext-tile is ON and --enable-ext-partition is ON:
     Same applies as above, except that in the bitstream the 2 fields
     containing the tile width/height are in units of the superblock
     size, and the superblock size itself is also coded in the bitstream.
     If the uncompressed header signals the use of 64x64 superblocks,
     then the tile width/height fields are 6 bits wide and are in units
     of 64 pixels. If the uncompressed header signals the use of 128x128
     superblocks, then the tile width/height fields are 5 bits wide and
     are in units of 128 pixels.

The above is a summary of the bitstream. The user interface to vpxenc
(and the equivalent encoder API) behaves a follows:

If --enable-ext-tile is OFF:
     No change in the user interface. --tile-columns and --tile-rows
     specify the base 2 logarithm of the desired number of tile columns
     and tile rows. The actual number of tile rows and tile columns,
     and the particular tile width and tile height are computed by the
     codec ensuring all of the above constraints are respected.

If --enable-ext-tile is ON, but --enable-ext-partition is OFF:
     No change in the user interface. --tile-columns and --tile-rows
     specify the WIDTH and HEIGHT of the tiles in unit of 64 pixels.
     The valid values are in the range [1, 64] (which corresponds to
     [64, 4096] pixels in increments of 64.

If both --enable-ext-tile is ON and --enable-ext-partition is ON:
     If --sb-size=64 (default):
         The user interface is the same as in the previous point.
         --tile-columns and --tile-rows specify tile WIDTH and HEIGHT,
         in units of 64 pixels, in the range [1, 64] (which corresponds
         to [64, 4096] pixels in increments of 64).
     If --sb-size=128 or --sb-size=dynamic:
         --tile-columns and --tile-rows specify tile WIDTH and HEIGHT,
         in units of 128 pixels in the range [1, 32] (which corresponds
         to [128, 4096] pixels in increments of 128).

Change-Id: Idc9beee1ad12ff1634e83671985d14c680f9179a
2016-04-07 10:34:25 +01:00
Debargha Mukherjee
0fc82ea1cf Refactoring and cosmetic changes to ext-inter expt
Change-Id: Icd457480744b7734b3c412c9fed43be738373334
2016-04-05 15:16:18 -07:00
Alex Converse
fb9186d68d ANS experiment: Use ANS everywhere.
Use ANS for all entropy coded data in VP10 including the compressed header and
modes and motion vectors. ANS tokens continue to be used for DCT tokens.

Change-Id: Idf709a747150601e4d95d81ecfb3dc7253d349df
2016-04-01 11:32:31 -07:00
Debargha Mukherjee
2a6389bb8b Merge "Fix interpolation values and decouple interintra" into nextgenv2 2016-03-31 21:47:10 +00:00
Debargha Mukherjee
2be211e971 Fix interpolation values and decouple interintra
Decouples interintra modes and probability models from regular
intra modes, to enable creating/optimizing new interintra modes.
Also, fixes interpolation values for 128x128 interintra and obmc.

Change-Id: I5c2016db49b8f029164e5fe84c6274d4e02ff90e
2016-03-31 12:12:51 -07:00
Geza Lore
511da8cbe5 Rename MI_BLOCK_SIZE and MI_MASK macros.
Rename MI_BLOCK_SIZE.* -> MAX_MIB_SIZE.* (MIB is for MI Block).
Rename MI_MASK.* -> MAX_MIB_MASK.*

There are no functional changes.

This is in preparation for coding the superblock size at the frame
level, which will require some of these constants to become variables.
The new names better reflect future semantics, and hence make the code
clearer.

Change-Id: Iee08d97554cf4cc16a5dc166a3ffd1ab91529992
2016-03-31 09:57:41 +01:00
Jingning Han
0586460938 Rework the predicted motion vector for sub8x8 block
This commit makes the sub8x8 block to use its nearest neighbor's
motion vector as predicted motion vector for NEWMV mode. It improves
the coding performance by 0.12%.

Change-Id: I99e56715b327573ce7e8a26e3515a4984dadfd98
2016-03-28 14:58:17 -07:00
Jingning Han
1fcb5fc755 Refactor motion vector residual coding process
This commit separates the predicted motion vector from the nearestmv
motion vector in the coding process for both regular and sub8x8
block sizes.

Change-Id: I703490513b0194e6669ebf719352db015facb3e1
2016-03-23 12:10:38 -07:00
Jingning Han
66df6e7c7f Refactor sub8x8 reference motion vector search function
Rework the interface to allow codec store the reference motion
vector list information for coding process.

Change-Id: I47e26587f6c0808655e4626f316ec7614a7ad8ed
2016-03-21 10:02:08 -07:00
Jingning Han
5c9d315572 Rework the DRL syntax entropy coding system
This commit re-designs the probability model for the syntax elements
of the dynamic motion vector referencing system.

Change-Id: Icfb8203c7e8f64e10e99f5890e25e6f6b15fe5d1
2016-03-21 09:52:33 -07:00
Jingning Han
4914ae4622 Merge "Enable dynamic motion vector referencing for newmv mode" into nextgenv2 2016-03-19 00:40:04 +00:00
Jingning Han
93748c3e4f Enable dynamic motion vector referencing for newmv mode
This commit enables the dynamic motion vector predictor for NEWMV
mode. It allows the codec to select the best motion vector predictor
in a rate-distortion optimization framework for motion vector
residual coding. The compression performance is improved:
lowres  0.14%
midres  0.27%
hdres   0.24%

Change-Id: I6a601c74eb6cb0b71a613336d40363359f2edecd
2016-03-18 09:51:37 -07:00
Geza Lore
efe7d4e5a2 Refactor mbmi->inter_tx_size to 2D array.
This is in preparation of increasing the superblock size.

Change-Id: I9197e397399fbe8aec1178a45ea0337dd90412d7
2016-03-18 15:30:09 +00:00
hui su
78b0bd0a0d Complete (mostly) migration of palette mode
Coding gain on screen_content is 12.2% (was 6.6%).

Some features such as frame-level color buffer, adaptive
entropy coding, are coming in future patches.

Change-Id: I2658cf5ec0cbb02cff685475759f3b68c9807697
2016-03-11 09:56:21 -08:00
Debargha Mukherjee
ce4b35d510 Merge "Adds compound wedge prediction modes" into nextgenv2 2016-03-10 17:44:45 +00:00
Debargha Mukherjee
f34deab243 Adds compound wedge prediction modes
Incorporates wedge compound prediction modes.

Change-Id: Ie73b54b629105b9dcc5f3763be87f35b09ad2ec7
2016-03-10 07:19:54 -08:00
hui su
954e560f9e Refactor entropy coding of transform size
No performance change.

Change-Id: If35125fed909d89235b303514f77a33183bb36b3
2016-03-08 16:46:00 -08:00
Debargha Mukherjee
48589e8d07 Merge "Some refactoring and cleanups of interp filter" into nextgenv2 2016-02-29 15:55:48 +00:00
Debargha Mukherjee
bab2912b5e Some refactoring and cleanups of interp filter
Includes various cosmetic changes and refactoring including
naming the sharp filters differently (since they are no longer
8-tap).

Change-Id: Ida5a19ca0daa9f6a64a6734394c685b2a4a2564a
2016-02-26 15:42:49 -08:00
Geza Lore
7ded038af5 Port interintra experiment from nextgen.
The interintra experiment, which combines an inter prediction and an
inter prediction have been ported from the nextgen branch. The
experiment is merged into ext_inter, so there is no separate configure
option to enable it.

Change-Id: I0cc20cefd29e9b77ab7bbbb709abc11512320325
2016-02-26 13:01:51 -08:00
Jingning Han
df59bb8986 Vectorize motion vector probability models
This commit converts the scalar motion vector probability model
into vector format for later precise estimate.

Change-Id: I7008d047ecc1b9577aa8442b4db2df312be869dc
2016-02-19 16:20:41 -08:00
Jingning Han
876c8b03e6 Store predicted motion vectors
Change-Id: I51307a217eeba14dbdaa2522be474530316a4faa
2016-02-19 14:25:34 -08:00