Commit Graph

3428 Commits

Author SHA1 Message Date
John Koleszar
6becad426c detokenize: use SEG_LVL_EOB feature consistently
Update decode_coefs() to break when c >= eob, since it's possible that
c starts the loop from 1 and eob is 0. The loop won't terminate in that
case.

Add new get_eob() function to consistently clamp the eob based on the
segment level EOB and the block size. It's possible to code a segment
level EOB that's greater than the block size, and that leads to an
out of bounds access.

Change-Id: I859563b30414615cf1b30dcc2aef8a1de358c42d
2012-11-15 11:44:29 +00:00
pascal massimino
5a955973d9 Merge changes I63348ae3,I658ea409 into experimental
* changes:
  Segment mode coding bug.
  Silenced a few warnings.
2012-11-15 00:24:57 -08:00
Ronald S. Bultje
120690989b Merge "fix costing bug in pick_uv_sb_mode." into experimental 2012-11-14 17:05:46 -08:00
Ronald S. Bultje
d7290d4974 Merge "Merge a few mostly-duplicate code fragments in SB/MB encoding." into experimental 2012-11-14 17:05:40 -08:00
Ronald S. Bultje
a77df0c473 Merge "Prevent overflow in variance32x32." into experimental 2012-11-14 15:43:19 -08:00
Ronald S. Bultje
a653c9d286 fix costing bug in pick_uv_sb_mode.
Change-Id: Ia24e0fddcca9125f8e41e95dbb22444dc51767c7
2012-11-14 15:19:45 -08:00
Ronald S. Bultje
fa1b356e4e Merge a few mostly-duplicate code fragments in SB/MB encoding.
Change-Id: I8e12fbab7ec4732b6400ae3a6964749d818c90c9
2012-11-14 15:19:45 -08:00
Ronald S. Bultje
a099370370 Prevent overflow in variance32x32.
Change-Id: I478878c78ef8a770186622d987d318176827ef5f
2012-11-14 15:18:21 -08:00
John Koleszar
16e2686682 Merge "SEG_LVL_MODE: don't code ref_frame if it's implicit" into experimental 2012-11-14 09:39:25 -08:00
Ronald S. Bultje
127836d11f Merge "Don't use hybrid transform (ADST) for superblocks." into experimental 2012-11-14 09:18:34 -08:00
Ronald S. Bultje
1e3dd49fe3 Don't use hybrid transform (ADST) for superblocks.
This is in line with other cases where we disable ADST if prediction
size and transform size don't match. Before this patch, the RD loop
will use ADST for superblocks, but frame encoding/decoding won't.

Change-Id: I700368c632eb72b5e089c22ef25649d99d7697d0
2012-11-14 08:58:24 -08:00
Paul Wilkins
b527c4dbb7 Segment mode coding bug.
There are now more than 16 possible modes so 5
bits required for segment mode feature.

Note that it is likely that the mode feature and how it is
coded will change but for now the 4 bits was a bug.

Change-Id: I63348ae3a9cc31566a656c2dc78f09f5e1a9dcc9
2012-11-14 14:38:03 +00:00
Paul Wilkins
19a1ba1e91 Silenced a few warnings.
Silenced a few VS compiler warnings.

Change-Id: I658ea409c36c05cd11042675e2e42ccde0ef2420
2012-11-14 14:27:37 +00:00
John Koleszar
854e41f057 Don't write recon.yuv by default
CONFIG_DEBUG was turning on some code to dump the reconstructed frame
to a buffer from within the decoder. Move this code to a more specific
debugging define.

Change-Id: I3ca9ea634bdbd186f2470bd644d3695ee0ab3037
2012-11-13 15:22:35 -08:00
John Koleszar
6d482706ef SEG_LVL_MODE: don't code ref_frame if it's implicit
If the SEG_LVL_MODE is an intra mode, then the reference frame must be
INTRA_FRAME.

Change-Id: I2cdeeac3780c077c74b39ce89a528bc280674231
2012-11-13 15:22:09 -08:00
Yaowu Xu
3fa1348d5f fix a few typos
Change-Id: I7b6f27826052eb706fc6080d4e3a940dff7d3a58
2012-11-13 14:45:53 -08:00
Ronald S. Bultje
1761a6b55a Merge "Use full 32-pixel edge for superblock bestrefmv motion vector ordering." into experimental 2012-11-13 14:12:58 -08:00
Ronald S. Bultje
b147c64c16 Merge "Fix edge MV handling in SBs." into experimental 2012-11-13 14:12:48 -08:00
Deb Mukherjee
7de64f35d3 A fix in MV_REF experiment
This fix ensures that the forward prob update is not turned off for
motion vectors.

Change-Id: I0b63c9401155926763c6294df6cca68b32bac340
2012-11-13 08:27:04 -08:00
Yunqing Wang
e60478d46d Optimize 8x8 dequant and idct
Similar to 16x16 dequant and idct, based on the value of eobs, the
8x8 dequant and idct calculation was simplified to improve decorder
performance.

Combined vp9_dequant_idct_add_8x8 and vp9_dequant_dc_idct_add_8x8
to eliminate duplicate code.

Change-Id: Ia58e50ab27f7012b7379c495837c9c0b5ba9cf7f
2012-11-12 17:41:53 -08:00
Ronald S. Bultje
c79ae1713c Use full 32-pixel edge for superblock bestrefmv motion vector ordering.
Change-Id: I417e39867c020a17d85370972446a8ce2bbe9a6d
2012-11-12 17:06:56 -08:00
Ronald S. Bultje
722972454c Fix edge MV handling in SBs.
Change-Id: Ia1eddb108ec463835e9de8769572d698e21bca49
2012-11-12 17:06:52 -08:00
Paul Wilkins
5d65614fdd Merge "New inter mode context" into experimental 2012-11-12 09:24:14 -08:00
Ronald S. Bultje
3ec56639bc Merge "Fix frame number on which the mismatch is reported in vpxenc." into experimental 2012-11-12 08:35:14 -08:00
Paul Wilkins
2669f42b0d New inter mode context
This change is a fix / extension of the newbestrefmv
experiment. As such it is presented without IFDEF.

The change creates a new context for coding inter modes
in vp9_find_mv_refs(). This replaces the context that
was previously calculated in vp9_find_near_mvs().
The new context is unoptimized and not necessarily
any better at this stage (results pending), but eliminates
the need for a legacy call to vp9_find_near_mvs().
Based on numbers from Scott, this could help decode
speed by several %.

In a later patch I will add support for forward update of
context (assuming this helps) and refine the context as
necessary.

Change-Id: I1cd991b82c8df86cc02237a34185e6d67510698a
2012-11-12 15:50:02 +00:00
Ronald S. Bultje
3a08b033b0 Merge "Fix data type for eobs[] array in SB 4x4 IDCT code." into experimental 2012-11-12 07:40:54 -08:00
Ronald S. Bultje
11fec1863d Merge "Remove 'thismb' data pointer when superblock experiment is on." into experimental 2012-11-12 07:22:22 -08:00
Paul Wilkins
6fb8953c19 Restrict ref mv search range.
Experiment to test speed trade off of reducing the
extent of the ref mv search.

Reducing the maximum number of tested candidates to 9 had
minimal net effect on quality in any of the tests sets.

Reduction to 7 has a small negative impact (worst was STD-HD
at about -0.2%).

This change is in response to the apparently high number of
decode cycles reported in regard to mv-ref selection.

Change-Id: I0e92e92e324337689358495a1ec9ccdeb23dc774
2012-11-12 11:31:12 +00:00
Ronald S. Bultje
195172cf87 Fix frame number on which the mismatch is reported in vpxenc.
Change-Id: I962be949845f881df551d101708e6ba9d8a3d6a2
2012-11-10 16:22:49 -08:00
Ronald S. Bultje
dd9d4f9e1a Fix data type for eobs[] array in SB 4x4 IDCT code.
This fixes encoder/decoder mismatches with the superblock experiment
turned on whenever a superblock is encoded using the 4x4 transform.

Change-Id: Iefec7055e8d25f8efdbba66c4261bbd322d335a3
2012-11-10 12:08:27 -08:00
Ronald S. Bultje
73987d140a Remove 'thismb' data pointer when superblock experiment is on.
This should prevent inconsistent results between identical encodes with
the superblock experiment turned on.

Change-Id: I41a005fae53f2eb59736cc70041185fb7d63cfca
2012-11-10 08:39:51 -08:00
Deb Mukherjee
d01357bbad New b-intra mode where direction is contextual
Preliminary patch on a new 4x4 intra mode B_CONTEXT_PRED where the
dominant direction from the context is used to encode. Various decoder
changes are needed to support decoding of B_CONTEXT_PRED in conjunction
with hybrid transforms since the scan order and tokenization depends on
the actual direction of prediction obtained from the context. Currently
the traditional directional modes are used in conjunction with the
B_CONTEXT_PRED, which also seems to provide the best results.

The gains are small - in the 0.1% range.

Change-Id: I5a7ea80b5218f42a9c0dfb42d3f79a68c7f0cdc2
2012-11-10 07:12:30 -08:00
Deb Mukherjee
3f7182cb0d Build fix in decoder/decodframe.c
Missing eobs agrument in vp9_dequant_idct_add_16x16_c

Change-Id: I826b1afa0a4ee6398f7373325aa0c75e6a866937
2012-11-09 12:48:35 -08:00
John Koleszar
3a0cfb3617 Merge "Packing Altref along with succeeding frame and length encoding frames" into experimental 2012-11-09 12:31:37 -08:00
Vignesh Venkatasubramanian
bc9670eee0 Packing Altref along with succeeding frame and length encoding frames
The altref frame is packed along with the next P frame. So that
outside of the codec there are now only two types of frames P and I.
Also, now it is one frame in and one frame out with respect to the
codec. Apart from that, all the frames are length encoded with the
length of each frame appended to the frame itself. There are
two categories of frames and each of them will look as follows:

  - Packed frames (an altref along with the succeeding p frame)
    - altref_frame_data | altref_lenngth | frame_data | length
  - Unpacked frames (all frames other than the above)
    - frame_data | length

Change-Id: If1eabf5c473f7d46b3f2d026bd30c803588c5330
2012-11-09 12:04:53 -08:00
Ronald S. Bultje
9f53661e8b Merge "Fix another crash in vpxenc with --pass=1 and --test-decode." into experimental 2012-11-09 09:14:06 -08:00
Ronald S. Bultje
88d703cd4d Fix another crash in vpxenc with --pass=1 and --test-decode.
Change-Id: Id23917ce2555519d2e9172dd6ce3a283f2235d26
2012-11-09 09:07:50 -08:00
Yunqing Wang
71b1885403 Merge "Optimize 16x16 dequant and idct" into experimental 2012-11-09 08:30:53 -08:00
Yaowu Xu
7ab951d8cc Merge changes I18407622,I34057b87 into experimental
* changes:
  fix a crash in 2-pass encoding with test-decode on
  enable build on windows with msvc
2012-11-08 16:30:53 -08:00
Yaowu Xu
606c6479c5 fix a crash in 2-pass encoding with test-decode on
only call decoding when encoder produce compressed data

Change-Id: I1840762246afaaad76db6c6d29032ad15aa38afd
2012-11-08 16:22:54 -08:00
Yaowu Xu
7798aef50b enable build on windows with msvc
Change-Id: I34057b87d9713dc819b8c69990dd1408b8c79342
2012-11-08 15:56:27 -08:00
Jim Bankoski
a186eb7f1b Merge "remove macros obfuscating mv costing" into experimental 2012-11-08 15:51:21 -08:00
Jim Bankoski
c72be96b0a remove macros obfuscating mv costing
cleanup

Change-Id: I565eee40d900e0441ad211b65ac829fc5b93d94a
2012-11-08 15:44:39 -08:00
Ronald S. Bultje
1d4fbeb32a Implement tx_select for superblock encoding.
Also split superblock handling code out of decode_macroblock() into
a new function decode_superblock(), for easier readability.

Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains
by allowing to select mb_skip_coeff for a subset of the complete SB
or something along those lines, because although this change allows
coding smaller transforms for bigger predictors, it increases the
overhead of coding EOBs to skip the parts where the residual is
near-zero, and thus the overall gain is not as high as we'd expect.

Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 11:03:00 -08:00
Yunqing Wang
6c17c9fae0 Optimize 16x16 dequant and idct
As suggested by Yaowu, simplified 16x16 dequant and idct. In decoder,
after detoken step, we know the number of non-zero dct coefficients
(eobs) in a macroblock. Idct calculation can be skipped or simplified
based on eobs, which improves the decoder performance.

Change-Id: I9ffa1cb134bcb5a7d64fcf90c81871a96d1b4018
2012-11-07 20:04:09 -08:00
John Koleszar
8959c8b11d Merge with upstream experimental changes (2)
Include upstream changes (variance fixes) into the merged code base.

Change-Id: I4182654c1411c1b15cd23235d3822702613abce1
2012-11-07 14:32:26 -08:00
James Zern
5338d983d6 Merge "Fix variance (signed integer) overflow" into experimental 2012-11-07 12:49:36 -08:00
John Koleszar
2c08c28191 Merge with upstream experimental changes
Include upstream changes (unit test fixes, in particular) into the
merged code base.

Change-Id: I096f8a9d09e2532fbec0c95d7a995ab22fa54b29
2012-11-07 11:46:23 -08:00
John Koleszar
7b8dfcb5a2 Rough merge of master into experimental
Creates a merge between the master and experimental branches. Fixes a
number of conflicts in the build system to allow *either* VP8 or VP9
to be built. Specifically either:

  $ configure --disable-vp9 $ configure --disable-vp8
  --disable-unit-tests

VP9 still exports its symbols and files as VP8, so that will be
resolved in the next commit.

Unit tests are broken in VP9, but this isn't a new issue. They are
fixed upstream on origin/experimental as of this writing, but rebasing
this merge proved difficult, so will tackle that in a second merge
commit.

Change-Id: I2b7d852c18efd58d1ebc621b8041fe0260442c21
2012-11-07 11:30:16 -08:00
Yaowu Xu
0cedaa3631 merge full pixel refmv experiment
Change-Id: Ib39ad47a7d188f3b45416937b7eeb28c3e79b74c
2012-11-07 10:52:45 -08:00