748 Commits

Author SHA1 Message Date
Alex Converse
ec6fb649da Partition the ans experiment into 'ans' and 'rans'
The (new) ans experiment replaces the bool coder with uABS bools. The
'rans' experiment adds multisymbol coding.

This matches the setup in aom/master.

Change-Id: Ida8372ccabf1e1e9afc45fe66362cda35a491222
2016-10-19 12:03:15 -07:00
Urvang Joshi
66b1fcc924 Merge changes I3922dea2,I3bab2848,I21f7478a,Ida5de713,Ib9f0eefe, ... into nextgenv2
* changes:
  Fix warnings reported by -Wshadow: Part4: main directory
  Fix warnings reported by -Wshadow: Part3: test/ directory
  Fix warnings reported by -Wshadow: Part2b: more from av1 directory
  Fix warnings reported by -Wshadow: Part2: av1 directory
  Fix warnings reported by -Wshadow: Part1b: scan_order struct and variable
  Fix warnings reported by -Wshadow: Part1: aom_dsp directory
  Move STAT_TYPE enum to source file.
  Code cleanup: mainly rd_pick_partition and methods called from there.
2016-10-19 18:25:52 +00:00
Hui Su
3e908b7f44 Merge "Temporary fix for 4X8 block intra prediction." into nextgenv2 2016-10-19 16:55:20 +00:00
Hui Su
e22a480225 Merge "Fix format in set_offsets()" into nextgenv2 2016-10-19 16:54:30 +00:00
Angie Chiang
d83fc3b8d9 Merge "Add av1_fdct64_new and av1_idct64_new" into nextgenv2 2016-10-19 16:34:24 +00:00
Urvang Joshi
4145bf05ae Fix warnings reported by -Wshadow: Part4: main directory
Now that all warnings are taken care of, add warning flag -Wshadow to
configure.

Note: Enabling this flag for C++ generates some useless warnings about
some function parameters shadowing class member function names. So, only
enabling this warning for C code.

Cherry-picked from aomedia/master: b96cbc4

Change-Id: I3922dea2e6976b16519c4aa4d1bd395c198134f1
2016-10-19 07:56:53 -07:00
Peter de Rivaz
74d0ad844e Fix for var_tx context update
The tx_partition_set_contexts function changes tx_size even
for blocks coded with a rectangular transform.
This causes an internal rd inconsistency when using all of
CONFIG_VAR_TX, CONFIG_RECT_TX, CONFIG_EXT_TX.

Change-Id: Ia45d4a8893b0961534219bb96d9652719038c7a1
2016-10-19 11:43:11 +01:00
Yaowu Xu
caf2023ae1 Reorder includes
Change-Id: I97487bf353471bf9d245cd620780adfb1d3fc2b1
2016-10-19 04:34:49 +00:00
Michael Bebenita
6048d05225 Bit accounting.
This patch adds bit account infrastructure to the bit reader API.
When configured with --enable-accounting, every bit reader API
function records the number of bits necessary to decoding a symbol.
Accounting symbol entries are collected in global accounting data
structure, that can be used to understand exactly where bits are
spent (http://aomanalyzer.org). The data structure is cleared and
reused each frame to reduce memory usage. When configured without
--enable-accounting, bit accounting does not incur any runtime
overhead.

All aom_read_xxx functions now have an additional string parameter
that specifies the symbol name. By default, the ACCT_STR macro is
used (which expands to __func__). For more precise accounting,
these should be replaced with more descriptive names.

Change-Id: Ia2e1343cb842c9391b12b77272587dfbe307a56d
2016-10-19 04:34:29 +00:00
Debargha Mukherjee
4bacfcffd0 Merge "Fix ransac random generator seeding" into nextgenv2 2016-10-19 01:39:08 +00:00
Yaowu Xu
321556a557 Merge "Update segment tree_cdf per frame." into nextgenv2 2016-10-19 01:09:52 +00:00
Sarah Parker
cd2750048f Merge "Add clamping to parameter search" into nextgenv2 2016-10-19 00:44:28 +00:00
Sarah Parker
5572486ed7 Merge "Adjust gm costing so GLOBAL_ZERO is treated as regular zeromv" into nextgenv2 2016-10-19 00:44:12 +00:00
Jingning Han
97d854831f Fix format in set_offsets()
Change-Id: I371297e6ee000e6dc01ba1544763cbed429b0e5a
2016-10-18 17:42:09 -07:00
Brennan Shacklett
7523a7ecd6 Temporary fix for 4X8 block intra prediction.
Currently the RD loop traverses 4X8 blocks in inverted N order while
the bitstream stores blocks smaller than 8x8 in Z order. This causes a
discrepancy where the RD loop reads uninitialized data while
performing intra prediction.  As a temporary fix simply disable the
use of the extended right edge for 4X8 blocks, until the bitstream can
be changed to match the logical structure of the blocks.

Change-Id: I44a9e4fc1a15cd551a7b38c3c1227bc5dac77e9a
2016-10-18 17:24:53 -07:00
Urvang Joshi
368fbc955d Fix warnings reported by -Wshadow: Part2b: more from av1 directory
From code only part of nextgenv2 (and not aomedia)

Change-Id: I21f7478a59d525dff23747efe5238ded16b743d2
2016-10-18 17:22:44 -07:00
Urvang Joshi
454280dabf Fix warnings reported by -Wshadow: Part2: av1 directory
While we are at it:
- Rename some variables to more meaningful names
- Reuse some common consts from a header instead of redefining them.

Cherry-picked from aomedia/master: 863b0499

Change-Id: Ida5de713156dc0126a27f90fdd36d29a398a3c88
2016-10-18 17:22:34 -07:00
Urvang Joshi
03f6fdcfca Fix warnings reported by -Wshadow: Part1b: scan_order struct and variable
- Change struct name to all caps SCAN_ORDER to be locally consistent.
- Rename struct pointers to 'scan_order' instead of hard to read short
  names 'so' and 'sc'.

Cherry-picked from aomedia/master: 30abc082

Change-Id: Ib9f0eefe28fa97d23d642b77d7dc8e5f8613177d
2016-10-18 17:22:23 -07:00
Urvang Joshi
b5ed35008d Move STAT_TYPE enum to source file.
In the header, all we need is number of stat types, not the names for actual
types.

Removing it avoids names like 'Y', 'U', 'V' and 'ALL' being visible
in all files that include the encoder.h header.

Change-Id: I874a73a3cfe6bcb29aedea102077a52addc49af6
2016-10-18 17:22:00 -07:00
Urvang Joshi
526484482a Code cleanup: mainly rd_pick_partition and methods called from there.
- Const correctness
- Refactoring
- Make variables local when possible etc
- Remove -Wcast-qual to allow explicitly casting away const.

Cherry-picked from aomedia/master: c27fcccc
And then a number of more const correctness changes to make sure other
experiments build OK.

Change-Id: I77c18d99d21218fbdc9b186d7ed3792dc401a0a0
2016-10-18 17:21:27 -07:00
Nathan E. Egge
f627e58e0f Update segment tree_cdf per frame.
Move computing the segmentation_probs.tree_cdf table per symbol to
 computing it only when the probabilities are updated.

Change-Id: I3826418094bbaca4ded87de5ff04d4b27c85e35a
2016-10-18 16:58:48 -07:00
Sarah Parker
081783dc67 Add clamping to parameter search
This fixes mismatches due to overflowing low precision parameters.

Change-Id: If34e39ca7ab0adc9688d46b0e8ed62cbb6fdaff0
2016-10-18 16:43:54 -07:00
Sarah Parker
ae51dd820d Adjust gm costing so GLOBAL_ZERO is treated as regular zeromv
Change-Id: I1b41146ae844c985566f5f9fdaeb5d4a4a5927b6
2016-10-18 16:18:23 -07:00
Sarah Parker
efa6582235 Fix ransac random generator seeding
Ransac's get_rand_indices originally used rand_r seeded with the
same value every time, producing the same random sequence at every
iteration. This causes the global motion parameters to be slightly
less accurate because ransac cannot improve the model fit after
the first attempt.

Change-Id: Idca2f88468ea21d19ba41ab66e5a2744ee33aade
2016-10-18 16:14:46 -07:00
Angie Chiang
792519bdef Add av1_fdct64_new and av1_idct64_new
Change-Id: If497816d7f6ee094d40872a2f988c91e90b78d7b
2016-10-18 16:07:56 -07:00
Guillaume Martres
470efbcf01 Remove rd_variance_adjustment
This function is called after `super_block_yrd` and assumes that the dst
buffer is correct but that is no longer always the case after
daf841b4a10ece1b6831300d79f271d00f9d027b since we don't call
`txfm_rd_in_plane` after the RDO loop in `choose_tx_size_from_rd`.
We could fix this by always saving and restoring the dst buffer but
removing `rd_variance_adjustment` is a better solution:
- Getting the dst buffer always right is tricky as demonstrated by the
  fact that it is wrong now, even if we fix it now we could break it later
  and not notice
- Perceptual weighting is a good idea but `rd_variance_adjustment` is the
  wrong approach as it weights both the rate and the distortion:
  to get meaningful units you should only weight the distortion,
  weighting rate means that we pretend some bits cost less than other
  bits, this is not the case. The distortion weighting approach is
  implemented by Daala in `od_compute_dist` and we plan to experiment
  with this in AV1 too.
- Removing `rd_variance_adjustment` improves coding efficiency on all
  metrics, here are the results for objective-1-fast using the Low
  Latency settings:

      PSNR Y:     -0.14%
     PSNRHVS:     -0.17%
        SSIM:     -0.12%
      MSSSIM:     -0.12%
   CIEDE2000:     -0.07%

Change-Id: I74b26b568ee65f56521646b8f30dd53bcd29fce3
2016-10-18 14:40:15 -07:00
Zoe Liu
a6a6dd509d A small bug fix in ext-refs on the RD mode selection
Change-Id: I25f14fec8e806cdf98d904488aaf200169def34d
2016-10-18 13:03:12 -07:00
Yushin Cho
40f1d487ad Remove unused PICK_MODE_CONTEXT::is_coded.
Change-Id: Ibc73b4066dcdee45d32355144124762d26a16a28
2016-10-18 12:54:12 -07:00
Urvang Joshi
8a02d76a93 Remove unused array 'last_frame_seg_map_copy'.
This array was allocated and used to save and restore segmentation map,
however the original segmentation map was never modified between the
calls to save and restore.

Change-Id: Iaf0fbfed733c097e84cf44d2aa6b8f35d2fb456b
2016-10-18 12:54:12 -07:00
Yushin Cho
e2b403b979 Remove macroblock::skip_optimize.
This is not used since the commint 00cd5de536fd5545d8fb663b2db81c014e3e6a41,
"Remove skip_recode speed feature".

Change-Id: Ic03da6c0095f6285a3889d5d22e8aaa2e6cbfd79
2016-10-18 11:26:11 -07:00
Hui Su
eafb2e62ac Skip 4x4 transform if maximum possible transform is 32x32
On average no compression performance changes. Encoding speed is
increased by 10~20% on some test clips in the derf set.

Change-Id: I9856caaa260303f6f6259686671bed7d51012277
2016-10-18 11:26:11 -07:00
Jingning Han
3f16725ff2 Take out some early termination speed features
Drop some speed features used in speed 2 and above, during the
algorithm development process. This helps simplify the codebase.

Change-Id: I3b2f5560d90b00d2d8fd57c2cb36f6ddd3f228e4
2016-10-18 11:26:11 -07:00
Yaowu Xu
8f7b1d3db9 Merge "Move a statement to match order in aom/master" into nextgenv2 2016-10-18 17:58:33 +00:00
Yaowu Xu
ee775b13e2 Move a statement to match order in aom/master
Change-Id: Ic11eae36c9c62a20699197847aa3ef9562d4ad7e
2016-10-18 10:00:21 -07:00
Yaowu Xu
85c5566559 Merge "Port aom_reader_tell() support" into nextgenv2 2016-10-18 16:48:57 +00:00
Debargha Mukherjee
d8ff1986d4 Merge "Fix for var_tx entropy context with rect_tx" into nextgenv2 2016-10-18 16:03:37 +00:00
Debargha Mukherjee
3f8b5b903f Merge "Correction to costing rect_tx" into nextgenv2 2016-10-18 16:03:18 +00:00
Michael Bebenita
868fc0b04a Port aom_reader_tell() support
This commit ports the following from aom/master:
4c46278 Add aom_reader_tell() support.
b9c9935 Remove an erroneous declaration.
56c9c3b Fix ANS build.

Change-Id: I59bd910f58c218c649a1de2a7b5fae0397e13cb1
2016-10-18 08:50:05 -07:00
Peter de Rivaz
46fcb05fde Fix for var_tx entropy context with rect_tx
This computation should match the code in encode_block
to increase the accuracy of the rd optimization.

Change-Id: Ibc9d9ab6d88d0c0f3af62e9cc233216aba48a57e
2016-10-18 15:38:01 +01:00
Peter de Rivaz
b85a5a7eac Correction to costing rect_tx
When built with var_tx and ext_tx, select_tx_size_fix_type is used
to compute the cost for using a particular tx_type.
The code indexes the array inter_tx_type_costs at the wrong location
resulting in a zero cost for signalling tx_type for rect_tx blocks.

Change-Id: Iba38be3a0d822109f778f0600b242dfb40359766
2016-10-18 11:55:36 +01:00
Nathan E. Egge
9ac1f7d770 Create aom_cdf_prob type for 16-bit probabilities.
Change-Id: I33899eca44300037816c9f20c965aa8311a1ef52
2016-10-17 20:22:48 -07:00
Yaowu Xu
9ce9e6d533 Merge "Rename aom_write_tree_cdf() to aom_write_symbol()." into nextgenv2 2016-10-18 00:49:41 +00:00
Yaowu Xu
007fd85007 Merge "Bug fix in super_block_uvrd()." into nextgenv2 2016-10-18 00:49:28 +00:00
Yaowu Xu
79644f615e Merge "Update partition_cdf per frame." into nextgenv2 2016-10-18 00:49:05 +00:00
Yaowu Xu
153df29bbf Merge "Update inter_ext_tx_cdf per frame." into nextgenv2 2016-10-18 00:48:53 +00:00
Yaowu Xu
a6fa5436ff Merge "Update intra_ext_tx_cdf per frame." into nextgenv2 2016-10-18 00:48:41 +00:00
Yaowu Xu
f507ba79fe Merge "Update switchable_interp_cdf once per frame." into nextgenv2 2016-10-18 00:48:26 +00:00
Yue Chen
3fcf53e381 Merge "Refactor motion estimation in MOTION_VAR experiment" into nextgenv2 2016-10-18 00:32:00 +00:00
Yue Chen
e9638ccfff Refactor motion estimation in MOTION_VAR experiment
To get ready for pulling AV1 to nextgenv2. Refactoring is done to
make the code structures similar, especially for the motion search
part.

Change-Id: I5d7636394408d97de55394d668540f5627827983
2016-10-17 12:48:10 -07:00
Nathan E. Egge
56eeaa5daf Rename aom_write_tree_cdf() to aom_write_symbol().
Change-Id: I7c088c55f1c461063976d5bd84ff2026c4f3bc69
2016-10-17 11:54:51 -07:00