Commit Graph

103 Commits

Author SHA1 Message Date
Thomas Davies
09ebbfb39f EC_ADAPT: send updates for the correct nodes.
EOB and ZERO token are not currently adapted.

Change-Id: Ie7d657b71fcb157b09e40874fb06a8b7cd95cc70
2016-10-29 17:05:40 -07:00
Alex Converse
aca9feba82 Add ec_multisymbol for common daala_ec and rans code
The new ec_multisymbol experiment supersedes the rans experiment and is
used for multisymbol features that can be backed by either daala_ec or
rans.

This experiment is automatically enabled by ec_adapt and will try to
enable daala_ec or ans (in that order).

Change-Id: Ie75b4002b7a9d7f5f7b4d130c1aacb3dbe97e54f
2016-10-29 17:05:40 -07:00
Thomas
9ac5508f32 Add EC_ADAPT experiment for symbol-adaptive entropy coding.
This experiment performs symbol-by-symbol statistics
adaptation for non-binary symbols. It requires DAALA_EC or
RANS and ANS to be enabled. The adaptation is currently
based on a simple recursive filter and is taken from
Daala. It has an adaptation rate dependent on alphabet size,
taken from Daala. It applies wherever non-binary symbols
are encoded using Cumulative Probability Functions rather
than trees.

Where symbols are adapted, forward updates in the compressed
header are removed.

In the case of RANS coefficient token values are adapted,
with the exception of the zero token which remains a
binary symbol. In the case of DAALA_EC other values
such as inter and intra modes are adapted as CDFs are
provided in those cases.

The experiment is configured with:

./configure --enable-experimental --enable-daala-ec --enable-ec-adapt

or

./configure --enable-experimental --enable-ans --enable-rans \
    --enable-ec-adapt

EC_ADAPT is not currently compatible with tiles.

BDR results on Objective-1-fast give a small loss:

PSNR YCbCr:      0.51%      0.49%      0.48%
PSNRHVS:      0.50%
SSIM:      0.50%
MSSSIM:      0.51%
CIEDE2000:      0.50%

Change-Id: I3888718e42616f3fd87144de7f125228446ac984
2016-10-29 16:57:48 -07:00
Yaowu Xu
d64eaf138e Merge "Tile groups: ensure each tile in a TG has a length." into nextgenv2 2016-10-28 21:26:32 +00:00
Yaowu Xu
edd3f9c418 Merge "Fix update_delta_q_probs compile warning" into nextgenv2 2016-10-28 21:26:23 +00:00
Yaowu Xu
efd5725242 Merge "Encode and decode multiple tile groups" into nextgenv2 2016-10-28 21:26:11 +00:00
Thomas Davies
8fe64a3a23 Tile groups: ensure each tile in a TG has a length.
This ensures TGs can be decoded even if the whole
frame has not been received and the frame length
is not known.

Change-Id: If24837fcc3b5c46554751be792e91100de73e8d6
2016-10-28 13:01:40 -07:00
Jingning Han
be44c5f46f Fix update_delta_q_probs compile warning
Change-Id: Ifb93970ed876ed61259b2f8da739171857c97fda
2016-10-28 13:01:40 -07:00
Thomas Davies
80188d1546 Encode and decode multiple tile groups
This is a manual adaptation of the following commit from aom/master:
ce12003d60a1c8d6c65ed07ba165c34062fcbcbd

The original commit message:

A tile group is a set of tiles in scan order.

Each tile group has a version of uncompressed and compressed headers,
identical apart from tile group parameters.
Encoding probability updates takes account of the number of
headers to control overheads.

The decoder supports arbitrary numbers of tile groups with
arbitrary number of tiles. The number of tiles in a TG is
signalled in the uncompressed header for that TG.

The encoder currently only supports a fixed number
of TGs (3, when error resilient mode is on) of equal size
(except possibly for the last one).

The average BDR performnce with 3 tile groups versus
anchor with error resilient mode and up to 16 tiles is:

NR YCbCr:      3.02%      3.04%      3.05%
PSNRHVS:      3.09%
SSIM:      3.06%
MSSSIM:      3.05%
CIEDE2000:      3.04%

Change-Id: I9b97c5ed733103b9160a3a5d4370de5322c00c0b
2016-10-28 11:52:13 -07:00
David Barker
f8935c9c92 Fix clpf and dering signalling when used with ext-partition-types
Previously, when ext-partition-types and either clpf or dering were
enabled, the signalling for clpf/dering would not be encoded or decoded,
as the code to do so was inside a #if !CONFIG_EXT_PARTITION_TYPES block.
This caused many tests (eg, AV1/EndToEndTestLarge.EndToEndPSNRTest/0)
to fail with encode/decode mismatches.

Change-Id: If1742deb1812877813b2c3e93a048430f9a504ba
2016-10-27 13:19:01 +00:00
Jingning Han
4e1737af64 Add depth to tx_size mapper to bit-stream coding
It serves as a helper function to integrate various transform coding
options.

Change-Id: I64e7d0c88ea10137fa1ff1072d865eb0054c2a25
2016-10-26 15:45:19 +00:00
Jingning Han
c83ef8b946 Merge "Refactor transform size coding" into nextgenv2 2016-10-26 01:12:04 +00:00
Jingning Han
aae72a69c3 Refactor transform size coding
Introduce the transform block partition depth macro definition.

Change-Id: I218dc77a77c8e967da4d270d4ec0d7691b712a5f
2016-10-25 15:42:30 -07:00
Yaowu Xu
dece603fdf Merge "Use constrained tokenset with --enable-daala_ec." into nextgenv2 2016-10-25 19:40:51 +00:00
Yaowu Xu
1f112841d2 Merge "Refactor extrabits packing" into nextgenv2 2016-10-25 17:14:57 +00:00
Yaowu Xu
d8dc1fc522 Merge "Linearize extrabits writing." into nextgenv2 2016-10-25 17:14:44 +00:00
Nathan E. Egge
46e8490498 Use constrained tokenset with --enable-daala_ec.
Change-Id: Ia09edf92bf9f7ecacc65c232ac6e656cde236634
2016-10-25 10:13:22 -07:00
Angie Chiang
6305abe114 Refactor: Add macro LOG_SWITCHABLE_FILTERS
Change-Id: I7593ff2f8949d8bc26ca1c8577faaefb09640b59
2016-10-25 09:22:35 -07:00
Alex Converse
d8fdfaa4da Refactor extrabits packing
- Eliminate the awkward _av1 suffix/infix in local variable names.
- Lift bitdepth selection out of the token loop.

Change-Id: I26d3397464f7808e0481a804033a93ca4f01f5d5
2016-10-25 08:59:08 -07:00
Alex Converse
81fd890773 Linearize extrabits writing.
The decoder is already linear so changing these tables would just create
a mismatch.

Change-Id: Ib888c0dc273e089c38298f569bb35b6e4c32dd60
2016-10-25 08:59:08 -07:00
Urvang Joshi
7bc1fa194d Merge changes I2153c57e,I0e291edd into nextgenv2
* changes:
  Palette: Generate encodings automatically from tree.
  Palette + Ext-Intra: shadowed declaration fix.
2016-10-25 01:06:28 +00:00
Urvang Joshi
0b325978d7 Palette: Generate encodings automatically from tree.
Ran some manual sanity checks:
- Verified that the automatically generated encodings match the
  hand-written encodings before the patch.
- Verified that the encoded bitstream before/after this patch is
  identical.

Change-Id: I2153c57e463cff09c1d03d619b432fb1015199c3
2016-10-24 14:37:25 -07:00
Jingning Han
571189c66d Add MAX_VARTX_DEPTH macro
Change-Id: I85532cf88f91f0f0cb4d9cb4b2dbda8a181297bf
2016-10-24 10:38:43 -07:00
Jingning Han
421af3538d Merge "Limit the transform block partition depth" into nextgenv2 2016-10-24 15:57:28 +00:00
Nathan E. Egge
5f7fd7ab5e Update joint_cdf table once per frame.
Move computing the joint_cdf table per coded mv joint symbol to
 computing it only when the probabilities are updated.

Change-Id: If5d195f70e6fad7b60f69606c8386ad5e69657d2
2016-10-21 22:55:31 +00:00
Nathan E. Egge
6ec4d10d3c Update inter_mode_cdf tables once per frame.
Move computing the inter_mode_cdf tables per coded inter mode symbol to
 computing them only when the probabilities are updated.

Change-Id: I7a7b059ee75723cb6f278ed82a20cf34c27915d8
2016-10-21 22:54:50 +00:00
Jingning Han
94d5bfccdd Limit the transform block partition depth
Limit the recursive transform block partition depth to 2. For a
32x32 transform block unit, one can maximally go down to 8x8 transform
block size.

Change-Id: I2caa92bb2eee64762b7ecca8920259f7c50fb0aa
2016-10-21 15:44:34 -07:00
Nathan E. Egge
380cb1a93c Update uv_mode_cdf tables once per frame.
Move computing the uv_mode_cdf tables per coded intra mode symbol to
 computing them only when the probabilities are updated.

Change-Id: I627b59d30726c913f5d7ba7753cb0446a12655bb
2016-10-21 12:39:04 -07:00
Nathan E. Egge
5710c722af Update y_mode_cdf tables once per frame.
Move computing the y_mode_cdf tables per coded intra mode symbol to
 computing them only when the probabilities are updated.

Change-Id: I8c43d09b8ef5febe2a3ec64bd51d28bd78ea73ed
2016-10-21 12:39:04 -07:00
Nathan E. Egge
3ef926edc2 Update kf_y_mode_cdf tables once per frame.
Move computing the kf_y_mode_cdf tables per coded intra mode symbol to
 computing them only when the probabilities are updated.

Change-Id: I5999447050c2f7d5dbccde80bee05ecd1c5440ab
2016-10-21 12:39:04 -07:00
Nathan E. Egge
72762a2827 Use intra_ext_tx_cdf when coding tx_type.
When building with --enable-daala_ec, the tx_type for intra blocks can be
 coded using the CDFs that are updated once per frame.
This patch converts a tx_type symbol to be coded with aom_write_symbol()
 and aom_read_symbol() that was missed in f3e8e267.

Change-Id: I34f8fef7525f88e156bbcb78dfc48994367610ce
2016-10-21 08:29:08 -07:00
Yaowu Xu
23f0604188 Merge "Fix encoder crash when --enable-daala-ec" into nextgenv2 2016-10-21 14:44:26 +00:00
Yaowu Xu
361f3fe3b0 Merge "Compute all token encodings from symbol trees." into nextgenv2 2016-10-21 03:20:01 +00:00
Yaowu Xu
e1466ad4e4 Fix encoder crash when --enable-daala-ec
Change-Id: I6855e18d92f693a9789eda7c91a3430566469bdd
2016-10-20 17:56:54 -07:00
Nathan E. Egge
3c05679017 Compute all token encodings from symbol trees.
The av1_token encodings must match the contents of the aom_tree_index
 structures so generate all encodings from the symbol trees.

Change-Id: I37be9f12c86a02693ae3c3c1d24b00f2abb29bfb
2016-10-20 15:34:08 -07:00
Urvang Joshi
43e6281f62 Encoder/Decoder mismatch fix: need a separate copy of eob_counts.
The bug was introduced here:
https://chromium-review.googlesource.com/#/c/399975/4/av1/encoder/bitstream.c
In that patch, I had removed 2nd declaration of a variable of the same
name. But it turns out that the two variables actually had a different
type (even though the name was same).

Now, we keep both variables, but rename one of them -- that fixes the
mismatch. While we are at it, made both variables local as well.

The fix can be verified as follows:
../../libvpx/configure --enable-experimental --enable-supertx
--enable-var-tx --enable-entropy --enable-internal-stats && make clean
&& make -j16

aomenc -o soccer_cif_1000_av1_b8.webm ../soccer_cif.y4m --codec=av1
--limit=50 --skip=0 -p 2 --pass=1 --fpf=soccer_cif_av1.fpf --good
--cpu-used=0 --target-bitrate=1000 --lag-in-frames=25 --min-q=0
--max-q=63 --auto-alt-ref=1 --kf-max-dist=150 --kf-min-dist=0
--drop-frame=0 --static-thresh=0 --bias-pct=50 --minsection-pct=0
--maxsection-pct=2000 --arnr-maxframes=7 --arnr-strength=5 --sharpness=0
--undershoot-pct=100 --overshoot-pct=100 --frame-parallel=0
--tile-columns=0 --profile=0 --test-decode=warn

aomenc -o soccer_cif_1000_av1_b8.webm ../soccer_cif.y4m --codec=av1
--limit=50 --skip=0 -p 2 --pass=2 --fpf=soccer_cif_av1.fpf --good
--cpu-used=0 --target-bitrate=1000 --lag-in-frames=25 --min-q=0
--max-q=63 --auto-alt-ref=1 --kf-max-dist=150 --kf-min-dist=0
--drop-frame=0 --static-thresh=0 --bias-pct=50 --minsection-pct=0
--maxsection-pct=2000 --arnr-maxframes=7 --arnr-strength=5 --sharpness=0
--undershoot-pct=100 --overshoot-pct=100 --frame-parallel=0
--tile-columns=0 --profile=0 --test-decode=warn -v --psnr

Change-Id: Ibd72dbe1f620e6de231513220ee4e190606613ae
2016-10-20 14:51:01 -07:00
hui su
5db9743fbb Seperate FILTER_INTRA from EXT_INTRA experiment
Prepare for the av1/nextgenv2 merge.

Coding gain (%):

               lowres     midres
ext-intra       0.69       0.97
filter-intra    0.67       0.83
both            1.05       1.48

Change-Id: Ia24d6fafb3e484c4f92192e0b7eee5e39f4f4ee6
2016-10-19 21:40:49 -07:00
Yaowu Xu
cfc5ac5034 Merge "Partition the ans experiment into 'ans' and 'rans'" into nextgenv2 2016-10-19 22:58:05 +00:00
Thomas Davies
f693610a1a Step size and arithmetic coding for delta quantization.
Example performance: 1.8% bit rate savings using
the AQ test mode aq-mode=4 :
./aomenc --codec=av1 --ivf --tile-columns=1 --tile-rows=1 \
                 --kf-max-dist=1000 --kf-min-dist=1000 --cpu-used=0 \
                 --passes=1 --threads=1 --lag-in-frames=0 \
                 --end-usage=q --limit=600 --cq-level=42 \
                 --aq-mode=4 --error-resilient=1 out.bits FourPeople_1280x720_60.y4m

Change-Id: Iba01cf2732a57f3c27481ac2a3c8fc37bb9e5533
2016-10-19 15:14:27 -07:00
Arild Fuldseth
07441165fe Support for delta-q at superblock level
Change-Id: I4128af44776d1f361bddc1fdffb75ed2224dbfa5
2016-10-19 15:14:27 -07:00
Arild Fuldseth
842e9b030f Always send frame size explicitly
This commit changes to send frame size explicitly when
error_resilient_mode=1. Purpose is to allow parsing of bitstream
after a packet loss.

Change-Id: I7d1c010a465aa18914762cc1a3e61db377304c08
2016-10-19 12:35:12 -07:00
Alex Converse
ec6fb649da Partition the ans experiment into 'ans' and 'rans'
The (new) ans experiment replaces the bool coder with uABS bools. The
'rans' experiment adds multisymbol coding.

This matches the setup in aom/master.

Change-Id: Ida8372ccabf1e1e9afc45fe66362cda35a491222
2016-10-19 12:03:15 -07:00
Urvang Joshi
66b1fcc924 Merge changes I3922dea2,I3bab2848,I21f7478a,Ida5de713,Ib9f0eefe, ... into nextgenv2
* changes:
  Fix warnings reported by -Wshadow: Part4: main directory
  Fix warnings reported by -Wshadow: Part3: test/ directory
  Fix warnings reported by -Wshadow: Part2b: more from av1 directory
  Fix warnings reported by -Wshadow: Part2: av1 directory
  Fix warnings reported by -Wshadow: Part1b: scan_order struct and variable
  Fix warnings reported by -Wshadow: Part1: aom_dsp directory
  Move STAT_TYPE enum to source file.
  Code cleanup: mainly rd_pick_partition and methods called from there.
2016-10-19 18:25:52 +00:00
Urvang Joshi
368fbc955d Fix warnings reported by -Wshadow: Part2b: more from av1 directory
From code only part of nextgenv2 (and not aomedia)

Change-Id: I21f7478a59d525dff23747efe5238ded16b743d2
2016-10-18 17:22:44 -07:00
Urvang Joshi
454280dabf Fix warnings reported by -Wshadow: Part2: av1 directory
While we are at it:
- Rename some variables to more meaningful names
- Reuse some common consts from a header instead of redefining them.

Cherry-picked from aomedia/master: 863b0499

Change-Id: Ida5de713156dc0126a27f90fdd36d29a398a3c88
2016-10-18 17:22:34 -07:00
Nathan E. Egge
f627e58e0f Update segment tree_cdf per frame.
Move computing the segmentation_probs.tree_cdf table per symbol to
 computing it only when the probabilities are updated.

Change-Id: I3826418094bbaca4ded87de5ff04d4b27c85e35a
2016-10-18 16:58:48 -07:00
Nathan E. Egge
56eeaa5daf Rename aom_write_tree_cdf() to aom_write_symbol().
Change-Id: I7c088c55f1c461063976d5bd84ff2026c4f3bc69
2016-10-17 11:54:51 -07:00
Nathan E. Egge
fba2be692f Update partition_cdf per frame.
Move computing the partition_cdf tables per symbol to
 computing them only when the probabilities are updated.

Change-Id: I442f9230ba00be7f5d0558d7c38d7324ad009ee8
2016-10-17 10:21:06 -07:00
Nathan E. Egge
93878c4243 Update inter_ext_tx_cdf per frame.
Move computing the inter_ext_tx_cdf tables per symbol to
 computing them only when the probabilities are updated.

Change-Id: I5e1e62f8eae8f6b2edbbd378beeb786649502c10
2016-10-17 10:20:53 -07:00
Nathan E. Egge
7c5b4c1665 Update intra_ext_tx_cdf per frame.
Move computing the intra_ext_tx_cdf tables per symbol to
 computing them only when the probabilities are updated.

Change-Id: I26d5e419e103093e98a7d896c196176305b50fc9
2016-10-17 08:47:02 -07:00