401 Commits

Author SHA1 Message Date
Jingning Han
6515afc6b9 Merge "Add min_tx_size variable to recursive transform block partition system" into nextgenv2 2016-11-08 19:14:33 +00:00
Yaowu Xu
f6e958b604 Merge "Fix the bug that PVQ commit broke dering" into nextgenv2 2016-11-08 18:00:53 +00:00
Angie Chiang
13ea019574 Merge changes Ib9428dc9,Ide04717a,If1dba7d8,I6da97880 into nextgenv2
* changes:
  Merge rd_stats only when it is valid
  Let parentheses in handle_inter_mode be symmetric
  Add RD_STATS into MB_MODE_INFO
  Add txb_coeff_cost_map
2016-11-08 17:42:04 +00:00
Jingning Han
e67b38aa7c Add min_tx_size variable to recursive transform block partition system
Replace max_tx_size with min_tx_size for transform type decision.

Change-Id: I64e39923a67903d52b381bd93eaac33b3400a201
2016-11-08 09:36:54 -08:00
Yushin Cho
48f84dbd1c Fix the bug that PVQ commit broke dering
Since PVQ's max block size equals to the max transform size,
daala's definition of OD_BSIZE_MAX was changed from 5 down to 4 to
use AV1's max trasform size 32x32. However, dering also uses
OD_BSIZE_MAX and assumes its value is 5, which caused dering
not working.

Change-Id: I9d82bb24adc7d57552a8e0a8a7e798e77d96fd4b
2016-11-08 08:15:57 -08:00
Tristan Matthews
cc37e36683 pvq: drop unused declaration
Change-Id: I5d95bb897d335dc17aa0ae5e873ba7dee46c6fda
2016-11-07 22:14:50 +00:00
Debargha Mukherjee
03dc29bdf3 Merge "Fix bug in bicubic filter in warped_motion.c" into nextgenv2 2016-11-07 17:58:47 +00:00
Yaowu Xu
00a0e010f7 Merge "New experiment: Perceptual Vector Quantization from Daala" into nextgenv2 2016-11-07 16:00:32 +00:00
David Barker
f23bdca6a8 Fix bug in bicubic filter in warped_motion.c
Previously, do_cubic_filter would return results with the
wrong precision if the sample point was exactly aligned to
a pixel.

Change-Id: I40139f9a6701a8e72e691f37bb352f7814a7f306
2016-11-07 13:47:13 +00:00
Yushin Cho
77bba8d30a New experiment: Perceptual Vector Quantization from Daala
PVQ replaces the scalar quantizer and coefficient coding with a new
design originally developed in Daala. It currently depends on the
Daala entropy coder although it could be adapted to work with another
entropy coder if needed:
./configure --enable-experimental --enable-daala_ec --enable-pvq

The version of PVQ in this commit is adapted from the following
revision of Daala:
fb51c1ade6

More information about PVQ:
- https://people.xiph.org/~jm/daala/pvq_demo/
- https://jmvalin.ca/papers/spie_pvq.pdf

The following files are copied as-is from Daala with minimal
adaptations, therefore we disable clang-format on those files
to make it easier to synchronize the AV1 and Daala codebases in the future:
 av1/common/generic_code.c
 av1/common/generic_code.h
 av1/common/laplace_tables.c
 av1/common/partition.c
 av1/common/partition.h
 av1/common/pvq.c
 av1/common/pvq.h
 av1/common/state.c
 av1/common/state.h
 av1/common/zigzag.h
 av1/common/zigzag16.c
 av1/common/zigzag32.c
 av1/common/zigzag4.c
 av1/common/zigzag64.c
 av1/common/zigzag8.c
 av1/decoder/decint.h
 av1/decoder/generic_decoder.c
 av1/decoder/laplace_decoder.c
 av1/decoder/pvq_decoder.c
 av1/decoder/pvq_decoder.h
 av1/encoder/daala_compat_enc.c
 av1/encoder/encint.h
 av1/encoder/generic_encoder.c
 av1/encoder/laplace_encoder.c
 av1/encoder/pvq_encoder.c
 av1/encoder/pvq_encoder.h

Known issues:
- Lossless mode is not supported, '--lossless=1' will give the same result as
'--end-usage=q --cq-level=1'.
- High bit depth is not supported by PVQ.

Change-Id: I1ae0d6517b87f4c1ccea944b2e12dc906979f25e
2016-11-06 22:18:01 -08:00
Angie Chiang
9a44f5fbc8 Add RD_STATS into MB_MODE_INFO
With RD_STATS in MB_MODE_INFO, we will be able to compare the results
from rate-distortion loop and the results from bitstream packing.

Change-Id: If1dba7d87126577a6f369ac087d4517f7cebb0c5
2016-11-06 12:21:34 -08:00
Debargha Mukherjee
5f305854e6 Increase gm precision from 16 to 32 bit ints
Change-Id: I7117a6c14dc8438e4225b50bd2d3ebbaa7f850cc
2016-11-05 16:50:08 -07:00
Angie Chiang
7a77169a35 Merge changes Ia37f170d,Ie3082db5 into nextgenv2
* changes:
  Record YUV planes' txfm block coeff cost in handle_inter_mode()
  Separate coefficient cost of U/V planes in write_modes_b()
2016-11-04 22:58:58 +00:00
Angie Chiang
59526ead45 Merge changes I3bc782d6,I8359e849,Iae50d0b0,Id1704d88,Ia69f13c4, ... into nextgenv2
* changes:
  Add av1_ prefix on ###_rd_stats functions
  Use init_rd_stats() in encodeframe.c
  Add transform block coefficient cost in RD_STATS for debugging
  Add helper functions to modify RD_STATS
  Add mi_row and mi_col into mbmi to facilitate rd_debug process
  Add token cost comparison in write_modes_b()
2016-11-04 22:43:30 +00:00
James Zern
653fdd6d55 Merge changes I139808f4,I3d97d8db into nextgenv2
* changes:
  warped_motion.c: delete unused filter_4tap[]
  warped_motion.c: quiet float-conversion warnings
2016-11-04 22:34:11 +00:00
Angie Chiang
75f45814ba Separate coefficient cost of U/V planes in write_modes_b()
Change-Id: Ie3082db5b0fead8c322b2aeede4eff7cd723ea12
2016-11-04 11:12:44 -07:00
Angie Chiang
d402282f69 Add token cost comparison in write_modes_b()
This is just partial implementation
Compare token cost of pack_mb_tokens/pack_txb_tokens with token cost
from rate-distortion loop. If there is any difference, dump out mode
info.

Change-Id: I46b373ee2522c5047f799f36baf7cec5fbc06f06
2016-11-04 11:09:24 -07:00
Debargha Mukherjee
68d695b7ca Merge "Further work on 64x64 fwd/inv transform support" into nextgenv2 2016-11-04 09:32:07 +00:00
Debargha Mukherjee
21378b8ad0 Merge "Fix bilateral filter asan error for highbitdepth" into nextgenv2 2016-11-04 05:25:49 +00:00
James Zern
5d54c175c2 warped_motion.c: delete unused filter_4tap[]
Change-Id: I139808f492a9e9dcac44a36237b61231ede7edc3
2016-11-03 20:12:20 -07:00
James Zern
4846e446c6 warped_motion.c: quiet float-conversion warnings
Change-Id: I3d97d8db51a5a5d6b2c1cae47492b53ab37100a7
2016-11-03 20:11:06 -07:00
James Zern
005ff81598 Merge "warped_motion: Fix ubsan warning for signed integer overflow" into nextgenv2 2016-11-04 00:58:07 +00:00
Sarah Parker
db92635745 warped_motion: Fix ubsan warning for signed integer overflow
Change-Id: Ie698aa02ef56128759c71079e9bfa1af25149644
2016-11-04 00:54:25 +00:00
Debargha Mukherjee
c57924cb9e Fix bilateral filter asan error for highbitdepth
BUG=webm:1334

Change-Id: I5886eec0a22a8cc056e1bdb493d2faf183816656
2016-11-03 16:23:09 -07:00
Alex Converse
9613758e71 Merge "Don't use a TX_SIZE as a TX_TYPE" into nextgenv2 2016-11-03 21:44:21 +00:00
Debargha Mukherjee
6a47cff882 Further work on 64x64 fwd/inv transform support
For higher level fwd and inv transform functions.

Change-Id: I91518250a0be7d94aada7519f6c9e7ed024574fb
2016-11-03 14:32:54 -07:00
Debargha Mukherjee
d65708a375 Merge "Replace hard coded numbers with TX_SIZES macro" into nextgenv2 2016-11-03 19:59:10 +00:00
Alex Converse
f0ede18718 Don't use a TX_SIZE as a TX_TYPE
Change-Id: I26b02e6578ad2d82aadfe1df2aeb84e6c11a747b
2016-11-03 10:28:05 -07:00
Yaowu Xu
5bb8f5b705 fix build issue with --enable-delta-q
BUG=webm:1330

Change-Id: I120ce8ea3581018b232b19ca7ffbb07d3e99d8d0
2016-11-03 09:03:39 -07:00
Debargha Mukherjee
e04fdb2308 Replace hard coded numbers with TX_SIZES macro
Replaces a couple of hard coded numbers with TX_SIZES macro
in common/reconiontra.c

Change-Id: I8a2a53ca16bc3ab51409cec340bea55292ff2dee
2016-11-03 08:51:42 -07:00
Yaowu Xu
c1ca945ce5 Merge changes from topic 'update_dering' into nextgenv2
* changes:
  Reformatting the deringing code
  Introducing OD_DERING_SIZE_LOG2 constant (3)
  Renaming deringing blockwise write-back functions to make code clearer
  Deringing refactoring: replace last_sbc with simpler dering_left flag
  Getting rid of the od_dering_in type
2016-11-03 14:03:25 +00:00
Yaowu Xu
7036aee1a4 Merge "Refactoring deringed block list code" into nextgenv2 2016-11-03 13:48:58 +00:00
Yaowu Xu
71c72561fa Merge "Deringing line buffer no longer depends on holding OD_DERING_VERY_LARGE" into nextgenv2 2016-11-03 05:02:32 +00:00
Jingning Han
e60d3294ea Merge "Make recursive txfm encoding process support rectangular tx_size" into nextgenv2 2016-11-03 04:36:55 +00:00
Jingning Han
141f7a9757 Merge "Fix a merge bug between dual_filter and sub8x8mc" into nextgenv2 2016-11-03 01:06:39 +00:00
Jingning Han
1e477f9833 Merge "Remove redundant experimental flags from common_data.h" into nextgenv2 2016-11-03 01:04:45 +00:00
Jean-Marc Valin
39d92a071d Reformatting the deringing code
Manally removed the "clang-format off" lines. The rest is done by clang

Change-Id: I88a2028b55a541729b4e8896cdf66b544e9898bb
2016-11-02 15:51:00 -07:00
Jean-Marc Valin
e04650347c Refactoring deringed block list code
Using a struct named dlist rather than an array named bskip. Simplified some
code.

No change in output

Change-Id: Id40d40b19b5d8f2ebafe347590fa1bb8cb80e6e1
2016-11-02 15:51:00 -07:00
Jean-Marc Valin
01b7780154 Deringing line buffer no longer depends on holding OD_DERING_VERY_LARGE
The OD_DERING_VERY_LARGE values are now explicitly copied to the buffer instead
of being read from the line buffer when we're on the edge of the frame. This
will make it possible to make the line buffer 8-bit for non-high-bitdepth.

No change in output

Change-Id: I1a4134d67ac7f8c239f08d73941405c56f01050b
2016-11-02 15:51:00 -07:00
Jean-Marc Valin
e254241ce7 Introducing OD_DERING_SIZE_LOG2 constant (3)
Also cleans up the size of the deringing destination buffer.

No change in output.

Change-Id: I7fc50d862d3906ce809c1031bf0789acdf39cf34
2016-11-02 15:51:00 -07:00
Jean-Marc Valin
58fdec2cbf Renaming deringing blockwise write-back functions to make code clearer
No change in output.

Change-Id: Ifa5df3adce9f24ef6dcd89a5f33a744bfb57194d
2016-11-02 15:51:00 -07:00
Jean-Marc Valin
3544d15130 Deringing refactoring: replace last_sbc with simpler dering_left flag
No change in output.

Change-Id: I1cc2e14b2bb6c343baa7f88348c875085e5863af
2016-11-02 15:51:00 -07:00
Jean-Marc Valin
39ee109333 Getting rid of the od_dering_in type
We no longer need the deringing code to be generic wrt the input depth.

No change in output.

Change-Id: I2db2beb82f1816e611cd2c0438dff217d363de33
2016-11-02 15:51:00 -07:00
Jingning Han
fee498255d Merge "Remove unused get_intra/inter_scan() from scan.h" into nextgenv2 2016-11-02 22:50:59 +00:00
Jingning Han
1a0faab642 Merge "Remove redundant config flags from get_entropy_context" into nextgenv2 2016-11-02 22:50:49 +00:00
Jean-Marc Valin
d95322a35c Now using a single line buffer
No change in output.

Change-Id: I4701a5517fb97889f970acfb0b44cee51c34fd95
2016-11-02 22:50:10 +00:00
Jean-Marc Valin
621e707259 Only copy data from deringed blocks to the line buffer
No change in output

Change-Id: I6ec4a8c635337562170153585e427afd6f9d9a0f
2016-11-02 22:49:54 +00:00
Jean-Marc Valin
50bb32ec87 Splitting out 8->16 block copy code into copy_sb8_16()
No change in output.

Change-Id: I4f0e37a879432e2647b3debe6a2c0c670a79dd6f
2016-11-02 22:49:39 +00:00
Jean-Marc Valin
39b0d2fb14 Eliminate the big superblock row buffer.
Now only buffering three lines across the entire frame and four lines
over the height of one superblock.

No change in output.

Change-Id: I6b99399974e197dc02f2e4ff2e60cdd7fdaa2e43
2016-11-02 22:48:47 +00:00
Jean-Marc Valin
b154a24283 Making deringing buffer only one row of superblocks at a time
This introduces a line buffer that hold the last three lines of each original
row so that the next row can be deringed with the original input of the upper
row.

No change in output

Change-Id: I8fad3bc48745e9ce3e440289f453477a0c5442c0
2016-11-02 22:48:19 +00:00