132 Commits

Author SHA1 Message Date
Jingning Han
6515afc6b9 Merge "Add min_tx_size variable to recursive transform block partition system" into nextgenv2 2016-11-08 19:14:33 +00:00
Angie Chiang
13ea019574 Merge changes Ib9428dc9,Ide04717a,If1dba7d8,I6da97880 into nextgenv2
* changes:
  Merge rd_stats only when it is valid
  Let parentheses in handle_inter_mode be symmetric
  Add RD_STATS into MB_MODE_INFO
  Add txb_coeff_cost_map
2016-11-08 17:42:04 +00:00
Jingning Han
e67b38aa7c Add min_tx_size variable to recursive transform block partition system
Replace max_tx_size with min_tx_size for transform type decision.

Change-Id: I64e39923a67903d52b381bd93eaac33b3400a201
2016-11-08 09:36:54 -08:00
Yaowu Xu
02d4c3b780 Fix compiler warning of un-used variables
Change-Id: I17d05bbf75a201fd010fc17e2d9bd0db8ef36d41
2016-11-07 19:56:13 +00:00
Yaowu Xu
3442b4b159 change to call fwd_txfm()
The transform functions have been refactored in nextgenv2, this commit
resolves the calls in pvq patch to use this new scheme.

Change-Id: I1b56e75106a3357bb19bd7df2b4ba305eb9ed185
2016-11-07 10:40:41 -08:00
Yaowu Xu
d6ea71cf73 Resolve merge issues with --enable-pvq
This commit resolves some compiling issues due to merge.

Change-Id: I0eef8aa36c404e185e0b0004948a49307c360d3e
2016-11-07 10:35:55 -08:00
Yaowu Xu
00a0e010f7 Merge "New experiment: Perceptual Vector Quantization from Daala" into nextgenv2 2016-11-07 16:00:32 +00:00
Yushin Cho
77bba8d30a New experiment: Perceptual Vector Quantization from Daala
PVQ replaces the scalar quantizer and coefficient coding with a new
design originally developed in Daala. It currently depends on the
Daala entropy coder although it could be adapted to work with another
entropy coder if needed:
./configure --enable-experimental --enable-daala_ec --enable-pvq

The version of PVQ in this commit is adapted from the following
revision of Daala:
fb51c1ade6

More information about PVQ:
- https://people.xiph.org/~jm/daala/pvq_demo/
- https://jmvalin.ca/papers/spie_pvq.pdf

The following files are copied as-is from Daala with minimal
adaptations, therefore we disable clang-format on those files
to make it easier to synchronize the AV1 and Daala codebases in the future:
 av1/common/generic_code.c
 av1/common/generic_code.h
 av1/common/laplace_tables.c
 av1/common/partition.c
 av1/common/partition.h
 av1/common/pvq.c
 av1/common/pvq.h
 av1/common/state.c
 av1/common/state.h
 av1/common/zigzag.h
 av1/common/zigzag16.c
 av1/common/zigzag32.c
 av1/common/zigzag4.c
 av1/common/zigzag64.c
 av1/common/zigzag8.c
 av1/decoder/decint.h
 av1/decoder/generic_decoder.c
 av1/decoder/laplace_decoder.c
 av1/decoder/pvq_decoder.c
 av1/decoder/pvq_decoder.h
 av1/encoder/daala_compat_enc.c
 av1/encoder/encint.h
 av1/encoder/generic_encoder.c
 av1/encoder/laplace_encoder.c
 av1/encoder/pvq_encoder.c
 av1/encoder/pvq_encoder.h

Known issues:
- Lossless mode is not supported, '--lossless=1' will give the same result as
'--end-usage=q --cq-level=1'.
- High bit depth is not supported by PVQ.

Change-Id: I1ae0d6517b87f4c1ccea944b2e12dc906979f25e
2016-11-06 22:18:01 -08:00
Angie Chiang
616990d607 Merge rd_stats only when it is valid
Change-Id: Ib9428dc9b6e224fdb5d410368c5b92042c96f68a
2016-11-06 15:25:37 -08:00
Angie Chiang
78a3bc165c Let parentheses in handle_inter_mode be symmetric
Change-Id: Ide04717a8ce2a7c1245f9614485647e296e96abd
2016-11-06 13:01:16 -08:00
Angie Chiang
9a44f5fbc8 Add RD_STATS into MB_MODE_INFO
With RD_STATS in MB_MODE_INFO, we will be able to compare the results
from rate-distortion loop and the results from bitstream packing.

Change-Id: If1dba7d87126577a6f369ac087d4517f7cebb0c5
2016-11-06 12:21:34 -08:00
Angie Chiang
85279f6668 Add txb_coeff_cost_map
The txb_coeff_cost_map is a 16x16 map which records each single
transform block's cost from the transform block's location in 4-pixel
unit in recursive transform experiment.

Change-Id: I6da97880c457680594bca56617084010891beaa2
2016-11-06 11:55:17 -08:00
Debargha Mukherjee
5f305854e6 Increase gm precision from 16 to 32 bit ints
Change-Id: I7117a6c14dc8438e4225b50bd2d3ebbaa7f850cc
2016-11-05 16:50:08 -07:00
James Zern
99ff89b6fb Merge "rdopt: clear maybe-uninitialized variable warning" into nextgenv2 2016-11-05 03:19:33 +00:00
Sarah Parker
70c4fab569 rdopt: clear maybe-uninitialized variable warning
av1/encoder/rdopt.c:9533 ‘zeromv[1].as_int’ may be used
uninitialized in this function [-Wmaybe-uninitialized]
this was spurious given the logic in the if

Change-Id: I8ddfe7e46d1bf5593cc8624f05c9f181243a87d4
2016-11-04 17:56:23 -07:00
Angie Chiang
628d7c915b Record YUV planes' txfm block coeff cost in handle_inter_mode()
Change-Id: Ia37f170d8fd961d78a751d84b9525ab7e973b81a
2016-11-04 11:12:44 -07:00
Angie Chiang
c0feea8a0c Add av1_ prefix on ###_rd_stats functions
Change-Id: I3bc782d68bcd9b52b38210eec9eecb21146fde75
2016-11-04 11:12:44 -07:00
Angie Chiang
d81fdb41e6 Add transform block coefficient cost in RD_STATS for debugging
Change-Id: Iae50d0b0c4f8f383ab4f91d2c1c2fa4e799c7250
2016-11-04 11:11:27 -07:00
Angie Chiang
d7246171b5 Add helper functions to modify RD_STATS
Those functions includes
init_rd_stats()
invalid_rd_stats()
merge_rd_stats()

This CL help simplify the code.

Change-Id: Id1704d883bd21a039b0478a940994ca14184ae1c
2016-11-04 11:11:27 -07:00
Jingning Han
4be3214fec Merge "Properly schedule the transform block recursion order" into nextgenv2 2016-11-04 17:53:53 +00:00
Yue Chen
95a3898cbd Merge "Remove duplicated variables in EXT_INTER" into nextgenv2 2016-11-04 17:11:08 +00:00
Jingning Han
98d6a1f247 Properly schedule the transform block recursion order
This commit replaces the offset based block index calculation with
incremental based one. It does not change the coding statistics.

Change-Id: I3789294eb45416bd0823e773ec30f05ed41ba0dc
2016-11-04 09:06:49 -07:00
Jingning Han
137b2671eb Fix format issue in handle_inter_mode()
Change-Id: I681fd799cf46991de419cc867ccb649a6990c19d
2016-11-04 08:31:24 -07:00
Angie Chiang
e89ea0ceb7 Merge "Refactor: Replace rate dist sse skip by RD_STATS in VAR_TX" into nextgenv2 2016-11-04 05:42:59 +00:00
Angie Chiang
b5dda4887b Refactor: Replace rate dist sse skip by RD_STATS in VAR_TX
This is to facilitate implementation of rd_debug tool; it doesn't change
coding behavior.

Change-Id: I0eb82b31473883ba6652ed11dca09b9ec4530183
2016-11-03 17:51:26 -07:00
James Zern
97a2c675e7 Merge "rdopt,global-motion: Fix -1 indexing ubsan warning" into nextgenv2 2016-11-03 22:59:34 +00:00
Sarah Parker
182953b299 rdopt,global-motion: Fix -1 indexing ubsan warning
Change-Id: I1b3caf3543ab385f39f5f253c9949ad89ea5af7d
2016-11-03 22:58:47 +00:00
Yue Chen
9d3e478e72 Remove duplicated variables in EXT_INTER
Introduced by merge commit 141f7a9

Change-Id: Idd68e09a6cd925d97466eabebe0e4905b5031340
2016-11-03 15:12:42 -07:00
Jingning Han
4b47c937d0 Fix txb_w/h use case in av1_tx_block_rd_b()
Match them with block_row/col index.

Change-Id: Idf0f924a093e5312b0a36b765d295e52d033eb5a
2016-11-03 09:20:08 -07:00
Jingning Han
18482fe32d Refactor recursive transform block partition search
This commit refactors the recursive transform block partition
search process to make it support rectangular transform block size
coding.

Change-Id: I0207ae40d83c7eae3cb5d460e403f470747590d3
2016-11-02 17:03:09 -07:00
Zoe Liu
bd163bc199 Merge "Make a small code clean on handle_inter_mode()" into nextgenv2 2016-11-02 16:39:36 +00:00
Jingning Han
c8b8936fdc Rework transform block partition context model
This commit allows the partition context model to account for the
maximum transform block size of the coding block.

Change-Id: I22b91e85fff70faa974afd362ce327d3f2eda81d
2016-11-01 15:00:04 -07:00
Zoe Liu
82c8c92cc5 Make a small code clean on handle_inter_mode()
Change-Id: I5fb4898045a481f7996c2ad019d2f741aab08fc7
2016-11-01 14:52:34 -07:00
Jingning Han
ce059e86fb Use get_entropy_context() in select_tx_block
Replace redundant separate handling to retrieve the context value.

Change-Id: I18dde4599cd08ffe33a78694ec377487609de1b1
2016-10-31 16:27:28 -07:00
Jingning Han
f65b870e27 Refactor max_blocks_wide/high computation
Factor common codes that show up in multiple places.

Change-Id: I0a72213a151f74bdad926d59f86f0a28d00968fc
2016-10-31 12:39:36 -07:00
Jingning Han
73d65a49a9 Merge "Refactor rate-distortion optimization of recursive transform partition" into nextgenv2 2016-10-29 23:01:14 +00:00
Jingning Han
9fdc42293f Refactor rate-distortion optimization of recursive transform partition
Support rectangular transform block in the rate-distortion cost
estimator.

Change-Id: I99201fcae797c1ed2f2184021a215867eac0288f
2016-10-28 14:48:40 -07:00
Sarah Parker
68a26b6b4a Bitwise to logical & in rdopt ext tx prune function
Making this change in case the future implementation changes and the
compairson is no longer between single bits.

Change-Id: I94f474ce7d82febfa23cec65cbe1b9d240b42e02
2016-10-28 13:19:33 -07:00
Angie Chiang
3655dcd4cf Fix tmp_rd type error in handle_inter_mode()
Change-Id: I9398c77c12e9c4caa19a76b92e3035a3135cfd7a
2016-10-28 09:05:27 -07:00
Jingning Han
cb277c0b82 Merge "Refactor recursive transform block partition search" into nextgenv2 2016-10-28 15:50:36 +00:00
Jingning Han
6675bbca0e Merge "Simplify logics in encode_inter_mb_segment" into nextgenv2 2016-10-28 15:50:15 +00:00
Yaowu Xu
2df83d5c10 Merge "Remove av1_use_hp_mv()" into nextgenv2 2016-10-28 15:26:35 +00:00
Jingning Han
5822404485 Refactor recursive transform block partition search
Use unified transform block size and coding block size map. This
prepares for the integration of 2x2 transform block size and the
rectangular transform block size.

Change-Id: I99f51017d19aef337639b708ee9c7faedcc20935
2016-10-28 05:12:19 +00:00
Jingning Han
c4049db573 Simplify logics in encode_inter_mb_segment
Unify coefficient context used by different experiments. Make
block size and transform block size consistent with rest codebase.

Change-Id: I237336f161d6c473b88c59c48ee68d24b75ce738
2016-10-28 05:12:05 +00:00
Zoe Liu
b99af6e3e9 Merge "A small bug fix in ext-refs on the RD mode selection" into nextgenv2 2016-10-27 22:43:55 +00:00
Alex Converse
6317c88f5a Remove av1_use_hp_mv()
It always returns true since the related misc_fix[1] was merged.

[1] 23e83574b6a5105bdc686c49f2d5909f33ea721f

Change-Id: Ie3af685572a2f0a42d2b9fb9903c1abeea225dfd
2016-10-27 14:33:48 -07:00
Angie Chiang
3d41cb339c Merge "Refactor: Localize tmp_rd in handle_inter_mode()" into nextgenv2 2016-10-27 18:27:18 +00:00
Jingning Han
607fa6a6ce Support potential 2x2 transform block unit
Make the codec support 2x2 tranform block unit for chroma components.

Change-Id: Ic454535bd5620abe88a2e99789160cc4664ee518
2016-10-26 15:38:13 -07:00
Jingning Han
b0a7130656 Convert tx_size to relative depth to fetch tx_size_cost
Use the relative transform partition depth as index to fetch the
tx_size_cost value.

Change-Id: I7d5119817baa96f23c32828065ff3175bb9f75cf
2016-10-26 17:12:41 +00:00
Angie Chiang
c352e79ee6 Merge "Simplify interpolation filter search in handle_inter_mode()" into nextgenv2 2016-10-26 16:51:58 +00:00