401 Commits

Author SHA1 Message Date
Jingning Han
a9336328d4 Make recursive txfm encoding process support rectangular tx_size
This commit makes the encoding process of the recursive transform
block partition support both rectangular and square transform block
sizes as the starting point. If the coding block size is rectangular,
it would allow the transform block size to start from the largest
rectangular transform size, and recursive parse to the selected
coding sizes.

Change-Id: I576628b9166565bada6a918f0a1e67849dfef4cd
2016-11-02 15:48:07 -07:00
Jean-Marc Valin
ca1eb5dc58 Duplicating deringing input superblock copy to make upcoming changes easier
No change in output

Change-Id: Iaa06043dcc31308c83f667424e5a83c2db50ed24
2016-11-02 22:46:53 +00:00
Jean-Marc Valin
8e941780be Using a uniform definition for "bsize" in deringing filter
No change in output

Change-Id: Ia3a1679aa75cb58f4bc6459791e061176eeafd52
2016-11-02 22:46:27 +00:00
Jean-Marc Valin
eab77ea936 Using the copied input for od_dir_find8()
No change in output

Change-Id: Iec1411c35bf175a462eade34e89a4c60eb2a1da4
2016-11-02 15:41:01 -07:00
Yaowu Xu
6285c6674d Merge "Increase deringing horizontal padding to 4 pixels on each side" into nextgenv2 2016-11-02 22:37:39 +00:00
Jean-Marc Valin
471687a9ac Increase deringing horizontal padding to 4 pixels on each side
This makes vectorization easier by having buffer lines be a multiple of 4.

No change in output

Change-Id: I7ec06e03a49554206af0a55aab03daccc411b50f
2016-11-02 22:37:35 +00:00
Yaowu Xu
4cb9a620db Merge "De-sparsifying the deringing output buffer" into nextgenv2 2016-11-02 22:35:05 +00:00
Jean-Marc Valin
82c65fc837 De-sparsifying the deringing output buffer
No change in output

Change-Id: I940203975564aedca8734d6f74b013edb513f517
2016-11-02 22:35:00 +00:00
Yaowu Xu
44f3587459 Merge "No need to store the deringing filter direction variance in an array" into nextgenv2 2016-11-02 22:34:46 +00:00
Jingning Han
46003149e5 Fix a merge bug between dual_filter and sub8x8mc
The function module in inter_predictor() has been changed to
universally support arbitrary block size inter prediction. Hence
sub8x8mc can be a standalone experiment now.

Change-Id: Ie9d87f61fc317b1d114edb4e0bf5544f918ed08e
2016-11-02 14:57:11 -07:00
Jingning Han
d611808324 Remove redundant experimental flags from common_data.h
No coding statistics change.

Change-Id: I88cbb828308b5796a2e87079c2f1bf0dabd99a11
2016-11-02 14:51:23 -07:00
Jingning Han
c104b8f269 Merge "Support rectangular tx_size in the common lib" into nextgenv2 2016-11-02 21:49:41 +00:00
Jingning Han
a834925778 Merge "Make highbd rectangular transform block available in the common lib" into nextgenv2 2016-11-02 21:49:07 +00:00
Jean-Marc Valin
643902d621 No need to store the deringing filter direction variance in an array
No change in output

Change-Id: Ifa5c5d4ed33ff11ea3c56ee5d559c7a40599b3dc
2016-11-02 13:15:42 -07:00
Sarah Parker
fcb2ca6eda Merge "Fix ubsan left shift warnings in warped motion library" into nextgenv2 2016-11-02 19:45:45 +00:00
Jingning Han
653102ab1c Remove unused get_intra/inter_scan() from scan.h
Change-Id: I96fc1da1ce56593ae35ebbc93a668e4ba241234a
2016-11-02 12:00:51 -07:00
Jingning Han
8b5380ac77 Remove redundant config flags from get_entropy_context
The rectangular transform syntax is by default supported, hence
no need to put it under the experimental flag. This does not change
the coding statistics.

Change-Id: I3a147503d973a03400f8a86e11f07c7d754e6234
2016-11-02 11:48:39 -07:00
Jingning Han
9fe31390ca Support rectangular tx_size in the common lib
Change-Id: I4128ab932a967a3d657bb1f95f0fa2af20a06469
2016-11-02 11:48:31 -07:00
Jingning Han
5238e6eaee Make highbd rectangular transform block available in the common lib
Change-Id: Ief08b23b30b78d640f6d7c702145e5bcf1b37b57
2016-11-02 11:47:48 -07:00
Debargha Mukherjee
deef66db01 Merge "Adding 64x64 forward and inverse transforms" into nextgenv2 2016-11-02 18:40:55 +00:00
Yaowu Xu
1af3d51685 Merge changes I313bde67,I2ddc2d70,Ifb9094c3,I9051ed6e,I5681e332, ... into nextgenv2
* changes:
  Avoid the "initial copy" in the deringing filter
  Only copy the deringed blocks back into the buffer
  Reducing copies in deringing filter
  sb_all_skip_out() now computes a list of deringed blocks
  compute bskip as we go
  Revert "Fix dering filter when using 4:2:2 or 4:4:0 subsampling"
2016-11-02 18:02:56 +00:00
Debargha Mukherjee
67d134772c Adding 64x64 forward and inverse transforms
Change-Id: I213f3111fc0656aecd1303a8b871ecded2b92bc2
2016-11-02 09:48:46 -07:00
Jingning Han
6a503e4110 Merge "Make rectangular transform block available in the common lib" into nextgenv2 2016-11-02 16:17:00 +00:00
Jingning Han
f8a29663be Merge "Simplify tx_size enums" into nextgenv2 2016-11-02 16:16:50 +00:00
Jean-Marc Valin
bcf3580b1e Avoid the "initial copy" in the deringing filter
No change in output

Change-Id: I313bde67e59835f88e3b2e6079b0df2d7ed1a903
2016-11-02 08:23:04 -07:00
Jean-Marc Valin
7618daa555 Only copy the deringed blocks back into the buffer
No change in output

Change-Id: I2ddc2d70c6534e7cfd315d66e838410677f91356
2016-11-02 08:22:58 -07:00
Jean-Marc Valin
cf23aefab5 Reducing copies in deringing filter
Only copy the modified pixels from the first filter back into the input of the
second filter.

Change-Id: Ifb9094c33c876a8c6caa0f68771fc7ef59c78b53
2016-11-02 08:22:51 -07:00
Jean-Marc Valin
3e44bccb50 sb_all_skip_out() now computes a list of deringed blocks
No change in output

Change-Id: I9051ed6e1fbca7d80412ba2b53f7aacbc3ef70eb
2016-11-02 08:22:45 -07:00
Jean-Marc Valin
71466d2288 compute bskip as we go
Change-Id: I5681e3329ad3677296161de59f5ff1236a14f086
2016-11-02 08:22:38 -07:00
Yaowu Xu
3e90f84a34 Revert "Fix dering filter when using 4:2:2 or 4:4:0 subsampling"
This reverts commit 401204a50bea8b21e5fcd721cd5db513b8f70e72.

Change-Id: Id27eadf679b0df2d2ccfab61155be29979b0b6ba
2016-11-02 08:22:02 -07:00
Jingning Han
ec419e0771 Make rectangular transform block available in the common lib
This prepares the integration of rectangular transform block size
with recursive transform block partition system.

Change-Id: Id96aa3790dace15619c665f438241938992d1730
2016-11-01 22:25:54 -07:00
Jingning Han
aad298ffcf Simplify tx_size enums
Remove redundant experimental flag. This does not change the coding
statistics.

Change-Id: I35b3cb04025c5c2d2744312e5efc00d0473c990d
2016-11-01 21:12:55 -07:00
Yi Luo
fb77385fd0 Merge "Remove unused copies of transform related source code" into nextgenv2 2016-11-02 01:43:19 +00:00
Yi Luo
7f6bf9c70d Merge "Hybrid inverse transforms 16x16 AVX2 optimization" into nextgenv2 2016-11-02 01:43:02 +00:00
Jingning Han
9679464e28 Merge "Change TXFM_CONTEXT from TX_SIZE to uint8_t" into nextgenv2 2016-11-02 01:18:19 +00:00
Jingning Han
746e2220b5 Merge "Rework transform block partition context model" into nextgenv2 2016-11-02 01:18:13 +00:00
Urvang Joshi
a5b09216b5 Merge "Revert of "Mark bogus palette color probabilities as zero"." into nextgenv2 2016-11-02 00:31:55 +00:00
Jingning Han
8b9478af1e Change TXFM_CONTEXT from TX_SIZE to uint8_t
Count the transform block partition context in the unit of pixels.

Change-Id: Ibb66f053526ed347ad0274b78db7ac35cc086b0e
2016-11-01 15:44:26 -07:00
Urvang Joshi
eb54e0cde8 Revert of "Mark bogus palette color probabilities as zero".
Reverted commit: f8306bfdc (with some changes).

Reason: This was triggering an assert in debug build because of zero
probability values. So, using an "UNUSED_PROB" macro to replace these to
retain clarity.

Assertion failure can be reproduced as follows:

$ make clean; extra_cflags='-O0 -g -fno-inline' ../../configure
--enable-debug --enable-experimental --enable-palette && make -j 16

$ ./aomenc -D --codec=av1 ~/videos/screen_content_set/gimp.y4m -o
/tmp/foo.webm --tune-content=screen --limit=50

Pass 1/2 frame   50/51      8976B    1436b/f   86169b/s 2902620 us
(17.23 fps)
Pass 2/2 frame   25/0          0B 2933053 us 8.52 fps [ETA  unknown]
aomenc: ../../av1/encoder/cost.c:46: cost: Assertion `prob != 0' failed.
Aborted (core dumped)

Change-Id: I47a76b8f415060909bc8448fae3002857eb61d8e
2016-11-01 15:25:57 -07:00
Yi Luo
ea1167c33f Remove unused copies of transform related source code
- Library size reduces: 165 kB, 292 kB (HBD).

Change-Id: I50cb630dde326bd2a28c0db4b7e2d53c2fd94a2a
2016-11-01 15:07:46 -07:00
Jingning Han
c8b8936fdc Rework transform block partition context model
This commit allows the partition context model to account for the
maximum transform block size of the coding block.

Change-Id: I22b91e85fff70faa974afd362ce327d3f2eda81d
2016-11-01 15:00:04 -07:00
Yaowu Xu
980eb2e9fa Merge "Change to use correct variable in for-loop" into nextgenv2 2016-11-01 21:07:11 +00:00
Yi Luo
7317200002 Hybrid inverse transforms 16x16 AVX2 optimization
- Add unit tests to verify the bit-exact result.
- User level time reduction (EXT_TX):
    encoder: 3.63%
    decoder: 2.36%
- Also add tx_type=V_DCT...H_FLIPADST SSE2 for 16x16 inv txfm.

Change-Id: Idc6d9e8254aa536e5f18a87fa0d37c6bd551c083
2016-11-01 13:38:20 -07:00
Yaowu Xu
ddcdd5b1e5 Merge "Fix a compiler warning with --enable-adapt-scan" into nextgenv2 2016-11-01 18:12:49 +00:00
Yaowu Xu
2ce9707910 Merge "Resolve build issue --enable-aom-qm" into nextgenv2 2016-11-01 18:12:39 +00:00
Yaowu Xu
6043bfdb03 Change to use correct variable in for-loop
Change-Id: I252c2f06dfe256d2d33fd1abc42aaadf50273cc8
2016-11-01 09:54:05 -07:00
Jingning Han
ae81f8b2ab Merge "Make txfm_partition_update support rectangular tx_size" into nextgenv2 2016-11-01 16:51:03 +00:00
Yaowu Xu
b386f0b762 Fix a compiler warning with --enable-adapt-scan
Change-Id: I93b191a522ed3e3ca9a363beab4292f64e869610
2016-11-01 09:40:12 -07:00
Yaowu Xu
a5924740a2 Resolve build issue --enable-aom-qm
Change-Id: I9f52ddb53b39cefd2e0ee7144203e1f3958d01aa
2016-11-01 09:32:03 -07:00
Yaowu Xu
fd601e346c Merge "Rename av1_convolve.[hc] to convolve.[hc]" into nextgenv2 2016-11-01 02:25:19 +00:00