1015 Commits

Author SHA1 Message Date
Yaowu Xu
102cdf94ed Move count buffers from stack to heap
This fixes the stack overflow issue on MSVC build.

Change-Id: Icb0a78e5992a097d2192979ec2432546eaa452dd
2016-05-10 14:49:26 -07:00
Jingning Han
5cf3408ba1 Remove unused highbd_fdct32x32 function
The encoder is using vp10_fwd_txfm2d_32x32 now.

Change-Id: I719f18ec0b065f1e062d01fd300533dd2f17c712
2016-05-10 14:33:34 -07:00
Debargha Mukherjee
3fbe6e5e49 Merge "Wedge rd improvements" into nextgenv2 2016-05-10 20:34:00 +00:00
Debargha Mukherjee
447032eb32 Wedge rd improvements
Improves speed by about 10-15% by combining y-only rd with
modeling function in a better way.
Also, coding efficiency improves by about 0.1%

lowres: -1.805% BDRATE with ext-inter

Change-Id: I6ef1f8942ec6806252f3fcf749ae4f30dffe42b1
2016-05-10 11:47:48 -07:00
Alex Converse
fcc7edd48f Merge "Fix some ans const warnings." into nextgenv2 2016-05-10 18:26:53 +00:00
Yaowu Xu
bf692e853d Merge "Fix build without dual-filter" into nextgenv2 2016-05-10 18:10:44 +00:00
Yaowu Xu
0f1ee1caeb Merge "Remove "const" for parameters passed by value" into nextgenv2 2016-05-10 18:05:27 +00:00
Debargha Mukherjee
61ac55314d Merge "Break tile row dependencies." into nextgenv2 2016-05-10 18:00:34 +00:00
Yaowu Xu
fc9deb6b0c Remove "const" for parameters passed by value
This commit removes const from parameters that are passed by value
for consistency in code style.

Change-Id: I2947c4e9cc6e809c4b9b4c162046e45127b8a41c
2016-05-10 09:30:44 -07:00
Yunqing Wang
a2676565d3 Merge "Refine VP10 REFRESH_FRAME_CONTEXT_MODE" into nextgenv2 2016-05-10 15:53:48 +00:00
Yaowu Xu
9e41323f13 Merge "Make type conversions explicit" into nextgenv2 2016-05-10 14:33:05 +00:00
Geza Lore
559e8d8e50 Fix build without dual-filter
Change-Id: I91946940c1540c9f935161da89155ed304055fda
2016-05-10 13:12:07 +01:00
Geza Lore
9ab9438fbb Break tile row dependencies.
When not using ext-tile, there were still dependencies between tile
rows due to various tools (eg intra predictors) relying on the above
row or above mode info, which can be in the above tile. This is now
broken (the same way as it was when ext-tile is enabled) by fixing
the appropriate predicates.

Change-Id: I107dd0d8481775a792f14e05cfbbd761f16cdc1e
2016-05-10 13:09:47 +01:00
Yi Luo
6f3e71606f Merge "HBD hybrid transform 16x16 SSE4.1 optimization" into nextgenv2 2016-05-09 23:58:05 +00:00
Alex Converse
a2db5815c0 Merge "Fix ans+ref_mv build." into nextgenv2 2016-05-09 23:55:48 +00:00
Alex Converse
6e14846c9f Fix some ans const warnings.
Change-Id: I508d497803d0c1085aa6a8b26d7a574cb27dd6e2
2016-05-09 16:50:03 -07:00
hui su
667f6320b0 Fix comment for target_bandwidth in VP9 and VP10
Unlike in VP8, it is in units of bits per second in VP9 and VP10.

Change-Id: Iee1936cc58cdfaff205624c2fe87cecdf7eda123
2016-05-09 16:43:02 -07:00
Sarah Parker
d119a5f5c8 Merge "Edit ext-tx so it isn't doing redundant prunes" into nextgenv2 2016-05-09 22:57:37 +00:00
Alex Converse
8c9da4e943 Fix ans+ref_mv build.
Use vp10_read/write instead of vpx_read/write.

Change-Id: I2b7f17e9cdbea14ff48f4bd9776dd3e6aff17a2b
2016-05-09 15:35:58 -07:00
Alex Converse
afad52c670 Merge "Remove the ANS rename on pack_mb_tokens()." into nextgenv2 2016-05-09 22:16:35 +00:00
Yunqing Wang
484ba02435 Refine VP10 REFRESH_FRAME_CONTEXT_MODE
In VP10, REFRESH_FRAME_CONTEXT_OFF mode is only set when the error
resillient mode is on. Instead of being used to decide how to update
the frame contexts, it is used to decide if or not to reset the
frame contexts.

To verify, ran borg test on lowres set. The result is neutral.
Overall PSNR: -0.006%; SSIM: -0.006%.

Change-Id: Ic48265cf7488e80c6f5aab3eef7ba1c273506419
2016-05-09 14:20:50 -07:00
Sarah Parker
f546383b73 Edit ext-tx so it isn't doing redundant prunes
The original pruning function was not taking into account
that certain tx sizes/block sizes use a reduced tx set.

Prune 1: -0.3% performance drop, 20% speedup on foreman video
Prune 2: -0.48% perfomance drop, 30% speedup on foreman video

Change-Id: I557e919d97a89f787b47b3c8579a080db57f91d0
2016-05-09 13:35:42 -07:00
Zoe Liu
b9d0d3f4c7 Turn on the use of upsampled refs for ext-refs
Without this patch, the experiment of ext-refs showed almost no coding
gains compared to the baseline. This is because when ext-refs is on, the
use of upsampled reference is off.

With this patch, the ext-refs experiment works with the upsampled
references and shows coding gains in Overall PSNR as follows, with ~5%
slow down for encoding time:

lowres: Avg - -0.965;  BDRate - -0.844
derflr: Avg - -0.847;  BDRate - -0.669

Note that the previous patch a912c6ec314d816767a4c3eb4e5e1bddcc4c1186
that "Make LAST_FRAME always point to the newly coded frame in ext-refs"
made ext-refs work with the upsampled refereces.

Change-Id: Id79248d71760109fb9198af4f45718b17455555f
2016-05-09 13:34:08 -07:00
Alex Converse
4d22cc1578 Remove the ANS rename on pack_mb_tokens().
This fixes the ans+var_tx combination.

Change-Id: I4c34edb1deac4475c97ce1907c1d6bdf23ce3fc0
2016-05-09 12:02:01 -07:00
Yi Luo
412ad22f46 HBD hybrid transform 16x16 SSE4.1 optimization
- Tx_type: DCT_DCT, DCT_ADST, ADST_DCT, ADST_ADST.
- Update vp10_fht16x16_test.cc to do bit-exact test against
  latest C version.
- HBD encoder speed improves ~1.8%.

Change-Id: Icfc799a212e5289bcf6cedcae3722032133a2bc6
2016-05-09 11:07:01 -07:00
Yaowu Xu
98c59c98ba Make type conversions explicit
This eliminates MSVC compiler warnings.

Change-Id: Id6ace2586ed7c6248366905b133448fe8ecbd53d
2016-05-07 20:33:40 +00:00
Yaowu Xu
569101bed8 Merge "Make parameter types consistent" into nextgenv2 2016-05-07 20:33:25 +00:00
Zoe Liu
a912c6ec31 Make LAST_FRAME always point to the newly coded frame in ext-refs
This patch changes the encoder only for the ext-refs experiment. For
each newly coded frame to refresh the LAST_FRAME, the decoder is
notified that the LAST4_FRAME is to be refreshed, and read out the
updated reference frame buffer vitural indexes for the next coded
frame in a way that:
LAST4_FRAME => LAST_FRAME,
LAST_FRAME  => LAST2_FRAME,
LAST2_FRAME => LAST3_FRAME, and
LAST3_FRAME => LAST4_FRAME.

Compared against the original ext-refs experiment in TOT, a small gain
is achieved in overall PSNR:
lowres Avg: -0.154
lowres BDRate: -0.044

Change-Id: I648810c146a3cd915b408274a9373b7d38324864
2016-05-07 00:27:51 -07:00
Jingning Han
bd33326372 Dual prediction filter type for motion compensated reference
Make the bit-stream level support per direction filter type coding
for motion compensated reference.

Change-Id: I61a2360b301075f6734cfd9711b7ae68f214174d
2016-05-07 03:03:04 +00:00
Yaowu Xu
ad841b7dac Make parameter types consistent
This fixes compiler warnings from MSVC.

Change-Id: Iaac0e994869561371295578a893f766493ce0544
2016-05-06 23:39:46 +00:00
Yaowu Xu
71d4e444c1 Merge "Change initializations of variables with type "int_mv"" into nextgenv2 2016-05-06 20:21:39 +00:00
Jingning Han
eb366a0312 Merge "Clean up ext-interp experiment" into nextgenv2 2016-05-06 17:16:56 +00:00
Yaowu Xu
824a8b228d Change initializations of variables with type "int_mv"
This is to make MSVC happy and eliminate build errors.

Change-Id: Ic81e7c7516923913e6e7a652b691953e4a1af8aa
2016-05-06 16:52:12 +00:00
Alex Converse
130cccba8d Rename pick_filter_intra.
The word 'pick' is usually used in functions that make decisions where
the bitstream allows multiple legal choices, and not to limit the
bitstream format itself.

Change-Id: Ia60709c29e004475e1aa8861aefded27ebaf4712
2016-05-05 17:06:54 -07:00
Jingning Han
8b084b683c Clean up ext-interp experiment
Remove the unused sub-experiments within the ext-interp experiment.

Change-Id: I716e3392412d02c56f9395a86c9cab02f580fa59
2016-05-05 16:29:21 -07:00
Sarah Parker
9dfe45a84d Merge "Add 1D tx set that corresponds to reduced ext tx inter sets" into nextgenv2 2016-05-05 23:06:14 +00:00
Jingning Han
4862363e6a Merge "Remove a redundant variable definition from sub8x8 RD loop" into nextgenv2 2016-05-05 21:17:31 +00:00
Debargha Mukherjee
47fd87e77f Merge "Fix mismatch with ext-interp." into nextgenv2 2016-05-05 21:06:51 +00:00
Jingning Han
cf51217148 Remove a redundant variable definition from sub8x8 RD loop
Change-Id: I464cbb75fbd3872f66ca024dd803605542a9d887
2016-05-05 12:41:05 -07:00
Geza Lore
1502d9e44a Fix mismatch with ext-interp.
The encoder signals the interp filter type in the frame header if all
blocks use the same filter (see bitstream.c:fix_interp_filter). This
decision is made based on the counts, but with ext-interp, the counts
are actually only incremented for blocks that fail vp10_is_interp_needed
(see for example encodeframe.c:update_state), otherwise a default value
is used (EIGHTTAP_REGULAR). The decoder however first checks if the
interp filter is signaled at the frame level, and uses that filter type
for all blocks, even if the default value should have been used.

This patch makes the decoder first check with vp10_is_interp_needed
to see if the default value should be used and then checks the frame
level signaling, which reconciles the difference between encoder and
decoder.

Change-Id: I87857ade42dea06b0d5ec2a029e9219268334dbb
2016-05-05 18:24:21 +01:00
Geza Lore
a905c45c77 Refactor supertx rd search
General code cleanup, but also use the same supertx condition for
ext-partition-types as for conventional partitions.

Change-Id: If86eb18b3c07b9c60434eec2c98b97ce93665b67
2016-05-05 18:20:12 +01:00
Geza Lore
c9cb346e56 Merge "Fix vp10_accumulate_frame_counts once and for all." into nextgenv2 2016-05-05 10:19:41 +00:00
Yaowu Xu
81eb71f00c Change to use proper type in vp{9,10}_token_state
"qc" in vp{9,10}_token_state is used to save quantized coefficients, this
commit changes the type from short to tran_low_t to properly reflect
the value range for highbitdepth build.

This fixes an out-of-range bug when optimize_b is used in highbitdepth
build.

Change-Id: Ibf330879e6ac6ae8f099e085caa9d3d9a889fde8
2016-05-04 12:14:11 -07:00
Yaowu Xu
0d7dc0cae1 Change to use proper type in vp10_token_state
"qc" in vp10_token_state is used to save quantized coefficients, this
commit changes the type from short to tran_low_t to properly reflect
the value range for highbitdepth build.

This fixes an out-of-range bug when optimize_b is used in highbitdepth
build.

Change-Id: I914c6fd3d3f4b9d061f9ed7cc5f08a883ab59dcd
2016-05-04 11:59:10 -07:00
Sarah Parker
3da61efe3b Add 1D tx set that corresponds to reduced ext tx inter sets
This is the set of 1D transforms that are used in each
ext_tx_used_inter set. The 1D sets will help speed up
the ext tx pruning functions.

Change-Id: Ib46ad26be2df60b3bfcd2f22d96e7f38ae286df5
2016-05-04 11:42:32 -07:00
Geza Lore
c959151fa2 Fix vp10_accumulate_frame_counts once and for all.
This ensures the multi-threaded and single-threaded encoder/decoder
always uses the same probability contexts.

Change-Id: I6f1e7c6bd8808c390c1dc0a628ae97db3acedf6d
2016-05-04 11:32:40 +01:00
Debargha Mukherjee
4f5045299e Merge "Refactoring and uv fix for wedge" into nextgenv2 2016-05-03 22:36:24 +00:00
Debargha Mukherjee
3407785536 Refactoring and uv fix for wedge
lowres: -1.72%

Change-Id: I4c883097caac72fab8e01945454579891617145e
2016-05-03 08:02:08 -07:00
Alex Converse
8307045cb5 Merge "Use signed variables in the lookahead." 2016-05-02 21:32:39 +00:00
Yi Luo
9be7075f61 Merge "HBD hybrid transform 8x8 SSE4.1 optimization" into nextgenv2 2016-05-02 17:34:50 +00:00