Commit Graph

34256 Commits

Author SHA1 Message Date
Michael Niedermayer
aa427537b5 avcodec/h264_refs: Check that long references match before use
Fixes out of array read
Fixes: 59bb925e90201fa0f87f0a31945d43b5/asan_heap-oob_4a52e5_3388_66027f11e3d072f1e02401ecc6193361.jvt

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-29 03:27:28 +01:00
Michael Niedermayer
abee0a1c60 avcodec/utils: Clear dimensions in ff_get_buffer() on failure
Fixes out of array access
Fixes: 482d8f2fd17c9f532b586458a33f267c/asan_heap-oob_4a52b6_7417_1d08d477736d66cdadd833d146bb8bae.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-28 20:10:51 +01:00
Michael Niedermayer
4f03bebc79 avcodec/utils: Use 64bit for aspect ratio calculation in avcodec_string()
Fixes integer overflow
Fixes: 3a45b2ae02f2cf12b7bd99543cdcdae5/asan_heap-oob_1dff502_8022_899f75e1e81046ebd7b6c2394a1419f4.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-28 20:10:51 +01:00
Michael Niedermayer
dad354f38d avcodec/hevc: Check max ctb addresses for WPP
Fixes out of array read
Fixes: 2f95ddd996db8a6281d2e18c184595a7/asan_heap-oob_192fe91_3330_58e4441181e30a66c19f743dcb392347.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-28 15:48:55 +01:00
Ganesh Ajjanagadde
79abf2d0de avcodec/aac_tablegen: get rid of hardcoded tables entirely
Commit 96786a12f6 makes runtime
initialization cheap.

Tested with FATE, with/without --enable-hardcoded-tables.

Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
2015-11-28 09:41:15 -05:00
Michael Niedermayer
861f2b2a53 avcodec/vp3: Fix several memleaks
Fixes: 1536b9b096a8f95b742bae9d3d761cc6/signal_sigsegv_294aaed_4460_b209bd1e7cebe458b53072a44191316d.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-28 01:01:41 +01:00
Michael Niedermayer
6105b7219a avcodec/vp3: Clear context on reinitialization failure
Fixes null pointer dereference
Fixes: 1536b9b096a8f95b742bae9d3d761cc6/signal_sigsegv_294aaed_2039_8d1797aeb823ea43858d0fa45c9eb899.ogv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-28 00:25:08 +01:00
Michael Niedermayer
d85aa76115 avcodec/hevc: allocate entries unconditionally
Fixes out of array access
Fixes: 08664a2a7921ef48172f26495c7455be/asan_heap-oob_23036c6_3301_523388ef84285a0270caf67a43247b59.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 23:37:42 +01:00
Michael Niedermayer
36205501ba avcodec/pthread_slice: Allow calling ff_alloc_entries() multiple times to readjust the entry count
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 23:37:42 +01:00
Michael Niedermayer
d5028f61e4 avcodec/hevc_cabac: Fix multiple integer overflows
Fixes: 04ec80eefa77aecd7a49a442cc02baea/asan_heap-oob_19544fa_3303_1905796cd9d8e15f86d664332caabc00.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 23:37:42 +01:00
Rostislav Pehlivanov
6b40755158 aacenc: fix broken build with hardcoded tables
ff_aac_tableinit is a macro in the case of hardcoded tables, so wrap
that up in a function (similar to how the decoder template does it) and
use that as the argument for ff_thread_once().

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-11-27 21:24:42 +00:00
Michael Niedermayer
feb3f39614 avcodec/jpeg2000dwt: Check ndeclevels before calling dwt_encode*()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 21:02:13 +01:00
Michael Niedermayer
75422280fb avcodec/jpeg2000dwt: Check ndeclevels before calling dwt_decode*()
Fixes out of array access
Fixes: 01859c9a9ac6cd60a008274123275574/asan_heap-oob_1dff571_8250_50d3d1611e294c3519fd1fa82198b69b.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 20:59:26 +01:00
Ganesh Ajjanagadde
8453095f3e avcodec/aac_tablegen: make exp2_lut static
Suggested-by: Clément Bœsch <u@pkh.me>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
2015-11-27 13:23:35 -05:00
Michael Niedermayer
ef9f7bbfa4 avcodec/hevc: Check entry_point_offsets
Fixes out of array read
Fixes: 007c4a36608ebdf27ee260ad60a81184/asan_heap-oob_32076b4_2243_116b1cb29d91cc4974d6680e3d10bd91.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 18:53:21 +01:00
Rostislav Pehlivanov
ec0719264c aac: temporarily un-share aac_table_init AVOnce variable
AAC-Fixed decoder segfaulted. This commit makes the aac encoder
and decoder init the table twice in case of transcoding again.

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-11-27 17:51:42 +00:00
Rostislav Pehlivanov
591fbd629e aactab.h: fix comment
Previous commit broke it.

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-11-27 15:36:57 +00:00
Rostislav Pehlivanov
222545cc7e aactab.h: update and correct comment
Tables in that file have been used by both for a long time now.

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-11-27 15:00:13 +00:00
Rostislav Pehlivanov
3d62e7a30f aacenc: make threadsafe
Since the ff_aac_tableinit() can be called by both the encoder and
the decoder (in case of transcoding) this commit shares the AVOnce
variable to prevent this.

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-11-27 14:52:35 +00:00
Michael Niedermayer
8000d484b8 avcodec/cabac: Check initial cabac decoder state
Fixes integer overflows
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2340_591e9810c7b09efe501ad84638c9e9f8.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Found-by: xiedingbao (Ticket4727)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 13:52:47 +01:00
Michael Niedermayer
a1f6b05f52 avcodec/cabac_functions: Fix "left shift of negative value -31767"
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2340_591e9810c7b09efe501ad84638c9e9f8.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Found-by: xiedingbao (Ticket4727)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 13:52:47 +01:00
Ganesh Ajjanagadde
96786a12f6 avcodec/aac_tablegen: speed up table initialization
This speeds up aac_tablegen to a ludicruous degree (~97%), i.e to the point
where it can be argued that runtime initialization can always be done instead of
hard-coded tables. The only cost is essentially a trivial increase in
the stack size.

Even if one does not care about this, the patch also improves accuracy
as detailed below.

Performance:
Benchmark obtained by looping 10^4 times over ff_aac_tableinit.

Sample benchmark (x86-64, Haswell, GNU/Linux):
old:
1295292 decicycles in ff_aac_tableinit,     512 runs,      0 skips
1275981 decicycles in ff_aac_tableinit,    1024 runs,      0 skips
1272932 decicycles in ff_aac_tableinit,    2048 runs,      0 skips
1262164 decicycles in ff_aac_tableinit,    4096 runs,      0 skips
1256720 decicycles in ff_aac_tableinit,    8192 runs,      0 skips

new:
21112 decicycles in ff_aac_tableinit,     511 runs,      1 skips
21269 decicycles in ff_aac_tableinit,    1023 runs,      1 skips
21352 decicycles in ff_aac_tableinit,    2043 runs,      5 skips
21386 decicycles in ff_aac_tableinit,    4080 runs,     16 skips
21299 decicycles in ff_aac_tableinit,    8173 runs,     19 skips

Accuracy:
The previous code was resulting in needless loss of
accuracy due to the pow being called in succession. As an illustration
of this:
ff_aac_pow34sf_tab[3]
old : 0.000000000007598092294225
new : 0.000000000007598091426864
real: 0.000000000007598091778545

truncated to float
old : 0.000000000007598092294225
new : 0.000000000007598091426864
real: 0.000000000007598091426864

showing that the old value was not correctly rounded. This affects a
large number of elements of the array.

Patch tested with FATE.

Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
2015-11-27 06:38:06 -05:00
Rostislav Pehlivanov
f5b7a29ae8 aac_ltp: actually signal LTP as off during EIGHT_SHORT windows
This hugely reduces the echo which was introduced with the previous
commit (though likely because previously everything was broken).
Makes LTP actually worthwhile now.

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-11-26 18:20:42 +00:00
Rostislav Pehlivanov
1e5dbb3409 aac_ltp: split, reorder and improve prediction algorithm
This commit attempts to mirror what the decoder does more closely
in addition to fixing some shortcomings.
2015-11-26 17:40:04 +00:00
Ganesh Ajjanagadde
a239ce7074 avcodec/faandct: remove L suffixes for floating point literal
Should fix issues with ppc, tested by bug reporter.

Reported-by: John Warburton <john@johnwarburton.net>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
2015-11-26 11:19:03 -05:00
Timothy Gu
9078a694f3 aaccoder_twoloop: Mark sfdiff as av_unused
Silences warning when building without assertions

Signed-off-by: Claudio Freire <klaussfreire@gmail.com>
2015-11-26 03:46:09 -03:00
Claudio Freire
3b1cab9351 AAC encoder: fix wrong gain sacalefactor being set
In some conditions, where the first band was being zeroed
mainly, the wrong global gain scalefactor would be written
to the stream since it's always taken from the first band
regardless of whether it's been marked as zero or not.

So, always make sure it contians something useful.
2015-11-26 03:37:29 -03:00
Claudio Freire
fc36d852ee AAC encoder: Fix application of M/S with PNS
When both M/S coding and PNS are enabled, scalefactors
and coding books would be mistakenly clobbered when setting
the M/S flag on PNS'd bands. The flag needs to be set to
signal the generation of correlated noise, but the scalefactors,
coefficients and the coding books need to be kept intact.
2015-11-26 03:27:06 -03:00
Michael Niedermayer
b3494e3c3e avcodec/pthread_slice: Remove rets_count
It appears rets_count is redundant

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 00:56:12 +01:00
James Almer
3885ef0c6c avcodec/mjpegdec: fix typo on a warning 2015-11-25 19:24:24 -03:00
Ganesh Ajjanagadde
81a0aec29e avcodec/aacps_tablegen: use hypot()
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
2015-11-24 20:36:40 -05:00
Ganesh Ajjanagadde
aececd11ab avcodec/aacps_tablegen_template: replace #define by typedef
See e.g https://stackoverflow.com/questions/1666353/are-typedef-and-define-the-same-in-c
for rationale.

Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
2015-11-24 20:36:40 -05:00
Ganesh Ajjanagadde
5472de5ca8 avcodec/aac_defines: replace #define by typedef
See e.g https://stackoverflow.com/questions/1666353/are-typedef-and-define-the-same-in-c
for rationale.

Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
2015-11-24 20:36:40 -05:00
Michael Niedermayer
4ea4d2f438 avcodec/h264_slice: Limit max_contexts when slice_context_count is initialized
Fixes out of array access
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2049_f2192b6829ab6e0eefcb035329c03c60.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-24 22:17:36 +01:00
Derek Buitenhuis
9c35b8e219 Merge commit 'fb8753ada23189076bdf903c1c001c0ca8287fae'
* commit 'fb8753ada23189076bdf903c1c001c0ca8287fae':
  qsvenc: factor out common options

Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2015-11-24 13:48:28 +00:00
Michael Niedermayer
13834c1016 avcodec/mpegvideo_enc: Remove slice structured mode from H.263 as well as the code automatically enabing it
There is no such thing as a slice structured mode in the original version 1  H.263,
that mode was added in H.263+ in 1998. Also the headers for slice structured mode
are not part of the older version 1 and this would result in unplayable files

An alternative to this patch would be to merge the H263 and H263P AVCodecs and use
other means to distinguish the older and newer versions.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-23 22:47:41 +01:00
Clément Bœsch
56bdf61baa avutil/motion_vector: export subpel motion information
FATE test changes because of the switch from shift to division.
2015-11-23 10:55:15 +01:00
Ganesh Ajjanagadde
90ccfdb65b Revert "avcodec/dct32_template: fix typo"
This reverts commit 82c5f31789.

Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
2015-11-22 17:01:42 -05:00
Ganesh Ajjanagadde
82c5f31789 avcodec/dct32_template: fix typo
Found-by: Clément Bœsch <u@pkh.me>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
2015-11-22 16:56:49 -05:00
Ganesh Ajjanagadde
61a1ca13ea avcodec/ppc/fdctdsp: use M_SQRT2 instead of ad-hoc SQRT2
This actually fixes an incorrect float literal. It is believed by
examining the precision that the literals were all pre-computed as
floats, resulting in this needless loss of precision. There is no
benefit to keeping such reduced precision:
1. These constants are used for static array computation, hence
compile-time.
2. They will be treated as doubles anyway, since f specifier was not
present.

Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
2015-11-22 16:16:16 -05:00
Ganesh Ajjanagadde
ccf3c69403 avcodec/dct32_template: use M_SQRT1_2 instead of actual literal
Fixed point value remains unchanged.

Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
2015-11-22 16:16:16 -05:00
Ganesh Ajjanagadde
824ba897bd avcodec/cos_tablegen: use M_PI instead of actual literal
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
2015-11-22 16:16:16 -05:00
Ganesh Ajjanagadde
94565f3791 avcodec/ac3: use M_SQRT1_2, M_SQRT2
This uses M_SQRT1_2, M_SQRT2 instead of the actual literals. This yields
greater precision in some places in avcodec/ac3, while fixed point
values remain unchanged.

Reviewed-by: Clément Bœsch <u@pkh.me>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
2015-11-22 16:16:16 -05:00
Michael Niedermayer
764900d645 avcodec/dvdsubdec: Fix types for offset1/2 in ff_dlog()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-22 19:26:05 +01:00
Derek Buitenhuis
98540a2c7d Merge commit '4d8f536b535487063a08609636e712ad86d2ad54'
* commit '4d8f536b535487063a08609636e712ad86d2ad54':
  qsvenc: print the actual video parameters used by MSDK

Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2015-11-22 17:46:57 +00:00
Derek Buitenhuis
09f9ef8536 Merge commit 'f6c94457b44f41d900cd0991857f54e1f0ccedd6'
* commit 'f6c94457b44f41d900cd0991857f54e1f0ccedd6':
  mpegvideo_enc: enable rtp_mode when multiple slices are used

Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2015-11-22 17:46:26 +00:00
Derek Buitenhuis
819a99ef21 Merge commit 'd80f0a4ad634b3949b91b85f21fd608c0cddeef7'
* commit 'd80f0a4ad634b3949b91b85f21fd608c0cddeef7':
  mpevideo_enc: disallow multiple slices for h261 and flv

Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2015-11-22 17:43:26 +00:00
Derek Buitenhuis
fe20e34b13 Merge commit '79ae1e630b476889c251fc905687a3831b43ab5e'
* commit '79ae1e630b476889c251fc905687a3831b43ab5e':
  avcodec: Define side data type for fallback track

Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2015-11-22 17:41:27 +00:00
Derek Buitenhuis
cde68661bf Merge commit '5c30ae1a09b66179e16694f6137658023ed1fef3'
* commit '5c30ae1a09b66179e16694f6137658023ed1fef3':
  dvdsubdec: Validate the RLE offsets

  Conflicts:
      libavcodec/dvdsubdec.c

Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2015-11-22 17:15:21 +00:00
Derek Buitenhuis
5ebaf1e1cb Merge commit '79d89cf2f4b62eeb653fd8139041c87e75f7ca65'
* commit '79d89cf2f4b62eeb653fd8139041c87e75f7ca65':
  flacenc: Clamp user-supplied min/max prediction orders

  Conflicts:
      libavcodec/flacenc.c

Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2015-11-22 17:10:42 +00:00