16787 Commits

Author SHA1 Message Date
Michael Niedermayer
3926ebc702 vorbisdec: Check bark_map_size.
This fixes potential divisions by zero and out of array accesses.

Reported-by: Dale Curtis <dalecurtis@chromium.org>
Found-by: inferno@chromium.org
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2c16bf2de07c68513072bf3cc96401d2c6291a3e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-21 04:23:59 +01:00
Michael Niedermayer
85162a60b8 avcodec/snow: split block clipping checks
Fixes out of array read
Fixes: d4476f68ca1c1c57afbc45806f581963-asan_heap-oob_2266b27_8607_cov_4044577381_snow_chroma_bug.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 61d59703c91869f4e5cdacd8d6be52f8b89d4ba4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-21 04:23:59 +01:00
Michael Niedermayer
573d5fdeda avcodec/msrle: use av_image_get_linesize() to calculate the linesize
Fixes out of array access
Fixes: 14a74a0a2dc67ede543f0e35d834fbbe-asan_heap-oob_49572c_556_cov_215466444_44_001_engine_room.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c919e1ca2ecfc47d796382973ba0e48b8f6f92a2)

Conflicts:

	libavcodec/msrle.c
(cherry picked from commit bc1c8ec5e65098fd2ccd8456f667151dfc9cda42)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-21 04:23:59 +01:00
Michael Niedermayer
954ce59a0a avcodec/wmalosslessdec: fix mclms_coeffs* array size
Fixes corruption of context
Fixes: 8835659dde6a4f7dcdf341de6a45c6c8-signal_sigsegv_1dce67b_4564_cov_2504444599_classical_22_16_1_14000_v3c_0_extend_0_29.wma
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ec9578d54d09b64bf112c2bf7a34b1ef3b93dbd3)

Conflicts:

	libavcodec/wmalosslessdec.c
(cherry picked from commit 5d683807be71002a6c3facef3e26350762a760e2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-21 04:23:59 +01:00
Michael Niedermayer
356f8634e9 wmalosslessdec: make mclms arrays big enough for whats written into them.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a0abefb0af64a311b15141062c77dd577ba590a3)

Conflicts:

	libavcodec/wmalosslessdec.c
2014-03-21 04:23:59 +01:00
Michael Niedermayer
605c3f0515 avcodec/vc1: reset fcm/field_mode in non advanced header parsing
Fixes NULL pointer dereference
Fixes: signal_sigsegv_1ab8bf4_2847_cov_4254117347_SA10091.vc1
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b51e9354772de446e8196dabf9aad1567b22f74d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-21 04:23:59 +01:00
Michael Niedermayer
e0f351a979 avcodec/vmnc: Check that rectangles are within the picture
Prevents out of array accesses with CODEC_FLAG_EMU_EDGE

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6ba02602aa7fc7d38db582e75b8b093fb3c1608d)

Conflicts:

	libavcodec/vmnc.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7c17207ab9acfaa934e8feb8fba90765c9d0b989)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-21 04:23:59 +01:00
Michael Niedermayer
c44ac4d81e dnxhdenc: fix mb_rc size
Fixes out of array access with RC_VARIANCE set to 0

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f1caaa1c61310beba705957e6366f0392a0b005b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-21 04:23:59 +01:00
Michael Niedermayer
84e1aa7768 avcodec/jpeglsdec: check err value for ls_get_code_runterm()
Fixes infinite loop
Fixes Ticket3086

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cc0e47b55096361723b364afa43b79a3f5619cdc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-21 04:23:59 +01:00
Michael Niedermayer
87d335c007 avcodec/parser: reset indexes on realloc failure
Fixes Ticket2982

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f31011e9abfb2ae75bb32bc44e2c34194c8dc40a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-21 04:23:59 +01:00
Martin Storsjö
320df1c7c0 arm: Don't clobber callee saved registers in scalarproduct
q4-q7/d8-d15 are supposed to not be clobbered by the callee.

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit d307e408d4a9ada22df443cc38be77cc5e492694)
2013-12-21 10:00:28 +01:00
Michael Niedermayer
ff3e385d84 avcodec/h264: reduce noisiness of "mmco: unref short failure"
Do not consider it an error if we have no frames and should discard one.
This condition can easily happen when decoding is started from an I frame

Fixes Ticket2811

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 08a89761964bdd0a023eff6d37a1131fb7e1d7a0)

Conflicts:
	libavcodec/h264_refs.c
2013-10-26 01:29:28 +02:00
Michael Niedermayer
1b3803e8c1 avcodec/h264_refs: modify key frame detection heuristic to detect more cases
Fixes Ticket2968

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5ac6b6028f17b64723884c9fa72cfcbd369a1ba2)

Conflicts:
	libavcodec/h264_refs.c
2013-10-24 11:52:05 +02:00
Michael Niedermayer
a6388942f5 avcodec/ffv1enc: update buffer check for 16bps
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3728603f1854b5c79d1a64dd3b41b80640ef1e7f)

Conflicts:

	libavcodec/ffv1enc.c
(cherry picked from commit c900c6e5c26cd86cf34f9c8d4347cedbd01f3935)
2013-09-23 00:02:30 +02:00
Michael Niedermayer
a62852bae0 avcodec/dsputil: fix signedness in sizeof() comparissions
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 454a11a1c9c686c78aa97954306fb63453299760)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-23 00:02:30 +02:00
Michael Niedermayer
40d4e82e2e avcodec/pngdsp: fix (un)signed type in end comparission
Fixes out of array accesses
Fixes Ticket2919

Found_by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 86736f59d6a527d8bc807d09b93f971c0fe0bb07)

Conflicts:

	libavcodec/pngdsp.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-23 00:02:30 +02:00
Michael Niedermayer
a4b4be7493 jpeg2000: check log2_cblk dimensions
Fixes out of array access
Fixes Ticket2895

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9a271a9368eaabf99e6c2046103acb33957e63b7)

Conflicts:

	libavcodec/jpeg2000dec.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Conflicts:

	libavcodec/j2kdec.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-22 22:52:12 +02:00
Michael Niedermayer
4490b3db23 avcodec/rpza: Perform pointer advance and checks before using the pointers
Fixes out of array accesses
Fixes Ticket2850

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3819db745da2ac7fb3faacb116788c32f4753f34)

Conflicts:

	libavcodec/rpza.c
2013-09-22 22:52:12 +02:00
Michael Niedermayer
edb4aad93d avcodec/flashsv: check diff_start/height
Fixes out of array accesses
Fixes Ticket2844

Found-by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 880c73cd76109697447fbfbaa8e5ee5683309446)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-22 22:52:12 +02:00
Michael Niedermayer
e4ad9e9377 update all trac links to use the trac subdomain
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-22 22:52:12 +02:00
Michael Niedermayer
21dd8f5baa huffyuvdec: Skip len==0 cases
Fixes vlc decoding for hypothetical files that would contain such cases.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0dfc01c2bbf4b71bb56201bc4a393321e15d1b31)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5ff41ffeb4cb9ea6df49757dc859619dc3d3ab4f)

Conflicts:

	libavcodec/huffyuv.c
(cherry picked from commit 9bc70fe1ae50fd2faa0b9429d47cfbda01a92ebc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-22 22:52:12 +02:00
Michael Niedermayer
03e2e95beb huffyuvdec: Check init_vlc() return codes.
Prevents out of array writes

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f67a0d115254461649470452058fa3c28c0df294)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 95ab8d33e1a680f30a5a9605175112008ab81afc)

Conflicts:

	libavcodec/huffyuv.c
(cherry picked from commit 277def59fce10d91e3113e5c0f63e22bc4abfa88)

Conflicts:

	libavcodec/huffyuv.c
2013-09-22 22:52:12 +02:00
Michael Niedermayer
d0a225d6c1 mpeg12dec: avoid reinitialization on PS changes when possible.
Fixes Ticket2574

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 970c8df73528659925819dec31c4c8c0887f0321)

Conflicts:
	libavcodec/mpeg12dec.c
2013-07-09 00:55:55 +02:00
Michael Niedermayer
92abb12863 alacenc: Fix missing sign_extend()
Fixes ticket #2497

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8aea2f05dc56f7e7d60767dd27ba8e846a05e8ae)
2013-06-13 00:17:43 +02:00
Michael Niedermayer
8e7e12dcbf h264_cavlc: fix reading skip run
Fixes Ticket2606

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 826b3a75cd295c03720e00d3de83e1abcbedd4b9)

Conflicts:
	libavcodec/h264_cavlc.c
2013-05-30 23:30:43 +02:00
Michael Niedermayer
0147e9f7c6 smacker: remove av_clip_int16()
Fixes Ticket2425

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2211c76287e073a9e176fde7dbb9a63ceb2af8d1)
2013-05-21 00:14:34 +02:00
Carl Eugen Hoyos
6642804c22 Fix type of shared flac table ff_flac_blocksize_table[].
Fixes ticket #2533.
(cherry picked from commit a07ac1f7888fd08e42da2bed0421e74f1cfac177)
2013-05-05 20:39:38 +02:00
Carl Eugen Hoyos
8b597077ae Do not (re-)set libx264 parameter b_tff if interlaced encoding was not requested.
Reconfiguring can break x264 lossless encoding.

Fixes ticket #2165.
(cherry picked from commit 75c7e4583f4fd727d236a12763a265502fe00988)
2013-03-18 02:23:20 +01:00
Piotr Bandurski
8b48e2c654 tiffdec: Use the correct height field.
Fixes Ticket913

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4784a135b2b0fe4d1b4c6256bd37265fc45aed3d)

Conflicts:
	libavcodec/tiff.c
2013-01-07 00:38:13 +01:00
Clément Bœsch
42abae1125 lavc/ass_split: check for NULL pointer in ff_ass_split_override_codes().
This is consistent with the other ff_ass_split_* functions.

It also fixes a crash when trying to split a dialog with text=NULL
(which seems to happen when the text of the dialog is empty); basically,
this commit fixes crashes when trying to encode an empty text subtitle
dialog (see subrip and mov_text encoders).

Fixes Ticket2048.
(cherry picked from commit c83002a4f8042ccfa0688a9a18e8fa0369c1fda8)
2013-01-01 18:23:54 +01:00
Ronald S. Bultje
3c3ee65ebb dxva2: include dxva.h if found
Apparently, some build environments require dxva.h even for dxva2,
while others lack this header entirely.  Including it conditionally
allows building in both cases.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit fa84506177f0246b30d4ea6a99ee5d419f3e4550)

Conflicts:

	configure

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-09-13 04:44:19 +02:00
Michael Niedermayer
5d6982c13b indeo4: check that num_mbs matches
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d3db8988d5befd8702a748cf1957415677bfe75c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-05-03 01:31:07 +02:00
Michael Niedermayer
e11fa0879a dsp: fix diff_bytes_mmx() with small width
Fixes Ticket1068

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 73089eccd3e48539555349b36d8aabbf1cea416e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-05-03 01:27:33 +02:00
Michael Niedermayer
e70d202275 vqavideodev: Check image dimensions
Fixes out of heap array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3583c8706df0abbfa3ecdd6730f4f3d72a01fe6d)
Independently-Found-by: Fabian Yamaguchi
Fixes: CVE-2012-0947

Conflicts:

	libavcodec/vqavideo.c
2012-05-03 00:29:18 +02:00
Michael Niedermayer
9de0c8c60c sonic: update to new API
Fixes Ticket1075

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6f9803e5e02c557e1003cface9f3084a7e1e43e4)

Conflicts:

	libavcodec/sonic.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-05-03 00:28:35 +02:00
Michael Niedermayer
db041fd115 qpeg: Fix out of array writes.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-05-03 00:28:35 +02:00
Fabian Greffrath
7a877418e3 srtdec: fix a format string vulnerability.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit aaa1173de775b9b865a714abcc270816d2f59dff)
2012-05-03 00:28:35 +02:00
ami_stuff
cf9b04c6f2 Replace SSE2 instruction in scalarproduct_float_sse() by SSE equivalent.
Fixes an AAC decoding issue with the sample from ticket #213 on machines
with SSE but without SSE2.
Based on 89411a by Reimar.

(cherry picked from commit f6b78638086beae9bcab672d4c9de1790be5a928)
2012-04-04 09:16:02 +02:00
Michael Niedermayer
89f2d6c349 h264: fix seeking in low delay streams without IDR
Fixes Ticket1165

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3360b8517a1f478c4102072e5eadd8ba78be0538)
2012-04-04 08:41:14 +02:00
Franz Brauße
32a79b5649 smacker audio: sign-extend the initial 16-bit predicted value
Fixes Bug #265

Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 12cbbbb4abda2de0ea123282ccf7ebee61517f7d)
2012-04-01 14:00:13 +02:00
Michael Niedermayer
63945e2226 indeo4: fix LE reader define for backport
Fixes Ticket920

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-01-16 22:26:52 +01:00
Carl Eugen Hoyos
ad6eefc0d8 Fix compilation with yasm-0.6.2. 2012-01-12 16:53:42 +01:00
John Brooks
e540446f6c vc1dec: fix invalid memory access for small video dimensions
For small video dimensions, these calculations of the upper bound
for pixel access may have a negative result. Using an unsigned
comparison to bound a potentially negative value only works if
the greater operand is non-negative. Fixed by doing edge emulation
when the upper bound is probably negative, everywhere that this
pattern appears.

Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>
(cherry picked from commit d209c27b09234cc40bbdbd680aa502b493edf595)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-01-05 02:23:08 +01:00
John Brooks
fc9c5ad9ea rv34: fix invalid memory access for small video dimensions
For small video dimensions calculations of the upper bound for pixel
access may result in negative value. Using an unsigned comparison
works only if the greater operand is non-negative. This is fixed by
doing edge emulation explicitly for such conditions.

Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>
(cherry picked from commit aacf6b3a2fd8bc8603e3deaa6e612ea03cf08707)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-01-05 02:23:02 +01:00
Michael Niedermayer
58afa73338 vorbis: make sure ch is non zero before calling vorbis_residue_decode
This possibly makes part of the CVE-2011-3895 fix unneeded.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ff7f198d7f9504f71676327be0be47661cfe39d6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-01-04 22:19:02 +01:00
Chris Evans
68ee43468e vorbis: An additional defense in the Vorbis codec.
BUG=101458
Review URL: http://codereview.chromium.org/8414025

Fixes second part of CVE-2011-3895

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f74ce3a60d6ef49080df85c44b54280357109f56)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-01-04 22:19:00 +01:00
Chris Evans
80440c5b1d vorbis: Fix decoder bug.
BUG=101458
Review URL: http://codereview.chromium.org/8413019

This fixes part of 2011-3895

bigned-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 68226ed9ecef675895dc55a0c58d587014639a0e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-01-04 22:19:00 +01:00
Michael Niedermayer
89bd49b25c vorbisdec: Make sure blocksize is not set to an invalid value.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 405e99bdfdb363e8dfda275faad9d4fdc9646434)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-01-04 22:18:59 +01:00
Michael Niedermayer
d7e5301e43 vorbis: Fix last quarter of CVE-2011-3893
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6fcf2bb8af0e7d6bb179e71e67e5fab8ef0d2ec2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-01-04 22:18:58 +01:00
Chris Evans
6a3fed8749 ogg: Avoid the possibility to read out-of-bounds of a static global array in Vorbis
decoding.

BUG=100543
Review URL: http://codereview.chromium.org/8365014
This fixes 25% of CVE-2011-3893

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7149fce2cac0474a5fbc5b47add1158cd8bb283e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-01-04 22:18:56 +01:00