32106 Commits

Author SHA1 Message Date
Janne Grunau
4457e6137d h264: check sps.log2_max_frame_num for validity
Fixes infinite or long taking loop in frame num gap code in
the fuzzed sample bipbop234.ts_s223302.

CC: libav-stable@libav.org
(cherry picked from commit d7d6efe42b0d2057e67999b96b9a391f533d2333)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:34:10 +01:00
Luca Barbato
08d9fd611e ppc: always use pic for shared libraries
CC: libav-stable@libav.org
(cherry picked from commit 1944d532a8a1c4b12222f0acfeb1153630dbc996)

Conflicts:

	configure
2013-01-12 19:33:10 +01:00
Janne Grunau
5fa739e685 h264: enable low delay only if no delayed frames were seen
Dropping frames is undesirable but that is the only way by which the
decoder could return to low delay mode. Instead emit a warning and
continue with delayed frames.
Fixes a crash in fuzzed sample nasa-8s2.ts_s20033 caused by a larger
than expected has_b_frames value. Low delay keeps getting re-enabled
from a presumely broken SPS.

CC: libav-stable@libav.org
(cherry picked from commit 706acb558a38eba633056773280155d66c2f4b24)

Conflicts:

	libavcodec/h264.c
2013-01-12 19:32:24 +01:00
Alex Converse
b143844ea0 aacdec: Fix an off-by-one overwrite when switching to LTP profile from MAIN.
Found-by: pawlkt
CC: libav-stable@libav.org
Fixes: CVE-2012-5144
(cherry picked from commit 6d5b0092678b2a95dfe209a207550bd2fe9ef646)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:27:42 +01:00
Janne Grunau
10ff052c60 lavf: avoid integer overflow in ff_compute_frame_duration()
Scaling the denominator instead of the numerator if it is too large
loses precision. Fixes an assert caused by a negative frame duration in
the fuzzed sample nasa-8s2.ts_s202310.

CC: libav-stable@libav.org
(cherry picked from commit 7709ce029a7bc101b9ac1ceee607cda10dcb89dc)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:27:42 +01:00
Luca Barbato
4ede95e69c vp6: properly fail on unsupported feature
Interlacing is not supported at all and mismanaged down the normal
codepaths causing possible buffer management issues.

Fixes: CVE-2012-2783
(cherry picked from commit be75fed9755c1285ba084574aff2d7ee0f81110d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:27:29 +01:00
Reinhard Tartler
ce8910d861 h264: Fix parameters to ff_er_add_slice() call
s->mb_x is reset to zero a couple of lines above. It does not make
sense to call ff_er_add_slice() with 0 as endx when the end of the
macroblock row was reached. Fixes unnecessary and counterproductive
error resilience in https://bugzilla.libav.org/show_bug.cgi?id=394.

(cherry picked from commit e6160bda98641b7d4f86de15761ad2a962f21a36)

Conflicts:

	libavcodec/h264.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:

	libavcodec/h264.c
2013-01-12 19:22:22 +01:00
Justin Ruggles
3d0c9c9af6 flacenc: ensure the order is within the min/max range in LPC order search
This fixes use of uninitialized values when the FLAC encoder uses the
2-level, 4-level, and 8-level search methods. Fixes failure of the
fate-flac-24-comp-8 test when run using valgrind.
(cherry picked from commit 3a2731cbd31d0c5681ddbc7c78edd5c53c4d0032)

Conflicts:

	libavcodec/flacenc.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:20:27 +01:00
Luca Barbato
f3f22f183f yuv4mpeg: reject unsupported codecs
The muxer already rejects unsupported pixel formats, reject also
unsupported codecs to prevent dangerous misuses.
(cherry picked from commit 424b1e764263b1493de4c34365ef367ddae856db)

Conflicts:

	libavformat/yuv4mpeg.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:20:27 +01:00
Sami Pietila
bfbff1c748 vp8: reset loopfilter delta values at keyframes.
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>

(cherry picked from commit 0bf511d579c7b21f1244eec688abf571ca1235bd)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:20:27 +01:00
Luca Barbato
7fd7950174 vp56: release frames on error
Fixes CVE-2012-2783

CC: libav-stable@libav.org

(cherry picked from commit f33b5ba63eee96c9d1c7f0e568169cb0c3694238)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:20:27 +01:00
Luca Barbato
700fb8c8dd vp56: make parse_header return standard error codes
Returning 0 for failure is misleading.

CC: libav-stable@libav.org

(cherry picked from commit bb675d3ac6d722d5e117ae9042a996b55ca05b1d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:20:27 +01:00
Anton Khirnov
9f80712454 ivi_common: check that scan pattern is set before using it.
Fixes CVE-2012-2791.

CC: libav-stable@libav.org

(cherry picked from commit deabb52ab4c1fdb3dd319f3980b1489a182011f1)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:20:27 +01:00
Piotr Bandurski
da5f4e4d19 tiffdec: Use the correct height field.
Fixes Ticket913

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4784a135b2b0fe4d1b4c6256bd37265fc45aed3d)

Conflicts:
	libavcodec/tiff.c
(cherry picked from commit fe0e64ca6431c2f606bc702c1a4e230f22531a4f)
2013-01-07 00:45:02 +01:00
Piotr Bandurski
fe9cbf582b tiffdec: Use the correct height field.
Fixes Ticket913

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4784a135b2b0fe4d1b4c6256bd37265fc45aed3d)

Conflicts:
	libavcodec/tiff.c
(cherry picked from commit fe0e64ca6431c2f606bc702c1a4e230f22531a4f)
2013-01-07 00:43:03 +01:00
Reinhard Tartler
642d758a2d Update RELEASE file for 0.7.7 2013-01-04 07:43:39 +01:00
Michael Niedermayer
aa45b90804 alsdec: Check k used for rice decoder.
Values that fail this check will cause failure of decode_rice()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 23aae62c2cb4504a09ceb8cd0cabc1c8b260f521)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:38 +01:00
Alex Converse
549b8083d6 tiffenc: Check av_malloc() results.
(cherry picked from commit b92dfb56d4582633571db18c3d904f8602eaa2a6)

Conflicts:

	libavcodec/tiffenc.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:38 +01:00
Luca Barbato
ec6719f655 mpegaudiodec: fix short_start calculation
The value should be always 3, as it follows from the specification.

Fix a stack buffer overflow in exponents_from_scale_factors as reported
by asan. Thanks to Dale Curtis for the sample vector.
(cherry picked from commit 97cfa55eea39cef30abe14682c56c1e4e7f6f10d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:38 +01:00
Jindřich Makovička
11ecd8574a h264: avoid stuck buffer pointer in decode_nal_units
When decode_nal_units() previously encountered a NAL_END_SEQUENCE,
and there are some junk bytes left in the input buffer, but no start codes,
buf_index gets stuck 3 bytes before the end of the buffer.

This can trigger an infinite loop in the caller code, eg. in
try_decode_trame(), as avcodec_decode_video() then keeps returning zeroes,
with 3 bytes of the input packet still available.

With this change, the remaining bytes are skipped so the whole packet gets
consumed.

CC:libav-stable@libav.org

Signed-off-by: Jindřich Makovička <makovick@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 1a8c6917f68f7378465e18f7615762bfd22704c2)

Conflicts:

	libavcodec/h264.c
2013-01-04 07:43:38 +01:00
Anton Khirnov
5754176b5b yuv4mpeg: return proper error codes.
Fixes Bug 373.

CC:libav-stable@libav.org
(cherry picked from commit d3a72becc6371563185a509b94f5daf32ddbb485)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:38 +01:00
Franz Brauße
fb3189ce8b smacker audio: sign-extend the initial 16-bit predicted value
Fixes Bug #265

Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 12cbbbb4abda2de0ea123282ccf7ebee61517f7d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-01-04 07:43:38 +01:00
Anton Khirnov
8168a7cec9 vf_pad: don't give up its own reference to the output buffer.
Conflicts:
	libavfilter/vf_pad.c

Fixes Bug 245

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-01-04 07:43:38 +01:00
Anton Khirnov
562d6fd5b5 avidec: return 0, not packet size from read_packet().
(cherry picked from commit eeade678f0a2bac127aeed2fb68d8717a6463420)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-01-04 07:43:38 +01:00
Sean McGovern
dd14723602 wmapro: prevent division by zero when sample rate is unspecified
This fixes Bugzilla #327:

Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>
(cherry picked from commit 3680b2435101a5de56821718a71c828320d535a0)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-01-04 07:43:38 +01:00
Thilo Borgmann
9474c93028 alsdec: fix number of decoded samples in first sub-block in BGMC mode.
Fixes CVE-2012-2790

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 66197988b1ee914825afbc3084e6da63f862068a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:38 +01:00
Mans Rullgard
7e070cf202 alsdec: remove dead assignments
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 4ca6d206d1b5beea42c4290d2ee801aaf5cd31f0)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:38 +01:00
Thilo Borgmann
1b48a426a9 alsdec: Fix out of ltp_gain_values read.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 97f0efbfb86d24f081b2caa39f6249e05c95c2ef)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:38 +01:00
Michael Niedermayer
e3e369f696 alsdec: Check that quantized parcor coeffs are within range.
ALS spec:
	11.6.3.1.1 Quantization and encoding of parcor coefficients
	...
	In all cases the resulting quantized values ak are restricted to the range [-64,63].

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 5b051ec3bdc78f3d89e8d1425674cde8fd6c9ccc)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:38 +01:00
Michael Niedermayer
6996a2f796 cavsdec: check for changing w/h.
Our decoder does not support changing w/h.

Fixes CVE-2012-2777 and CVE-2012-2784.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit c20a69630619d14ae92c5541d52c579d7c8f3e94)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Anton Khirnov
05f5a2eb62 avidec: use actually read size instead of requested size
Fixes CVE-2012-2788
(cherry picked from commit 0af49a63c7f87876486ab09482d5b26b95abce60)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Michael Niedermayer
4a636a5e43 wmaprodec: check num_vec_coeffs for validity
Fixes CVE-2012-2789

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 99f392a584dd10b553facc8e819f2c7e982e176d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Michael Niedermayer
44da556815 lagarith: check count before writing zeros.
Fixes CVE-2012-2793

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit b631e4ed64f7d1b9ca8f897fda31140e8d1fad81)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Michael Niedermayer
aa097b4d5f indeo5: check tile size in decode_mb_info().
This prevents writing into a too small array if some parameters changed
without the tile being reallocated.

Fixes CVE-2012-2794

CC:libav-stable@libav.org

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 2d09cdbaf2f449ba23d54e97e94bd97ca22208c6)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Janne Grunau
8148833193 indeo5: prevent null pointer dereference on broken files
Found by John Villamil <johnv@matasano.com>
(cherry picked from commit 366ac22ea5a8bab63c7f46cdad2ddb2ff22cdbed)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Kostya Shishkov
3c0f84402b indeo: check for invalid motion vectors
(cherry picked from commit cf61aaaca16810b9b3a28395ed48fda8db0e87d9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Kostya Shishkov
601fa56582 indeo: clear allocated band buffers
(cherry picked from commit 23ba1503f2b11057c65052b4a07961236d8d69c7)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Kostya Shishkov
c0df6a24ce indeo: check custom Huffman tables for errors
(cherry picked from commit fe7a37c36febd71576cbefc385d995a8d6e444e7)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Kostya Shishkov
2d63f9b4ef dfa: add some checks to ensure that decoder won't write past frame end
(cherry picked from commit 8099187e897ddc90cb3902332c76fb2542dac308)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Anton Khirnov
4c849c6991 dfa: check that the caller set width/height properly.
Fixes CVE-2012-2786.
(cherry picked from commit ee715f49a06bf3898246d01b056284a9bb1bcbb9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Aneesh Dogra
42c3a3719b bytestream: add a new set of bytestream functions with overread checking
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
2013-01-04 07:43:37 +01:00
Michael Niedermayer
7a0ff7566b avsdec: Set dimensions instead of relying on the demuxer.
The decode function assumes that the video will have those dimensions.

Fixes CVE-2012-2801

CC:libav-stable@libav.org

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 85f477935cd6b34e6ec2716b20e15ce748277a89)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Mina Nagy Zaki
10c244cc89 lavfi: avfilter_merge_formats: handle case where inputs are same
This fixes a double-free crash if lists are the same due to the two
merge_ref() calls at the end of the (useless) merging that happens.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 11b6a82412bcd372adf694a26d83b07d337e1325)

Conflicts:

	libavfilter/formats.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:21 +01:00
Janne Grunau
99008ba366 rv34: use AVERROR return values in ff_rv34_decode_frame()
Also adds an error message.
(cherry picked from commit 29330721b0e8514f9f8b4d54be75a662a2b79e44)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-01-04 07:43:21 +01:00
Martin Storsjö
a81c1ea2eb h263: Add ff_ prefix to nonstatic symbols
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit ddce8953a5056800ec795df2dfd84fc17a11b5fc)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-01-04 07:43:21 +01:00
Max Lazarov
0892a6340f eval: fix swapping of lt() and lte()
CC: libav-stable@libav.org
(cherry picked from commit caac3ab6efde4fc9769e8a7472269356f262970a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-01-04 07:43:21 +01:00
Anton Khirnov
d3e2f35f7a bmpdec: only initialize palette for pal8.
Gray8 is not considered to be paletted, so this would cause an invalid
write.

Fixes bug 367.

CC: libav-stable@libav.org
(cherry picked from commit 8b78c2969a5b7dca939d93bf525aa2bcd737b5d9)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-01-04 07:43:21 +01:00
Kostya Shishkov
e39fc137ae vc1dec: add flush function for WMV9 and VC-1 decoders
CC: libav-stable@libav.org
(cherry picked from commit 4dc8c8386eef942dba35c4f2fb3210e22b511a5b)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-01-04 07:43:20 +01:00
Mans Rullgard
a2ae183a38 h264: allow cropping to AVCodecContext.width/height
Override the frame size from the SPS with AVCodecContext values
if the latter specify a size smaller by less than one macroblock.
This is required for correct cropping of MOV files from Canon cameras.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 30f515091c323da59c0f1b533703dedca2f4b95d)

Conflicts:

	libavcodec/h264.c
2013-01-04 07:43:20 +01:00
Clément Bœsch
20940780c7 lavc/ass_split: check for NULL pointer in ff_ass_split_override_codes().
This is consistent with the other ff_ass_split_* functions.

It also fixes a crash when trying to split a dialog with text=NULL
(which seems to happen when the text of the dialog is empty); basically,
this commit fixes crashes when trying to encode an empty text subtitle
dialog (see subrip and mov_text encoders).

Fixes Ticket2048.
(cherry picked from commit c83002a4f8042ccfa0688a9a18e8fa0369c1fda8)
2013-01-01 18:27:28 +01:00