32071 Commits

Author SHA1 Message Date
Jindřich Makovička
9822e3aa52 h264: avoid stuck buffer pointer in decode_nal_units
When decode_nal_units() previously encountered a NAL_END_SEQUENCE,
and there are some junk bytes left in the input buffer, but no start codes,
buf_index gets stuck 3 bytes before the end of the buffer.

This can trigger an infinite loop in the caller code, eg. in
try_decode_trame(), as avcodec_decode_video() then keeps returning zeroes,
with 3 bytes of the input packet still available.

With this change, the remaining bytes are skipped so the whole packet gets
consumed.

CC:libav-stable@libav.org

Signed-off-by: Jindřich Makovička <makovick@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 1a8c6917f68f7378465e18f7615762bfd22704c2)

Conflicts:

	libavcodec/h264.c
2012-10-19 19:23:14 +02:00
Anton Khirnov
0b923a2b72 vf_pad/scale: use double precision for aspect ratios.
Fixes Bug 203.

CC:libav-stable@libav.org
(cherry picked from commit ba04177eeb690ba4e93ec30fc8eb02f5319f844b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-19 19:23:14 +02:00
Anton Khirnov
d792be5681 yuv4mpeg: return proper error codes.
Fixes Bug 373.

CC:libav-stable@libav.org
(cherry picked from commit d3a72becc6371563185a509b94f5daf32ddbb485)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-19 19:23:14 +02:00
Franz Brauße
443f1463c0 smacker audio: sign-extend the initial 16-bit predicted value
Fixes Bug #265

Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 12cbbbb4abda2de0ea123282ccf7ebee61517f7d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-10-18 11:13:05 +02:00
Anton Khirnov
be209bdabb vf_pad: don't give up its own reference to the output buffer.
Conflicts:
	libavfilter/vf_pad.c

Fixes Bug 245

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-10-18 11:06:57 +02:00
Justin Ruggles
24025cc0b9 libvorbis: use VBR by default, with default quality of 3
(cherry picked from commit 147ff24a0e8d819615a0f596df3ea47dddd79fdc)

Conflicts:
	libavcodec/libvorbis.c

Fixes a part of Bug 277

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-10-18 10:53:51 +02:00
Justin Ruggles
5920d00d74 libvorbis: fix use of minrate/maxrate AVOptions
- enable the options for audio encoding
- properly check for user-set maxrate
- use correct calling order in vorbis_encode_setup_managed()
(cherry picked from commit 182d4f1f3855460ee8634ea052f33332cf9d174e)

Conflicts:
	libavcodec/libvorbis.c

Fixes a part of Bug 277

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-10-18 10:27:50 +02:00
Ronald S. Bultje
79fb7bc667 h264: fix deadlocks on incomplete reference frame decoding.
If decoding a second complementary field, and the first was
decoded in our thread, mark decoding of that field as complete.
If decoding fails, mark the decoded field/frame as complete.
Do not allow switching between field modes or field/frame mode
between slices within the same field/frame. Ensure that two
subsequent fields cover top/bottom (rather than top/frame,
bottom/frame or such nonsense situations).

Fixes various deadlocks when decoding samples with errors in
reference frames.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 1e26a48fa23ef8e1cbc424667d387184d8155f15)

Fixes Bug 118

Conflicts:
	libavcodec/h264.c

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-10-18 10:00:14 +02:00
Anton Khirnov
141d4ed6c0 cmdutils: avoid setting data pointers to invalid values in alloc_buffer()
Fixes bug 352.
(cherry picked from commit 990450c5bf17afc31a81d6225afaac86d0dca5dd)

Conflicts:
	cmdutils.c

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-10-18 07:41:22 +02:00
Anton Khirnov
5acd1c6561 avidec: return 0, not packet size from read_packet().
(cherry picked from commit eeade678f0a2bac127aeed2fb68d8717a6463420)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-10-18 07:41:21 +02:00
Sean McGovern
a2d4d9f4fb wmapro: prevent division by zero when sample rate is unspecified
This fixes Bugzilla #327:

Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>
(cherry picked from commit 3680b2435101a5de56821718a71c828320d535a0)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-10-18 07:41:21 +02:00
Michael Niedermayer
3c55bf1201 vc1dec: check that coded slice positions and interlacing match.
This fixes out of array writes.

Addresses: CVE-2012-2796

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>
(cherry picked from commit 1100acbab26883007898c53efeb289f562c6e514)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-17 21:35:25 +02:00
Thilo Borgmann
dc5283dffc alsdec: fix number of decoded samples in first sub-block in BGMC mode.
Fixes CVE-2012-2790

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 66197988b1ee914825afbc3084e6da63f862068a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-17 21:31:21 +02:00
Mans Rullgard
c28e1c12ad alsdec: remove dead assignments
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 4ca6d206d1b5beea42c4290d2ee801aaf5cd31f0)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-17 21:31:01 +02:00
Thilo Borgmann
c5f9c272e9 alsdec: Fix out of ltp_gain_values read.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 97f0efbfb86d24f081b2caa39f6249e05c95c2ef)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-17 21:27:02 +02:00
Michael Niedermayer
0f81057c12 alsdec: Check that quantized parcor coeffs are within range.
ALS spec:
	11.6.3.1.1 Quantization and encoding of parcor coefficients
	...
	In all cases the resulting quantized values ak are restricted to the range [-64,63].

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 5b051ec3bdc78f3d89e8d1425674cde8fd6c9ccc)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-17 21:26:46 +02:00
Michael Niedermayer
592ba67815 alsdec: Check k used for rice decoder.
Values that fail this check will cause failure of decode_rice()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 23aae62c2cb4504a09ceb8cd0cabc1c8b260f521)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-17 21:26:15 +02:00
Anton Khirnov
15c2e8027f wav: do not fail on empty INFO tags
Fixes Bug 379

CC: libav-stable@libav.org
2012-10-16 20:20:55 +02:00
Michael Niedermayer
2051adbfa0 cavsdec: check for changing w/h.
Our decoder does not support changing w/h.

Fixes CVE-2012-2777 and CVE-2012-2784.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit c20a69630619d14ae92c5541d52c579d7c8f3e94)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:24 -04:00
Michael Niedermayer
2bc1e4fcb9 indeo4: update AVCodecContext width/height on size change
Fixes CVE-2012-2787

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit b146d74730ab9ec5abede9066f770ad851e45fbc)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:24 -04:00
Anton Khirnov
0582b8e3ea avidec: use actually read size instead of requested size
Fixes CVE-2012-2788
(cherry picked from commit 0af49a63c7f87876486ab09482d5b26b95abce60)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:24 -04:00
Michael Niedermayer
6744eee1e5 wmaprodec: check num_vec_coeffs for validity
Fixes CVE-2012-2789

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 99f392a584dd10b553facc8e819f2c7e982e176d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:24 -04:00
Michael Niedermayer
14bba214fa lagarith: check count before writing zeros.
Fixes CVE-2012-2793

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit b631e4ed64f7d1b9ca8f897fda31140e8d1fad81)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:24 -04:00
Anton Khirnov
1c8e2561b4 indeo3: fix out of cell write.
Fixes CVE-2012-2776.

CC:libav-stable@libav.org

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit e4d4044339b9c3b0f45f7203cd026eda3c0414c0)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:24 -04:00
Michael Niedermayer
5c413648c1 indeo5: check tile size in decode_mb_info().
This prevents writing into a too small array if some parameters changed
without the tile being reallocated.

Fixes CVE-2012-2794

CC:libav-stable@libav.org

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 2d09cdbaf2f449ba23d54e97e94bd97ca22208c6)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:24 -04:00
Janne Grunau
3efe6becc7 indeo5: prevent null pointer dereference on broken files
Found by John Villamil <johnv@matasano.com>
(cherry picked from commit 366ac22ea5a8bab63c7f46cdad2ddb2ff22cdbed)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:24 -04:00
Michael Niedermayer
dc8371b2b1 indeo5dec: Make sure we have had a valid gop header.
This prevents decoding happening on a half initialized context.

Fixes CVE-2012-2779

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 891918431db628db17885ed947ee387b29826a64)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:24 -04:00
Anton Khirnov
0815d9174c indeo4/5: check empty tile size in decode_mb_info().
This prevents writing into a too small array if some parameters changed
without the tile being reallocated.

Based on a patch by Michael Niedermayer <michaelni@gmx.at>

Fixes CVE-2012-2800

CC:libav-stable@libav.org

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit ae3da0ae5550053583a6f281ea7fd940497ea0d1)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:24 -04:00
Anton Khirnov
332555f660 ivi_common: make ff_ivi_process_empty_tile() static.
It's not used outside of ivi_common.c
(cherry picked from commit 5d2170c53bf4c2b0499f230c43764e4acf228f88)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:24 -04:00
Kostya Shishkov
c5ec190859 indeo: check for invalid motion vectors
(cherry picked from commit cf61aaaca16810b9b3a28395ed48fda8db0e87d9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:23 -04:00
Kostya Shishkov
b561618014 indeo: clear allocated band buffers
(cherry picked from commit 23ba1503f2b11057c65052b4a07961236d8d69c7)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:23 -04:00
Kostya Shishkov
e0daa15a96 indeo: track tile macroblock size
(cherry picked from commit a6e4ac40a62930d3c90f869990f96fedb9a5d654)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:23 -04:00
Kostya Shishkov
911c250aef factor out common decoding code for Indeo 4 and Indeo 5
(cherry picked from commit aa372cf4705343a9fff422ab9ead99cef7e0b415)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:23 -04:00
Kostya Shishkov
965302c9f3 indeo: check custom Huffman tables for errors
(cherry picked from commit fe7a37c36febd71576cbefc385d995a8d6e444e7)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:23 -04:00
Anton Khirnov
0c19855539 dfa: improve boundary checks in decode_dds1()
Fixes CVE-2012-2798

CC:libav-stable@libav.org
(cherry picked from commit d05f72c75445969cd7bdb1d860635c9880c67fb6)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:23 -04:00
Kostya Shishkov
d0267ecf76 dfa: use more meaningful return codes
(cherry picked from commit fb5c1aaea60a714dab3d4e6e71228855fd816222)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:20 -04:00
Kostya Shishkov
2281ac9ffd dfa: add some checks to ensure that decoder won't write past frame end
(cherry picked from commit 8099187e897ddc90cb3902332c76fb2542dac308)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 16:03:18 -04:00
Paul B Mahol
12941dbe2c dfa: convert to bytestream2 API
Protects from overreads.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 29b0d94b43ac960cb442049a5d737a3386ff0337)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 15:33:38 -04:00
Anton Khirnov
9e575e54a0 dfa: check that the caller set width/height properly.
Fixes CVE-2012-2786.
(cherry picked from commit ee715f49a06bf3898246d01b056284a9bb1bcbb9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 15:31:46 -04:00
Michael Niedermayer
9a76b7375e avsdec: Set dimensions instead of relying on the demuxer.
The decode function assumes that the video will have those dimensions.

Fixes CVE-2012-2801

CC:libav-stable@libav.org

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 85f477935cd6b34e6ec2716b20e15ce748277a89)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 15:29:00 -04:00
Justin Ruggles
d7de11260b ac3dec: ensure get_buffer() gets a buffer for the correct number of channels
If there is an error during frame parsing, but AVCodecContext.channels was
changed and AC3DecodeContext.out_channels was set previously, the two may not
match.

Fixes CVE-2012-2802
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

CC: libav-stable@libav.org
(cherry picked from commit 56b6a43056235fc110a018678da590595734203d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-14 15:27:39 -04:00
Martin Storsjö
31bc3fb563 snow: Check mallocs at init
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 4d8516fdb15d0177ad745228508254dee187dff9)

Conflicts:

	libavcodec/snow.c
2012-10-06 08:43:43 +02:00
Alex Converse
9aaaeba45c vorbis: Validate that the floor 1 X values contain no duplicates.
Duplicate values in this vector are explicitly banned by the Vorbis I spec
and cause divide-by-zero crashes later on.
(cherry picked from commit ecf79c4d3e8baaf2f303278ef81db6f8407656bc)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-06 08:43:43 +02:00
Justin Ruggles
e46cf805b1 vorbisenc: check all allocations for failure
(cherry picked from commit be8d812c9635f31f69c30dff9ebf565a07a7dab7)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-10-06 08:43:43 +02:00
Kostya Shishkov
d4f3abca6a indeo3: validate new frame size before resetting decoder
(cherry picked from commit 6de226a2b8b703abc823f18c3fd7f39a0787aeb5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-06 08:43:43 +02:00
Mina Nagy Zaki
e5f4e24942 lavfi: avfilter_merge_formats: handle case where inputs are same
This fixes a double-free crash if lists are the same due to the two
merge_ref() calls at the end of the (useless) merging that happens.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 11b6a82412bcd372adf694a26d83b07d337e1325)

Conflicts:

	libavfilter/formats.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-10-06 08:43:43 +02:00
Janne Grunau
b1ad5a21da rv34: error out on size changes with frame threading
Fixes CVE-2012-2772

(cherry picked from commit cb7190cd2c691fd93e4d3664f3fce6c19ee001dd)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-10-06 08:43:42 +02:00
Janne Grunau
90575bd7dd rv34: Handle only complete frames in frame-mt.
Correct handling of errors to prevent hags or crashes is very complex
otherwise.

The frame initializing is also moved from decode_slice() to
decode_frame() for clarity.
(cherry picked from commit 73ad4471a48bd02b2c2a55de116161b87e061023)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-10-06 08:43:22 +02:00
Janne Grunau
f695bd6016 rv34: use AVERROR return values in ff_rv34_decode_frame()
Also adds an error message.
(cherry picked from commit 29330721b0e8514f9f8b4d54be75a662a2b79e44)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-09-29 09:34:19 +02:00
Martin Storsjö
8c0bbe5156 vlc/rl: Add ff_ prefix to the nonstatic symbols
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit e96b4a53df101403c54e329abfadad2edddc47c4)

Conflicts:
	libavcodec/4xm.c

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-09-29 09:34:19 +02:00