22711 Commits

Author SHA1 Message Date
Michael Niedermayer
3ef1538121 h264: check the pixel format directly and force a reinit on mismatches.
The existing checks are insufficient to detect a pixel format
changes in case of some damaged streams.
Fixes inconsistency and later out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 11c99c78bafa77f679a1a3ba06ad00984b9a4cae)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-05 16:50:52 +01:00
Michael Niedermayer
47e462eecc aacdec: check channel count
Prevent out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 96f452ac647dae33c53c242ef3266b65a9beafb6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-05 16:48:04 +01:00
Michael Niedermayer
f3d1670606 vqavideo: check chunk sizes before reading chunks
Fixes out of array writes

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ab6c9332bfa1e20127a16392a0b85a4aa4840889)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-05 16:47:59 +01:00
Michael Niedermayer
9547034f91 gifdec: gif_copy_img_rect: Fix end pointer
Fixes out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c10350358da58600884292c08a8690289b81de29)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-05 16:47:53 +01:00
Michael Niedermayer
62c9beda0c sanm: Check decoded_size.
This prevents a buffer overflow in rle_decode()

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7357ca900efcf829de4cce4cec6ddc286526d417)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-05 16:47:41 +01:00
Diego Biurrun
0e68b6ddce Use proper "" quotes for local header #includes
(cherry picked from commit 6c1a7d07eb41b4cf36e48ebb560fecd9504fd968)

Conflicts:

	libavcodec/kbdwin.c
2013-02-05 16:35:28 +01:00
Michael Niedermayer
75e88db330 huffyuvdec: Skip len==0 cases
Fixes vlc decoding for hypothetical files that would contain such cases.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0dfc01c2bbf4b71bb56201bc4a393321e15d1b31)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-05 16:33:37 +01:00
Michael Niedermayer
6baa549249 huffyuvdec: Check init_vlc() return codes.
Prevents out of array writes

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f67a0d115254461649470452058fa3c28c0df294)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-05 16:33:37 +01:00
Piotr Bandurski
22561bc0e9 aasc: fix 16bpp on big-endian
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-05 16:33:37 +01:00
Michael Niedermayer
8a4464514f Merge remote-tracking branch 'qatar/release/9' into release/1.1
* qatar/release/9:
  arm: vp8: Fix the plain-armv6 version of vp8_luma_dc_wht
  Prepare for 9.2 Release
  lavr: call mix_function_init() in ff_audio_mix_set_matrix()
  rtpenc_chain: Use the original AVFormatContext for getting payload type
  rtp: Make sure the output format pointer is set

Conflicts:
	RELEASE

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-05 16:28:36 +01:00
Michael Niedermayer
85e94a30ee Merge commit '62de693a17f9b107be7867d822d5accacd4be544' into release/1.1
* commit '62de693a17f9b107be7867d822d5accacd4be544':
  rtp: Make sure priv_data is set before reading it
  videodsp_armv5te: remove #if HAVE_ARMV5TE_EXTERNAL
  get_bits: change the failure condition in init_get_bits
  mpegvideo: fix loop condition in draw_line()

Conflicts:
	libavcodec/get_bits.h
	libavcodec/mpegvideo.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-05 16:19:17 +01:00
Martin Storsjö
3d67f52f9d arm: vp8: Fix the plain-armv6 version of vp8_luma_dc_wht
This makes the plain-armv6 version use the same registers as the
armv6t2 version above.

This fixes fate-vp8 on plain-armv6 devices.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 2026eb1408a718c37835eb4b258c63714ab3205e)

Signed-off-by: Martin Storsjö <martin@martin.st>
2013-01-28 22:57:07 +02:00
Michael Niedermayer
f4fb841ad1 sanm: check image dimensions before using them
Avoids integer overflows and out of array accesses.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 49b729d3af8464de431362e6c5b3027102bc2f88)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-28 01:57:39 +01:00
Michael Niedermayer
59f7d583a3 mpeg1enc: Disable threads for resolutions too large for multi-threading
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0c6b0409af070a3bfb02b55fde8ba18219edc76b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-28 01:57:39 +01:00
Paul B Mahol
c2d2bf1d6b lavc/iff: ilbm: unbreak decoding on big endian
Fixes ticket #2192.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit 25c75525bf1da38179ec67924f0be7a2bd8faa0d)
2013-01-26 15:10:02 +01:00
Michael Karcher
302094e1d2 Fix atrac3 decoder broken in e55d53905f34f8e8747f6d321e9a695dc02ebb2f
Signed-off-by: Michael Karcher <ffmpeg@mkarcher.dialup.fu-berlin.de>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dcbb920f1587d1fce777aae947a49304665436b5)
2013-01-26 03:36:18 +01:00
Janne Grunau
1a28948eb3 videodsp_armv5te: remove #if HAVE_ARMV5TE_EXTERNAL
libavutil/arm/asm.S sets '.arch' depending on HAVE_ARMV5TE so that
assembling armv5te code will always succeed even if the default -march
flag does not support it. HAVE_ARMV5TE_EXTERNAL tests assembling code
with the default arch.
Fixes the missing symbol ff_prefetch_arm with --cpu= not including
armv5te.

CC: libav-stable@libav.org
2013-01-22 13:43:16 +01:00
Luca Barbato
01050448cf get_bits: change the failure condition in init_get_bits
Too much code relies in having init_get_bits fed with a valid
buffer and set its dimension to 0.

Check for NULL buffer instead.
(cherry picked from commit 4603ec85ed620e585fc6e2e072c99858ed421855)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-01-20 14:06:52 +01:00
Xi Wang
8d0631c8fa mpegvideo: fix loop condition in draw_line()
The loop condition `x = ex' is incorrect.  It should be `x <= ex'.

This bug was introduced in commit c65dfac4 "mpegvideo.c: K&R formatting
and cosmetics."

CC:libav-stable@libav.org

(cherry picked from commit 992b03183819553a73b4f870a710ef500b4eb6d0)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-01-19 22:21:23 +01:00
Michael Niedermayer
1135928903 init_get_bits: fix off by 1 error
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7980cca05c7c72fc8b0be4268eea2e156e538228)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-19 17:59:23 +01:00
Michael Niedermayer
6f3bc92c29 init_get_bits8: zero pointers & struct on error
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 153fad14e5a2f85637aa6c254ced0fc1c68974e2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-19 17:59:22 +01:00
Michael Niedermayer
bd531038e8 init_get_bits8: check byte_size against being positive
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ac73d3a12a33c5e4e6357d5f8824e19801663eb5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-19 17:59:21 +01:00
Michael Niedermayer
43c6b45a53 avcodec_decode_audio: do not trust the channel layout, use the channel count.
Fixes memory corruption

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d270c3202539e8364c46410e15f7570800e33343)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-19 16:18:08 +01:00
Michael Niedermayer
68a0477bc0 error_concealment: Check that the picture is not in a half setup state.
Fixes state becoming inconsistent
Fixes a null pointer dereference

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 23318a57358358e7a4dc551e830e4503f0638cfe)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-19 16:18:01 +01:00
Paul B Mahol
ccf0cd967d 012v: remove double ; and return correct error code if ff_get_buffer() fails
Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit 25160236957647d81e8beecd6c8fb7f1949fc26e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-19 16:17:24 +01:00
Michael Niedermayer
002ad7cd39 Merge remote-tracking branch 'qatar/release/9' into release/1.1
* qatar/release/9:
  fate: update ref after rv30_loop_filter fix
  rv30: fix masking in rv30_loop_filter()
  libcdio: support recent cdio-paranoia
  theora: Skip zero-sized headers
  h264: add 3 pixels below for subpixel filter wait position
  h264: fix ff_generate_sliding_window_mmcos() prototype.
  h264: don't clobber mmco opcode tables for non-first slice headers.

Conflicts:
	configure
	libavcodec/h264_refs.c
	tests/ref/fate/filter-delogo
	tests/ref/fate/rv30

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-19 15:54:36 +01:00
Michael Niedermayer
30f0cd2f1e h264: fix () placement
Fixes null pointer dereference

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c13e4e288c84ba0629ead15e1460c0e498ee2bce)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-18 05:14:31 +01:00
Michael Niedermayer
2338eda8d8 tiff: Fix assignments in if()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 659546b42d6550e67fcdbb4937cd1982c60448aa)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-18 05:14:31 +01:00
Michael Niedermayer
6a0633e961 svq1enc: Fix assignments in if()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 37be1d802f2e2c21036a54bb15423a41d5aabefb)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-18 05:14:31 +01:00
Michael Niedermayer
16dc41de27 ra144enc: Fix assignments in if()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e2704381e5f13f54506f69b7a05a05dc27ce1d7d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-18 05:14:31 +01:00
Michael Niedermayer
ab471e17e4 nellymoserenc: Fix assignments in if()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 795d2dc23b16a678d60a681e906aa87c14478597)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-18 05:14:31 +01:00
Michael Niedermayer
3be8aeb14e libvorbisenc: Fix assignments in if()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bdd71abe5f34ca37612e17d912060f4dc9b94796)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-18 05:14:31 +01:00
Michael Niedermayer
b48e251360 libvo-aacenc: Fix assignments in if()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0ccb31dcad5a1543fbb284d66b0410b91ebd171d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-18 05:14:31 +01:00
Michael Niedermayer
65a4b90840 libspeexenc: Fix assignments in if()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3b8d66d5317d91288751869206b3acbb84dc44c7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-18 05:14:31 +01:00
Michael Niedermayer
59956a5957 libopencore-amr: Fix assignments in if()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d6180aa29741334cf69f691b27ffceb33f49d36a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-18 05:14:31 +01:00
Michael Niedermayer
d4a08e560d libmp3lame: Fix assignments in if()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 871b6ec01d27a74702b7cf1d61446709de037948)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-18 05:14:31 +01:00
Michael Niedermayer
dacac91973 libfdk-aacenc: Fix assignments in if()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9302ad1ac89d5443505cf0418f9d62786513032e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-18 05:14:30 +01:00
Michael Niedermayer
d39400fed7 libfaac: Fix assignments in if()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 68a25c64cda16e12ef3a051ff8661c71ef574683)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-18 05:14:30 +01:00
Michael Niedermayer
07174ed841 aacenc: Fix assignments in if()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 98fed59427cec17ce55ac137e7e250cff7db81cf)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-18 05:14:30 +01:00
Michael Niedermayer
cc8ab98656 mpeg12enc: check dimension validity
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-18 05:14:30 +01:00
Michael Niedermayer
d7cff9f8e8 mpeg12enc: Correctly mask dimensions
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-18 05:14:30 +01:00
Michael Niedermayer
9bfda9df71 mpeg12: Support decoding dimensions that are a multiple of 4096
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-18 05:14:30 +01:00
Xi Wang
c3c1db7c56 rv30: fix masking in rv30_loop_filter()
The mask `x && (1 << y)' is incorrect and always yields true.

The correct form should be `x & (1 << y)'.

CC: libav-stable@libav.org

Signed-off-by: Xi Wang <xi.wang@gmail.com>
(cherry picked from commit 783e37f7ef3b3cdcfe7aa927a25b4184ae46cd53)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-01-17 22:00:01 +01:00
Martin Storsjö
c749bec8c3 theora: Skip zero-sized headers
This fixes a regression since d9cf5f51/7a2ee770f5 with theora
over RTP (possibly with other variants of theora as well).

In theora over RTP, the second of the 3 headers turns out to be
0 bytes long, which prior to d9cf5f51 worked just fine. After
d9cf5f51, reading from the bitstream reader fails (since the reader
wasn't initialized but returned an error if initialized with 0 bits).

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit e33db35b4a91ad543d9dde3a981a89118ba68937)

Signed-off-by: Martin Storsjö <martin@martin.st>
2013-01-17 19:11:54 +02:00
Carl Eugen Hoyos
ed12d1ecad Fix compilation with --disable-everything.
(cherry picked from commit f023003ce610a8fd6377cf4a8e98002ac3117ef4)
2013-01-17 17:39:00 +01:00
Michael Niedermayer
76477c3843 Merge remote-tracking branch 'qatar/release/9' into release/1.1
* qatar/release/9:
  libx264: use the library specific default rc_initial_buffer_occupancy
  lavc: set the default rc_initial_buffer_occupancy
  lavc: introduce the convenience function init_get_bits8
  lavc: check for overflow in init_get_bits
  APIchanges: Fill in missing hashes and dates; fix a version number typo.
  configure: enable pic for shared libs on AArch64
  zmbv: Reset the decoder on keyframe errors
  vc1dec: prevent a crash due missing pred_flag parameter
  matroska: Fix use after free
  vp3: Fix double free in vp3_decode_end()
  update Changelog
  oggdec: make sure the private parse data is cleaned up
  oggdec: free the ogg streams on read_header failure
  update Changelog
  x86: lavr: use the x86inc.asm automatic stack alignment in mixing functions
  Prepare 9.1 Release

Conflicts:
	Changelog
	RELEASE
	doc/APIchanges
	libavcodec/utils.c
	libavformat/oggdec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-17 04:26:42 +01:00
Ronald S. Bultje
9d60f608af h264: add 3 pixels below for subpixel filter wait position
If the motion vector is at a subpixel position, we need 3 pixels below
the motion vector's wholepel position available, not 2, since the MC
filter is a sixtap filter for the hpel position, and then a bilin filter
for the qpel position.

This patch fixes highly irreproducible (0.1%) fate failures in frame 2
and 4 of h264-conformance-cama2_vtc_b (e.g. first P-frame, first field,
last line of MB x=40,y=2 and second field and last lines of MBs x=39-40,
y=3). These used pre-loopfilter instead of post-loopfilter data because
the await_progress() waited for one line too little in that field, and
the motion vector of these particular MBs happened to align exactly to a
position where that demonstrates the bug.

CC: libav-stable@libav.org

(cherry picked from commit fb845ffdd335a1efd6dfd43e8adeb530397b348e)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-01-15 22:20:10 +01:00
Anton Khirnov
6a4803a6a9 h264: fix ff_generate_sliding_window_mmcos() prototype.
It's been returning an error value since
bad446e251405dc250c3cbee199072e083a1e4b9

Also check for the errors it returns.
(cherry picked from commit ea382767ad2191acbe97e90624059723e15f0e4b)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-01-15 22:19:42 +01:00
Ronald S. Bultje
c3b67720f9 h264: don't clobber mmco opcode tables for non-first slice headers.
Clobbering these tables will temporarily clobber the template used
as a basis for other threads to start decoding from. If the other
decoding thread updates from the template right at that moment,
subsequent threads will get invalid (or, usually, none at all) mmco
tables. This leads to invalid reference lists and subsequent decode
failures.

Therefore, instead, decode the mmco tables only for the first slice in
a field or frame. For other slices, decode the bits and ensure they
are identical to the mmco tables in the first slice, but don't ever
clobber the context state. This prevents other threads from using a
clobbered/invalid template as starting point for decoding, and thus
fixes decoding in these cases.

This fixes occasional (~1%) failures of h264-conformance-mr1_bt_a with
frame-multithreading enabled.

(cherry picked from commit bad446e251405dc250c3cbee199072e083a1e4b9)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-01-15 22:19:07 +01:00
Michael Niedermayer
9636266cbd vorbisdec: handle midstream parameter changes
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e9ffee23f372390684d6d94bf8abf51350fc66a3)
2013-01-15 21:12:03 +01:00