15017 Commits

Author SHA1 Message Date
Ronald S. Bultje
d680295d0c huffyuv: do not abort on unknown pix_fmt; instead, return an error.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 63c9de6469005974288f4e4d89fc79a590e38c06)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:27 +02:00
Ronald S. Bultje
ced190c96c vmnc: return error on decode_init() failure.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 07a180972fb369bb59bf6d4f8edb4598c51e80d2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:27 +02:00
Ronald S. Bultje
e15d137ecf rpza: error out on buffer overreads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 78e9852a2e3b198ecd69ffa0deab3fa22a8e5378)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:27 +02:00
Ronald S. Bultje
87a1169ab8 qtrle: return error on decode_init() failure.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit e54ae60e46f737b8e9a96548971091f7ab6b8f7c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:27 +02:00
Ronald S. Bultje
f28ec73379 vp56: error out on invalid stream dimensions.
Prevents crashes when playing corrupt vp5/6 streams.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 8bc396fc0e8769a056375c1c211f389ce0e3ecc5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:27 +02:00
Ronald S. Bultje
117b8b00cc vorbis: fix overflows in floor1[] vector and inverse db table index.
(cherry picked from commit 24947d4988012f1f0fd467c83418615adc11c3e8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:27 +02:00
Reinhard Tartler
a02da9ceaf Fix parser not to clobber has_b_frames when extradata is set.
Because in contrast to the decoder, the parser does not setup low_delay.
The code in parse_nal_units would always end up setting has_b_frames
to "1", except when stream is explicitly marked as low delay.
Since the parser itself would create 'extradata', simply reopening
the parser would cause this.

This happens for instance in estimate_timings_from_pts(), which causes the
parser to be reopened on the same stream.

This fixes Libav #22 and FFmpeg (trac) #360

CC: libav-stable@libav.org

Based on a patch by Reimar Döffinger <Reimar.Doeffinger@gmx.de>
(commit 31ac0ac29b6bba744493f7d1040757a3f51b9ad7)

Comments and description adapted by Reinhard Tartler.

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 790a367d9ecd04360f78616765ee723f3fe65645)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:27 +02:00
Ronald S. Bultje
678737c26b fraps: release reference buffer on pix_fmt change.
Prevents crash when trying to copy from a non-existing plane in e.g.
a RGB32 reference image to a YUV420P target image

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 830f70442a87a31f7c75565e9380e3caf8333b8a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:27 +02:00
Ronald S. Bultje
25784c0409 kgv1: release reference picture on size change.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 6c4c27adb61b2881a94ce5c7d97ee1c8adadb5fe)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:27 +02:00
Ronald S. Bultje
d10c22d33c lcl: error out if uncompressed input buffer is smaller than framesize.
This prevents crashes when trying to read beyond the end of the buffer
while decoding frame data.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit be129271eac04f91393bf42a490ec631e1a9abea)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:27 +02:00
Alex Converse
b1d9a80863 tiff: Prevent overreads in the type_sizes array.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 447363870f2f91e125e07ac2d0820359a5d86b06)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:27 +02:00
Ronald S. Bultje
18b2f23ef8 truemotion2: error out if the huffman tree has no nodes.
This prevents crashers and errors further down when reading nodes in the
empty tree.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 2b83e8b7005d531bc78b0fd4f699e9faa54ce9bb)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:26 +02:00
Ronald S. Bultje
33149928ed mjpegb: don't return 0 at the end of frame decoding.
Return 0 indicates "please return the same data again", i.e. it causes
an infinite loop. Instead, return that we consumed the buffer if we
finished decoding succesfully, or return an error if an error occurred.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 74699ac8c8b562e9f8d26e21482b89585365774a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:26 +02:00
Ronald S. Bultje
2380a3d37f huffyuv: error out on bit overrun.
On EOF, get_bits() will continuously return 0, causing an infinite
loop.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 84c202cc37024bd78261e4222e46631ea73c48dd)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:26 +02:00
Ronald S. Bultje
4509129e9d als: prevent infinite loop in zero_remaining().
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit af468015d972c0dec5c8c37b2685ffa5cbe4ae87)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:26 +02:00
Ronald S. Bultje
d031302e0e cook: prevent div-by-zero if channels is zero.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 941fc1ea1ed7f7d99a8b9e2607b41f2f2820394a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:26 +02:00
Alex Converse
fd3af2950a smacker: Sanity check huffman tables found in the headers.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

CC: libav-stable@libav.org
(cherry picked from commit 9adf25c1cf78dbf1d71bf386c49dc74cb8a60df0)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:26 +02:00
Alex Converse
bf9f26cef7 aac: fix infinite loop on end-of-frame with sequence of 1-bits.
Based-on-work-by: Ronald S. Bultje <rsbultje@gmail.com>
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 1cd9a6154bc1ac1193c703cea980ed21c3e53792)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:26 +02:00
Alex Converse
0fbde741cb wma: Clip WMA1 and WMA2 frame length to 11 bits.
The MDCT buffers in the decoder are only sized for up to 11 bits. The
reverse engineered documentation for WMA1/2 headers say that that for
all samplerates above 32kHz 11 bits are used. 12 and 13 bit support
were added for WMAPro. I was unable to make any Microsoft tools generate
a test file at a samplerate above 48kHz.

Discovered by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

CC: libav-stable@libav.org
(cherry picked from commit d78bb1a4b2a3a415b68e4e6dd448779eccec64e3)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:26 +02:00
Ronald S. Bultje
ec961c8919 flac: fix infinite loops on all-zero input or end-of-stream.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 52e4018be47697a60f4f18f83551766df31f5adf)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:26 +02:00
Ronald S. Bultje
3b5e1494c6 golomb: avoid infinite loop on all-zero input (or end of buffer).
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit c6643fddba73560f26f90d327c84d8832222a720)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:26 +02:00
Alex Converse
ccd528cc32 qdm2: Check data block size for bytes to bits overflow.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

CC: libav-stable@libav.org
(cherry picked from commit dac56d9ce01eb9963f28f26b97a81db5cbd46c1c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:26 +02:00
Martin Storsjö
ceeaf42451 avcodec: Remove a misplaced and useless attribute_deprecated
If attribute_deprecated is used in an enum declaration, it
should follow the 'enum' keyword, otherwise it's ignored
silently. This is the only case of attribute_deprecated for
enum declarations currently.

Currently, this attribute_deprecated doesn't have any effect.
If moved to the right place, it emits a warning every single
time avcodec.h is included, like this:

avcodec.h:2827: warning: ‘AVLPCType’ is deprecated (declared at avcodec.h:543)

There is already a working attribute_deprecated for the
corresponding field in AVCodecContext, so therefore this
one shouldn't be needed.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 1b6da627d49e98fe7661c9aa9ec4e16ab04dfda4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-26 09:41:15 +02:00
Martin Storsjö
c321f2abca avcodec: Remove a misplaced and useless attribute_deprecated
If attribute_deprecated is used in an enum declaration, it
should follow the 'enum' keyword, otherwise it's ignored
silently. This is the only case of attribute_deprecated for
enum declarations currently.

Currently, this attribute_deprecated doesn't have any effect.
If moved to the right place, it emits a warning every single
time avcodec.h is included, like this:

avcodec.h:2827: warning: ‘AVLPCType’ is deprecated (declared at avcodec.h:543)

There is already a working attribute_deprecated for the
corresponding field in AVCodecContext, so therefore this
one shouldn't be needed.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 1b6da627d49e98fe7661c9aa9ec4e16ab04dfda4)

Signed-off-by: Martin Storsjö <martin@martin.st>
2012-03-23 11:59:07 +02:00
Michael Niedermayer
a3d331f2d8 Merge remote-tracking branch 'qatar/release/0.7' into release/0.8
* qatar/release/0.7: (96 commits)
  intfloat_readwrite: fix signed addition overflows
  smacker: validate channels and sample format.
  smacker: check buffer size before reading output size
  smacker: validate number of channels
  sipr: fix get_bits(0) calls
  motion_est: make MotionExtContext.map_generation unsigned
  4xm: prevent NULL dereference with invalid huffman table
  4xmdemux: prevent use of uninitialized memory
  4xm: clear FF_INPUT_BUFFER_PADDING_SIZE bytes in temporary buffers
  ptx: check for out of bound reads
  tiffdec: fix out of bound reads/writes
  eacmv: check for out of bound reads
  eacmv: fix potential pointer arithmetic overflows
  adpcm: fix out of bound reads due to integer overflow
  anm: prevent infinite loop
  avsdemux: check for out of bound writes
  avs: check for out of bound reads
  avsdemux: check for corrupted data
  mxfdec: Fix some buffer overreads caused by the misuse of AVPacket related functions.
  vaapi: Fix VC-1 decoding (reconstruct bitstream TTFRM correctly).
  ...

Conflicts:
	libavcodec/adpcm.c
	libavcodec/bink.c
	libavcodec/h264.c
	libavcodec/h264.h
	libavcodec/h264_cabac.c
	libavcodec/h264_cavlc.c
	libavcodec/motion_est_template.c
	libavcodec/mpegvideo.c
	libavcodec/nellymoserdec.c
	libavcodec/ptx.c
	libavcodec/svq3.c
	libavcodec/vaapi_vc1.c
	libavcodec/xan.c
	libavfilter/vf_scale.c
	libavformat/4xm.c
	libavformat/flvdec.c
	libavformat/mpeg.c
	tests/ref/fate/motionpixels

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-19 05:14:44 +01:00
Justin Ruggles
1cc0b08635 smacker: validate channels and sample format.
(cherry picked from commit ff1f89de2da3472d133e2c95bf7c9ad2d88df33d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:46 +01:00
Justin Ruggles
b3d7fffee3 smacker: check buffer size before reading output size
(cherry picked from commit cf044f8bff0d28dbc34492f18b0d18b3ba8bad9d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:43 +01:00
Justin Ruggles
ef7a4df458 smacker: validate number of channels
(cherry picked from commit e190e453bd1e4d4b409ed3556b3a50d1087c15d7)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:41 +01:00
Mans Rullgard
3b7a1ba90e sipr: fix get_bits(0) calls
Zero-length get_bits() is undefined, must check before calling.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit c79d2a20bad59298188171f1316a830d563a41ee)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:41 +01:00
Mans Rullgard
da73a2005a motion_est: make MotionExtContext.map_generation unsigned
The way this value is used, it should be an unsigned type.
While the numerical value has no meaning, unsigned wraparound
is relied upon.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit cb668476ab1343d27e03edc0b32f57ca7a187471)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:41 +01:00
Laurent Aimar
6b011631e9 4xm: prevent NULL dereference with invalid huffman table
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 1b1182ce97db7a97914bb7713eba66fee5d93937)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:41 +01:00
Laurent Aimar
39fd8d0083 4xm: clear FF_INPUT_BUFFER_PADDING_SIZE bytes in temporary buffers
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 8d518a9c4fe92e2497565f1765da7f913be8b1e7)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
b3bdefb01b ptx: check for out of bound reads
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit dc64f203a62083c3d5f81e8201018279c29581af)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
4eb51d96dd tiffdec: fix out of bound reads/writes
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 04a845caa7cdcdd1457f8c0dde52a7b2085ed92f)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
d75c80e942 eacmv: check for out of bound reads
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 46cb2f6a2928a7fa4bee3f09b0475ccb8cdd2064)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
34d6f22a57 eacmv: fix potential pointer arithmetic overflows
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 8df8a87e3fd5bd0c3dabc676aae8fd84992932dc)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
518c72474d adpcm: fix out of bound reads due to integer overflow
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit c7f89064e2f0fef8198aadf64b0daf12787404ee)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
39fed2e95b anm: prevent infinite loop
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 2475f1a83ccf313d828b25f1769e3a37442ecf64)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
ab201f6f1b avs: check for out of bound reads
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit de049a95f4a8089b2878c7fcef6cac7e88a8f1bf)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Gwenole Beauchesne
559261ce49 vaapi: Fix VC-1 decoding (reconstruct bitstream TTFRM correctly).
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 53efb758c045900f512c947074900c0dbc988685)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:36 +01:00
Mans Rullgard
f9d17e6f54 4xm: fix signed overflow
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 84dda407628e298f33d610e9e04a8b2945d24665)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:36 +01:00
Mans Rullgard
0b1ac7bf4f wmavoice: fix a signed overflow
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit ba3f07d0611d9a6c10eaa90b3c058ecdffe76676)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:36 +01:00
Mans Rullgard
af0a56e6ef mpegvideo_enc: fix a signed overflow
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 05795f35be4b479bfa8d60ed3eb13e0f89e439c0)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:36 +01:00
Mans Rullgard
48f9a80072 mpeg12enc: use sign_extend() function
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 2f329db90e5d72ad383a0ba05fde3641a34ef73b)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
fdc669fcbb vp8: fix signed overflows
In addition to avoiding undefined behaviour, an unsigned type
makes more sense for packing multiple 8-bit values.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit bb59156606e00057a706ed30165bc7329db3823f)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
fe3314a413 motion_est: fix some signed overflows
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit e708afd3c026a9eb547dab07781320a7e2564312)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
58afe6061a dca: fix signed overflow in shift
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 559c244d42be7a02c23976216b47fd63b80d6c7f)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
8c2ae575ad aacdec: fix undefined shifts
Since nnz can be zero, this is needed to avoid a shift by 32.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit d12294304acd82cb219e3f66ca9cd6efb2194fa4)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Laurent Aimar
9c78fe9360 bink: Check for various out of bound writes
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit a00676e48e49a3d794d6d2063ceca539e945a4a4)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Laurent Aimar
c98d7882d8 bink: Check for out of bound writes when building tree
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 24adf7832b8370f3c1febbef6c686f574d360d32)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00