15051 Commits

Author SHA1 Message Date
Alex Converse
fd3af2950a smacker: Sanity check huffman tables found in the headers.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

CC: libav-stable@libav.org
(cherry picked from commit 9adf25c1cf78dbf1d71bf386c49dc74cb8a60df0)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:26 +02:00
Alex Converse
bf9f26cef7 aac: fix infinite loop on end-of-frame with sequence of 1-bits.
Based-on-work-by: Ronald S. Bultje <rsbultje@gmail.com>
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 1cd9a6154bc1ac1193c703cea980ed21c3e53792)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:26 +02:00
Alex Converse
0fbde741cb wma: Clip WMA1 and WMA2 frame length to 11 bits.
The MDCT buffers in the decoder are only sized for up to 11 bits. The
reverse engineered documentation for WMA1/2 headers say that that for
all samplerates above 32kHz 11 bits are used. 12 and 13 bit support
were added for WMAPro. I was unable to make any Microsoft tools generate
a test file at a samplerate above 48kHz.

Discovered by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

CC: libav-stable@libav.org
(cherry picked from commit d78bb1a4b2a3a415b68e4e6dd448779eccec64e3)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:26 +02:00
Ronald S. Bultje
ec961c8919 flac: fix infinite loops on all-zero input or end-of-stream.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 52e4018be47697a60f4f18f83551766df31f5adf)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:26 +02:00
Ronald S. Bultje
3b5e1494c6 golomb: avoid infinite loop on all-zero input (or end of buffer).
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit c6643fddba73560f26f90d327c84d8832222a720)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:26 +02:00
Alex Converse
ccd528cc32 qdm2: Check data block size for bytes to bits overflow.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

CC: libav-stable@libav.org
(cherry picked from commit dac56d9ce01eb9963f28f26b97a81db5cbd46c1c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:26 +02:00
Martin Storsjö
ceeaf42451 avcodec: Remove a misplaced and useless attribute_deprecated
If attribute_deprecated is used in an enum declaration, it
should follow the 'enum' keyword, otherwise it's ignored
silently. This is the only case of attribute_deprecated for
enum declarations currently.

Currently, this attribute_deprecated doesn't have any effect.
If moved to the right place, it emits a warning every single
time avcodec.h is included, like this:

avcodec.h:2827: warning: ‘AVLPCType’ is deprecated (declared at avcodec.h:543)

There is already a working attribute_deprecated for the
corresponding field in AVCodecContext, so therefore this
one shouldn't be needed.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 1b6da627d49e98fe7661c9aa9ec4e16ab04dfda4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-26 09:41:15 +02:00
Martin Storsjö
c321f2abca avcodec: Remove a misplaced and useless attribute_deprecated
If attribute_deprecated is used in an enum declaration, it
should follow the 'enum' keyword, otherwise it's ignored
silently. This is the only case of attribute_deprecated for
enum declarations currently.

Currently, this attribute_deprecated doesn't have any effect.
If moved to the right place, it emits a warning every single
time avcodec.h is included, like this:

avcodec.h:2827: warning: ‘AVLPCType’ is deprecated (declared at avcodec.h:543)

There is already a working attribute_deprecated for the
corresponding field in AVCodecContext, so therefore this
one shouldn't be needed.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 1b6da627d49e98fe7661c9aa9ec4e16ab04dfda4)

Signed-off-by: Martin Storsjö <martin@martin.st>
2012-03-23 11:59:07 +02:00
Michael Niedermayer
a3d331f2d8 Merge remote-tracking branch 'qatar/release/0.7' into release/0.8
* qatar/release/0.7: (96 commits)
  intfloat_readwrite: fix signed addition overflows
  smacker: validate channels and sample format.
  smacker: check buffer size before reading output size
  smacker: validate number of channels
  sipr: fix get_bits(0) calls
  motion_est: make MotionExtContext.map_generation unsigned
  4xm: prevent NULL dereference with invalid huffman table
  4xmdemux: prevent use of uninitialized memory
  4xm: clear FF_INPUT_BUFFER_PADDING_SIZE bytes in temporary buffers
  ptx: check for out of bound reads
  tiffdec: fix out of bound reads/writes
  eacmv: check for out of bound reads
  eacmv: fix potential pointer arithmetic overflows
  adpcm: fix out of bound reads due to integer overflow
  anm: prevent infinite loop
  avsdemux: check for out of bound writes
  avs: check for out of bound reads
  avsdemux: check for corrupted data
  mxfdec: Fix some buffer overreads caused by the misuse of AVPacket related functions.
  vaapi: Fix VC-1 decoding (reconstruct bitstream TTFRM correctly).
  ...

Conflicts:
	libavcodec/adpcm.c
	libavcodec/bink.c
	libavcodec/h264.c
	libavcodec/h264.h
	libavcodec/h264_cabac.c
	libavcodec/h264_cavlc.c
	libavcodec/motion_est_template.c
	libavcodec/mpegvideo.c
	libavcodec/nellymoserdec.c
	libavcodec/ptx.c
	libavcodec/svq3.c
	libavcodec/vaapi_vc1.c
	libavcodec/xan.c
	libavfilter/vf_scale.c
	libavformat/4xm.c
	libavformat/flvdec.c
	libavformat/mpeg.c
	tests/ref/fate/motionpixels

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-19 05:14:44 +01:00
Justin Ruggles
1cc0b08635 smacker: validate channels and sample format.
(cherry picked from commit ff1f89de2da3472d133e2c95bf7c9ad2d88df33d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:46 +01:00
Justin Ruggles
b3d7fffee3 smacker: check buffer size before reading output size
(cherry picked from commit cf044f8bff0d28dbc34492f18b0d18b3ba8bad9d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:43 +01:00
Justin Ruggles
ef7a4df458 smacker: validate number of channels
(cherry picked from commit e190e453bd1e4d4b409ed3556b3a50d1087c15d7)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:41 +01:00
Mans Rullgard
3b7a1ba90e sipr: fix get_bits(0) calls
Zero-length get_bits() is undefined, must check before calling.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit c79d2a20bad59298188171f1316a830d563a41ee)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:41 +01:00
Mans Rullgard
da73a2005a motion_est: make MotionExtContext.map_generation unsigned
The way this value is used, it should be an unsigned type.
While the numerical value has no meaning, unsigned wraparound
is relied upon.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit cb668476ab1343d27e03edc0b32f57ca7a187471)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:41 +01:00
Laurent Aimar
6b011631e9 4xm: prevent NULL dereference with invalid huffman table
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 1b1182ce97db7a97914bb7713eba66fee5d93937)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:41 +01:00
Laurent Aimar
39fd8d0083 4xm: clear FF_INPUT_BUFFER_PADDING_SIZE bytes in temporary buffers
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 8d518a9c4fe92e2497565f1765da7f913be8b1e7)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
b3bdefb01b ptx: check for out of bound reads
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit dc64f203a62083c3d5f81e8201018279c29581af)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
4eb51d96dd tiffdec: fix out of bound reads/writes
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 04a845caa7cdcdd1457f8c0dde52a7b2085ed92f)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
d75c80e942 eacmv: check for out of bound reads
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 46cb2f6a2928a7fa4bee3f09b0475ccb8cdd2064)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
34d6f22a57 eacmv: fix potential pointer arithmetic overflows
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 8df8a87e3fd5bd0c3dabc676aae8fd84992932dc)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
518c72474d adpcm: fix out of bound reads due to integer overflow
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit c7f89064e2f0fef8198aadf64b0daf12787404ee)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
39fed2e95b anm: prevent infinite loop
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 2475f1a83ccf313d828b25f1769e3a37442ecf64)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
ab201f6f1b avs: check for out of bound reads
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit de049a95f4a8089b2878c7fcef6cac7e88a8f1bf)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Gwenole Beauchesne
559261ce49 vaapi: Fix VC-1 decoding (reconstruct bitstream TTFRM correctly).
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 53efb758c045900f512c947074900c0dbc988685)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:36 +01:00
Mans Rullgard
f9d17e6f54 4xm: fix signed overflow
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 84dda407628e298f33d610e9e04a8b2945d24665)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:36 +01:00
Mans Rullgard
0b1ac7bf4f wmavoice: fix a signed overflow
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit ba3f07d0611d9a6c10eaa90b3c058ecdffe76676)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:36 +01:00
Mans Rullgard
af0a56e6ef mpegvideo_enc: fix a signed overflow
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 05795f35be4b479bfa8d60ed3eb13e0f89e439c0)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:36 +01:00
Mans Rullgard
48f9a80072 mpeg12enc: use sign_extend() function
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 2f329db90e5d72ad383a0ba05fde3641a34ef73b)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
fdc669fcbb vp8: fix signed overflows
In addition to avoiding undefined behaviour, an unsigned type
makes more sense for packing multiple 8-bit values.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit bb59156606e00057a706ed30165bc7329db3823f)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
fe3314a413 motion_est: fix some signed overflows
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit e708afd3c026a9eb547dab07781320a7e2564312)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
58afe6061a dca: fix signed overflow in shift
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 559c244d42be7a02c23976216b47fd63b80d6c7f)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
8c2ae575ad aacdec: fix undefined shifts
Since nnz can be zero, this is needed to avoid a shift by 32.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit d12294304acd82cb219e3f66ca9cd6efb2194fa4)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Laurent Aimar
9c78fe9360 bink: Check for various out of bound writes
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit a00676e48e49a3d794d6d2063ceca539e945a4a4)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Laurent Aimar
c98d7882d8 bink: Check for out of bound writes when building tree
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 24adf7832b8370f3c1febbef6c686f574d360d32)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
e52e85ac3a put_bits: fix invalid shift by 32 in flush_put_bits()
If flush_put_bits() is called when the 32-bit buffer is empty,
e.g. after writing a multiple of 32 bits, and invalid shift by
32 is performed.  Since flush_put_bits() is called infrequently,
this additional check should have negligible performance impact.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit ac6eab1496aad6f8b09deabbef4fe5fd829e142d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Laurent Aimar
59050c0629 mpc8: Check out of bound bands limit
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 9bd854b1ff342f82efa6d2ad4e8fefddce5fa731)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
be2404b06d xan: Prevent NULL dereference with missing palette
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 7d17a794f0348ba40d5cda7d969564cb83981001)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
49007b494e xan: Check for out of bound reads in xan_huffman_decode()
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 3db3fdf4c669aed9379be430c17f151d4d0697c5)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
0277c82de2 xan: Fixed out of bound accesses in xan_unpack()
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 3e0757c2a87c8cf3e452f67bca279001c64cedff)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
5fa8e43b54 motionpixels: Prevent calling init_vlc() with invalid parameters
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 1cd0a5516396bd6fb54e4df1e7c88ed18416299b)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
737bea21b6 shorten: Fix out of bound writes in fix_bitshift()
The data pointers s->decoded[*] already take into account s->nwrap.

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 5f05cf4ea9aaafed8edcabe785c2719786103ec1)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
aa9e308580 dsicinav: Check for out of bounds writes
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 172060328771d149b076f00352b004b5b5272d38)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
d57d039e04 tiertexseqv: Check for out of bound reads
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 64263dd526ec25ede1591fc1144715a20cc7bc4e)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
97a1ab4bce quickdraw: Check for out of bound reads
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 4fd56f842cbaecf74df94c38f9c10452342f436a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
914b9b0b2b dsicinav: Check for out of bounds reads
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit e3ca9b93d9f464861638dda3280fcf65e402466a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
39de0e008d motionpixels: Fix the size of workspace buffers
Some buffers must be mod 4 in width and/or height.

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 210c80331e0604edf9c800865c26ba06ed3c2082)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
f2f2a00d39 motionpixels: Clear FF_INPUT_BUFFER_PADDING_SIZE bytes at the end of the temporary buffer
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit d337dd3a907110b32c6305bb65e4beca5b830c5d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
905d0633a6 wmavoice: Check for corrupted extra data
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit d99427cb8ba099375d8cce6df808d4acf045ab43)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
95605595b5 wmavoice: Check for out of bound writes
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 1c1449b548a2a0bf0295a522051b04107286653c)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
fb20141563 xan: Prevent NULL dereferences with missing reference frame
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 06be075cda0a6ba8bab8f543571b380884f562ac)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00