22711 Commits

Author SHA1 Message Date
Luca Barbato
73d5d7acb0 indeo: reject negative array indexes
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

(cherry picked from commit 6a10142faa1cca8ba2bfe51b970754f62d60f320)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-06 15:06:31 +02:00
Luca Barbato
80d73b4ada indeo: Cosmetic formatting
Trim some overly long lines.

(cherry picked from commit 6dfacd7ab126aea1392949d1aa10fdc3d3eeb911)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-06 15:06:31 +02:00
Luca Barbato
b9892e1813 indeo: Refactor ff_ivi_init_tiles and ivi_decode_blocks
Spin large and mostly self contained blocks into stand alone
functions.

(cherry picked from commit 62256010e9bc8879e2bf7f3b94af8ff85e239082)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-06 15:06:31 +02:00
Luca Barbato
d76480e6ba indeo: Refactor ff_ivi_dec_huff_desc
Spare an indentation level.

(cherry picked from commit f6f36ca8ca1b2526d3abff7d7c627322d3bce912)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-06 15:06:31 +02:00
Luca Barbato
33388299fb indeo: use a typedef for the mc function pointer
(cherry picked from commit e6d8acf6a8fba4743eb56eabe72a741d1bbee3cb)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-06 15:06:31 +02:00
Luca Barbato
d8dab6c3b8 indeo: use proper error code
(cherry picked from commit dd3754a48854cd570d38db72394491aab0f36570)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-07-06 15:06:31 +02:00
Luca Barbato
5f7944a308 indeo: check for reference when inheriting mvs
The same is done already for qdelta.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit b36e1893ef3430f039c1eaddeedcbb378f9c4444)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-07-04 22:06:13 +02:00
Luca Barbato
f518fa6bee indeo: use proper error code
(cherry picked from commit dd3754a48854cd570d38db72394491aab0f36570)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-07-04 22:05:48 +02:00
Luca Barbato
51a23b0e95 indeo: Properly forward the error codes
If the tile data size does not match the buffer size it did not
return an AVERROR_INVALIDDATA causing futher corruption later.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 7388c0c58601477db076e2e74e8b11f8a644384a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-07-04 22:05:15 +02:00
Luca Barbato
2cdc976320 mjpeg: Check the unescaped size for overflows
And contextually check init_get_bits success and fix the reporting
message.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 6765ee7b9cba46818a45b051438b2552f0a1b70a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:
	libavcodec/mjpegdec.c
2013-06-30 16:03:27 +02:00
Luca Barbato
efcfd50c9f wmapro: error out on impossible scale factor offsets
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 02ec656af72030eea4f3d63e30b25625cce6a3df)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-06-30 16:03:27 +02:00
Luca Barbato
8bd0372937 wmapro: check the min_samples_per_subframe
Must be at least WMAPRO_BLOCK_MIN_SIZE.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit d4a217a408da4bd63acc02cd8f9ebe378a2ad65a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-06-30 16:03:27 +02:00
Luca Barbato
9761abffb6 wmapro: return early on unsupported condition
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 6652338f43ef623045912d7f28b61adea05d27ae)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:
	libavcodec/wmaprodec.c
2013-06-30 16:03:27 +02:00
Luca Barbato
fbeae4a951 wmapro: check num_vec_coeffs against the actual available buffer
Prevent yet another buffer overwrite.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 38229362529ed1619d8ebcc81ecde85b23b45895)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-06-30 16:03:27 +02:00
Luca Barbato
88433979c2 wmapro: make sure there is room to store the current packet
Prevent horrid and hard to trace struct overwrite.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit e30b068ef79f604ff439418da07f7e2efd01d4ea)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-06-30 16:03:27 +02:00
Luca Barbato
9d1b173aae lavc: move put_bits_left in put_bits.h
(cherry picked from commit afe03092dd693d025d43e1620283d8d285c92772)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-06-30 16:03:27 +02:00
Luca Barbato
c7934c6c0b 4xm: do not overread the source buffer in decode_p_block
Check for out of picture macroblocks before calling mcdc.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 94aefb1932be882fd93f66cf790ceb19ff575c19)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-06-30 16:03:27 +02:00
Luca Barbato
04c29196ad 4xm: check bitstream_size boundary before using it
Prevent buffer overread.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 59d7bb99b6a963b7e11c637228b2203adf535eee)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-06-30 16:03:27 +02:00
Hendrik Leppkes
24dc6b1a06 mathops/x86: work around inline asm miscompilation with GCC 4.8.1
The volatile is not required here, and prevents a miscompilation with GCC
4.8.1 when building on x86 with --cpu=i686

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 659df32a9d8984081ccd54adc3aee7daeb33388d)
2013-06-24 08:45:50 +02:00
Kostya Shishkov
5e6122ddad smacker: check the return value of smacker_decode_tree
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit a2f9937bb04b23a341b0ec0eb1d923bbeb420277)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-06-16 15:58:26 +02:00
Kostya Shishkov
1a0cdd18b0 smacker: fix an off by one in huff.length computation
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit ee205588b250fe5cae0681be8eba51a5403c3272)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-06-16 15:58:22 +02:00
Luca Barbato
d33b0f7224 4xm: do not overread the prestream buffer
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit be373cb50d3c411366fec7eef2eb3681abe48f96)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-06-16 15:54:23 +02:00
Luca Barbato
6ddc1eb037 4xm: validate the buffer size before parsing it
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit de2e5777e225e75813daf2373c95e223651fd89a)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-06-16 15:54:15 +02:00
Luca Barbato
ded74ab5d1 4xm: reject frames not compatible with the declared version
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 145023f57262d21474e35b4a6069cf95136339d4)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-06-16 15:54:06 +02:00
Luca Barbato
f82e9deec2 4xm: drop pointless assert
Make sure the value of wlog2 is always between 0 and 3.
(cherry picked from commit 1f0c6075604c271d5627480f1243d22795f9a315)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-06-16 15:53:55 +02:00
Luca Barbato
d0cabcc789 4xm: forward errors from decode_p_block
Partially mitigate out of memory writes.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit b8b809908ec547b2609dbac24194f4fd2df61aea)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-06-16 15:53:44 +02:00
Luca Barbato
dac0d4f354 4xm: fold last_picture lazy allocation in decode_p_frame
(cherry picked from commit 50ec1db62d977b6e864f315a53c1c580a6d7efa4)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/4xm.c
2013-06-16 15:53:33 +02:00
Luca Barbato
04c506e912 4xm: use the correct logging context
(cherry picked from commit 08859d19b429c522d6494c186656f4a2d3ff8e21)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-06-16 15:15:11 +02:00
Michael Niedermayer
bc4dc32b2a alacenc: Fix missing sign_extend()
Fixes ticket #2497

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8aea2f05dc56f7e7d60767dd27ba8e846a05e8ae)
2013-06-13 00:04:23 +02:00
Michael Niedermayer
01580c0955 Merge remote-tracking branch 'qatar/release/9' into release/1.1
* qatar/release/9:
  tiff: do not overread the source buffer
  apetag: use int64_t for filesize

Conflicts:
	libavcodec/tiff.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-09 11:06:28 +02:00
Luca Barbato
8eb7c2566c tiff: do not overread the source buffer
At least 2 bytes from the source are read every loop.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 9c2216976907336dfae0e8e38a4d70ca2465a92c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:
	libavcodec/tiff.c
2013-06-08 16:31:54 +02:00
Michael Niedermayer
2fae70db2a vmdav: Try to fix unpack_rle()
This fixes out of array accesses
The code prior to this commit could not have worked, thus obviously
was untested. I was also not able to find a valid sample that uses this
code.
This fix is thus only based on the description of the format

If someone has a sample that uses unpack_rle(), please mail me.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c1f2c4c3b49277d65b71ccdd3b6b2878f1b593eb)

Conflicts:

	libavcodec/vmdav.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-03 02:34:10 +02:00
Michael Niedermayer
f08b0ff051 Merge remote-tracking branch 'qatar/release/9' into release/1.1
* qatar/release/9:
  vmd: refactor the inner decode loop

Conflicts:
	libavcodec/vmdav.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-03 02:21:38 +02:00
Michael Niedermayer
d6373f1586 Merge commit 'dbaf3f7b0bc9e99dff8e06bd29fcb3e84eebfe7c' into release/1.1
* commit 'dbaf3f7b0bc9e99dff8e06bd29fcb3e84eebfe7c':
  vmd: drop incomplete chunks and spurious samples
  vmd: return meaningful errors

Conflicts:
	libavcodec/vmdav.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-03 01:57:01 +02:00
Michael Niedermayer
2a39548181 Merge commit '4f6fbe47a9f784373c277870d9d4989762873bf1' into release/1.1
* commit '4f6fbe47a9f784373c277870d9d4989762873bf1':
  vmdav: convert to bytestream2

Conflicts:
	libavcodec/vmdav.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-03 01:32:19 +02:00
Michael Niedermayer
4c052a7b8b Merge commit '7251de30322aff5660e571856132dc6c7256fe94' into release/1.1
* commit '7251de30322aff5660e571856132dc6c7256fe94':
  wavpack: use bytestream2 in wavpack_decode_block

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-03 01:09:23 +02:00
Michael Niedermayer
30394adc44 Merge commit '5ba83e90919cdeef38e2b5343b48f3f367292564' into release/1.1
* commit '5ba83e90919cdeef38e2b5343b48f3f367292564':
  wavpack: return meaningful errors

Conflicts:
	libavcodec/wavpack.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-03 00:57:23 +02:00
Michael Niedermayer
f908e3ce92 Merge commit '93fbf034c94caf7ddfecd3c1947e3139fef6bfca' into release/1.1
* commit '93fbf034c94caf7ddfecd3c1947e3139fef6bfca':
  wavpack: check packet size early

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-03 00:52:17 +02:00
Michael Niedermayer
683cbbb721 Merge commit '10f77c165c3b3e881bb174a0f57dd62083639072' into release/1.1
* commit '10f77c165c3b3e881bb174a0f57dd62083639072':
  pixdesc: mark gray8 as pseudopal
  mjpegdec: validate parameters in mjpeg_decode_scan_progressive_ac
  mjpeg: Validate sampling factors
  ljpeg: use the correct number of components in yuv
  wavpack: validate samples size parsed in wavpack_decode_block

Conflicts:
	libavcodec/mjpegdec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-03 00:40:23 +02:00
Michael Niedermayer
cff8d01e15 Merge commit '0af5a774ebc96ae9018926dc8b276c7f39767e3e' into release/1.1
* commit '0af5a774ebc96ae9018926dc8b276c7f39767e3e':
  jpegls: check the scan offset
  jpegls: factorize return paths
  jpegls: return meaningful errors
  mpegvideo: allocate sufficiently large scratch buffer for interlaced vid

Conflicts:
	libavcodec/jpeglsdec.c
	libavcodec/mpegvideo.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-03 00:16:52 +02:00
Michael Niedermayer
8c118207ea Merge commit 'aaeef7fa0d6ebb1a3668894e67a70cd5084ce4f4' into release/1.1
* commit 'aaeef7fa0d6ebb1a3668894e67a70cd5084ce4f4':
  mjpegdec: properly report unsupported disabled features
  Prepare for 9.7 Release
  update Changelog
  proresdec: support mixed interlaced/non-interlaced content

Conflicts:
	RELEASE
	libavcodec/mjpegdec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-03 00:06:22 +02:00
Claudio Freire
c320f9f5e9 AAC encoder: Fix rate control on twoloop.
Fixes a case where multichannel bitrate isn't accurately
targetted by psy model alone, never achieving the target bitrate.
Now fixed.

Fixes ticket #2625.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Carl Eugen Hoyos <cehoyos@ag.or.at>
2013-06-02 16:26:36 +02:00
Luca Barbato
5fed47b94f vmd: refactor the inner decode loop
Simplify a little, assume empty frames are acceptable and
do not pointlessly reinit the bytestream2 contexts using
possibly wrong size values.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 676da248cad49debc40720baa13214f0b94dcc71)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/vmdav.c
2013-06-01 15:28:19 +02:00
Luca Barbato
5a01ab0e62 vmd: use the PALETTE_COUNT constant uniformly
While at it drop useless parentheses.
(cherry picked from commit 91a6944e56236234f0a7ba162404665753cbcb51)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-06-01 05:38:38 +02:00
Luca Barbato
dbaf3f7b0b vmd: drop incomplete chunks and spurious samples
Odd chunk size makes no sense for stereo and incomplete chunks are
not supported.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 701966730ce10290fd49c5ccedd73f505680f764)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-05-31 23:14:01 +02:00
Alexandra Khirnova
4f6fbe47a9 vmdav: convert to bytestream2
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 0afcf97e1ece51d29bb791698b00cd1b7ba97dcf)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:
	libavcodec/vmdav.c
2013-05-31 23:00:31 +02:00
Luca Barbato
7251de3032 wavpack: use bytestream2 in wavpack_decode_block
Prevent most out of buffer reads.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 3f0b6d7a6248a33df37b98cfcb37a1acce263f62)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:
	libavcodec/wavpack.c
2013-05-31 23:00:31 +02:00
Luca Barbato
5ba83e9091 wavpack: return meaningful errors
And forward those that were already meaningful.
(cherry picked from commit 8c34558131d846d2b10389564caadaa206372fd4)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:
	libavcodec/wavpack.c
2013-05-31 23:00:31 +02:00
Luca Barbato
93fbf034c9 wavpack: check packet size early
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit fd06291239c1bb616bf303b5696cc432710b2530)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-05-31 23:00:31 +02:00
Luca Barbato
5a8dcc993d vmd: return meaningful errors
CC: libav-stable@libav.org
(cherry picked from commit c8f3cb9119c2183680d44a509a1b5a9817a3bee9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:
	libavcodec/vmdav.c
2013-05-31 23:00:31 +02:00