22589 Commits

Author SHA1 Message Date
Michael Niedermayer
cb0d255e2f avcodec/mjpegdec: Fix undefined shift
Fixes CID1194388

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b4329605289e25bb071ec1c1182bf25fc83b09aa)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-05-10 21:55:41 +02:00
Michael Niedermayer
ebefc92a3d Merge commit '945a1b7ab9b9df317e175e480a135a41e7e05b5c' into release/1.1
* commit '945a1b7ab9b9df317e175e480a135a41e7e05b5c':
  h264: reset next_output_pic earlier in start_frame()

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-05-04 22:14:45 +02:00
Michael Niedermayer
11b3462aa3 Merge commit 'cf7bb6ceb1da7682aa54989f83d81c67a5e6790d' into release/1.1
* commit 'cf7bb6ceb1da7682aa54989f83d81c67a5e6790d':
  Revert "pthread: flush all threads on flush, not just the first one"

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-05-04 22:02:08 +02:00
Anton Khirnov
945a1b7ab9 h264: reset next_output_pic earlier in start_frame()
In case start_frame() fails, this potentially invalid frame can still be
output to the caller.

Bug-Id: 672
Bug-Id: debian/741240
Bug-Id: ubuntu/1288206
2014-04-24 13:06:02 +02:00
Anton Khirnov
cf7bb6ceb1 Revert "pthread: flush all threads on flush, not just the first one"
This reverts commit 2eb15cdeef29eb8a0a32658154decba94b4b89cb.

It does not work correctly in pre-refcounting threading code.
2014-04-24 07:46:48 +02:00
Michael Niedermayer
47c23f9392 avcodec/g723_1: add assert to help static code analyzers
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1457f3fd90e17745791354fbb87899fc4803085a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-22 00:21:28 +02:00
Michael Niedermayer
d51e0dd9b4 avcodec/diracdec: fix undefined behavior with shifts
Fixes part of Ticket3466
Found-by: Andrey_Karpov / PVS-Studio
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b8598f6ce61ccda3f2ff0c730b009fb650e42986)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-22 00:21:28 +02:00
Michael Niedermayer
15f122316e avcodec/x86/idct_sse2_xvid: fix non C99 inline function
Found-by: Matt Oliver <protogonoi@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 46d5625f44185271862337d61cd246fd569c42a4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-22 00:21:28 +02:00
Anthoine Bourgeois
b546b7a029 avcodec/dirac_arith: Fix build with PIC and stack-check options
Fixes Ticket3540

The function dirac_get_arith_bit in libavcodec/dirac_arith.h can't be
built with PIC and check-stack because the asm code needs 6 registers
and PIC and check-stack options take 1 each and x86 is quite limited
in this area.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d8ab7f31dd819f7b3e0d460a2fa4261aaae87b98)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-22 00:21:28 +02:00
Michael Niedermayer
93fe53da91 avcodec/wma: use av_freep(), do not leave stale pointers in memory
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d167faafe9dfa0b82bebb267c3c4e5fa5286bd67)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-22 00:21:28 +02:00
Michael Niedermayer
8130449f32 avcodec/h264_mp4toannexb_bsf: prepend global headers before any in stream parameter sets
Fixes h264_mp4toannexb_bsf_failure.mkv

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 289b149cecb381522cc9ccdf382825330169c655)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-22 00:21:28 +02:00
Michael Niedermayer
f741c39752 avcodec/x86/mpegvideoenc_template: fix integer overflow
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-22 00:21:27 +02:00
Michael Niedermayer
af1bf7f277 avcodec/h263dec: Fix use of uninitialized memory from the bitstream buffer
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f07cebcd910c97ff6012085c21493231752990e9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-22 00:21:27 +02:00
Michael Niedermayer
63b38d4a91 Merge commit '85b8b169175a55fc862e01ecc96f649374bc14d2' into release/1.1
* commit '85b8b169175a55fc862e01ecc96f649374bc14d2':
  tiffdec: use bytestream2 to simplify overread/overwrite protection

Conflicts:
	libavcodec/tiff.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 17:06:03 +02:00
Michael Niedermayer
260f37ee4d Merge commit 'd946e4c1bcfb6accd97f6d4cbfa324ef01913c8d' into release/1.1
* commit 'd946e4c1bcfb6accd97f6d4cbfa324ef01913c8d':
  bytestream: add bytestream2_copy_buffer() functions

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 16:57:01 +02:00
Michael Niedermayer
0162ad1a59 Merge commit '2eb15cdeef29eb8a0a32658154decba94b4b89cb' into release/1.1
* commit '2eb15cdeef29eb8a0a32658154decba94b4b89cb':
  pthread: flush all threads on flush, not just the first one

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 16:25:24 +02:00
Michael Niedermayer
fa73d13678 Merge commit 'b701e26a4e89e3816490bc74bc47f82e6b7f7002' into release/1.1
* commit 'b701e26a4e89e3816490bc74bc47f82e6b7f7002':
  mpeg12: check scantable indices in all decode_block functions

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 16:18:24 +02:00
Michael Niedermayer
8401ed651b Merge commit 'ea1806ce650f0502dd25939c335b9216fa4a955f' into release/1.1
* commit 'ea1806ce650f0502dd25939c335b9216fa4a955f':
  sgidec: fix buffer size check in expand_rle_row()

Conflicts:
	libavcodec/sgidec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 16:00:51 +02:00
Michael Niedermayer
fcaf78f5f9 Merge commit 'ff409c7d80b6781e4b81dd860badb569061ddaa5' into release/1.1
* commit 'ff409c7d80b6781e4b81dd860badb569061ddaa5':
  adx: check that the offset is not negative

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 15:55:28 +02:00
Michael Niedermayer
8e5a021512 Merge commit 'c9be276f8b040e7d441a77f774afe94b6f137be6' into release/1.1
* commit 'c9be276f8b040e7d441a77f774afe94b6f137be6':
  mpegvideo: set reference/pict_type on generated reference frames

Conflicts:
	libavcodec/mpegvideo.c

See: 532b93b51631ee58443bee8aec7c2a2b2a1746be
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 15:54:30 +02:00
Michael Niedermayer
a23748a804 Merge commit 'd8e89a37267f276afd404bd062e5112a336d1a36' into release/1.1
* commit 'd8e89a37267f276afd404bd062e5112a336d1a36':
  h264: reset data partitioning at the beginning of each decode call

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 15:33:52 +02:00
Michael Niedermayer
6961f9f9f4 Merge commit '7e8d27c8510a024e6d66d2e1116bb8692d9cb98a' into release/1.1
* commit '7e8d27c8510a024e6d66d2e1116bb8692d9cb98a':
  h264: reset ref count if decoding the slice header fails

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 15:29:34 +02:00
Michael Niedermayer
11dfd2a74a Merge commit '7fd6c9fb49573870b48a995947edc83eedd7e86e' into release/1.1
* commit '7fd6c9fb49573870b48a995947edc83eedd7e86e':
  h264: reset first_field if frame_start() fails for missing refs

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 15:28:24 +02:00
Michael Niedermayer
f294748cbc h264: fix size of arrays in ff_h264_check_intra_pred_mode()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d6a33f5d20b6ef2eae2cbb959b001cb125a564b7)

Conflicts:

	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 15:17:43 +02:00
Michael Niedermayer
40e6c148f5 h264/ff_h264_check_intra_pred_mode: fix input value check
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2005fddcbb4e18e8f7c34326e40609e4a2d83c31)

Conflicts:

	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 15:17:04 +02:00
Michael Niedermayer
9081b4dddd Merge commit '7f604a048e9b6128cdf9ce7e95f21d1a9822ba39' into release/1.1
* commit '7f604a048e9b6128cdf9ce7e95f21d1a9822ba39':
  h264: reject mismatching luma/chroma bit depths during sps parsing

Conflicts:
	libavcodec/h264_ps.c

See: bdeb61ccc67911cfc5e20c7cfb1312d0501ca90a
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 15:15:41 +02:00
Michael Niedermayer
58e2d529b3 Merge commit '7e513d85e80d730718695d09fcaf0295ae24699e' into release/1.1
* commit '7e513d85e80d730718695d09fcaf0295ae24699e':
  h264: check that execute_decode_slices() is not called too many times

See: 64591f8f86f2dfeac13ee6b4e971d069675ca814
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 15:02:54 +02:00
Michael Niedermayer
b9550ea116 Merge commit 'b6a0f5cde808d84626f07b11c381cc6ef0954940' into release/1.1
* commit 'b6a0f5cde808d84626f07b11c381cc6ef0954940':
  h264: do not use 422 functions for monochrome

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 15:01:30 +02:00
Michael Niedermayer
58bdec57fa Merge commit '1147d39bca135df8673031d16989ba486a52172a' into release/1.1
* commit '1147d39bca135df8673031d16989ba486a52172a':
  h264: reset data_partitioning if decoding the slice header for NAL_DPA fails

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 15:00:53 +02:00
Michael Niedermayer
7434225ed1 Merge commit '30e58e65e543d04d5e52bf188b31abf4dd41a104' into release/1.1
* commit '30e58e65e543d04d5e52bf188b31abf4dd41a104':
  h264_refs: make sure not to write over the bounds of the default ref list

Conflicts:
	libavcodec/h264_refs.c

See: 0a5ca63c24b8a07f94611637ba29e41b8b47ce24
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 14:39:55 +02:00
Michael Niedermayer
c9f17afe4e Merge commit 'af40847681b978441fdd25276685414f9e17820a' into release/1.1
* commit 'af40847681b978441fdd25276685414f9e17820a':
  h264: check buffer size before accessing it

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 14:34:02 +02:00
Michael Niedermayer
f6cb3827df Merge commit '7a2254ab26b24255a4d6ea476ba5fd38525a9bb3' into release/1.1
* commit '7a2254ab26b24255a4d6ea476ba5fd38525a9bb3':
  arm: hpeldsp: fix put_pixels8_y2_{,no_rnd_}armv6

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-21 13:55:37 +02:00
Justin Ruggles
85b8b16917 tiffdec: use bytestream2 to simplify overread/overwrite protection
Based on a patch by Paul B Mahol <onemda@gmail.com>

CC:libav-stable@libav.org
2014-04-15 00:45:06 -04:00
Justin Ruggles
d946e4c1bc bytestream: add bytestream2_copy_buffer() functions
This is basically an overread/overwrite-safe memcpy between a
GetByteContext and a PutByteContext.

CC:libav-stable@libav.org
(cherry picked from commit 5748faf291fec297ef25d81962b52b3438f54278)
2014-04-14 23:19:18 -04:00
Paul B Mahol
0c5e6c542f bytestream: add functions for accessing size of buffer
Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>

CC:libav-stable@libav.org
(cherry picked from commit de9d2705f61ef569487ec5f8974a9c7ce34ec783)
2014-04-14 23:18:53 -04:00
Diego Biurrun
2eb15cdeef pthread: flush all threads on flush, not just the first one
avcodec_flush_buffers() must release all internally held references
according to its documentation, for which all the threads need to be
flushed.

CC:libav-stable@libav.org
Bug-Id: vlc/9665
2014-04-14 18:09:09 -04:00
Janne Grunau
b701e26a4e mpeg12: check scantable indices in all decode_block functions
Add checks to the fast functions used with CODEC_FLAGS2_FAST and move
the check for all other functions to before the invalid memory is
accessed. Fixes https://trac.videolan.org/vlc/ticket/9713 with
CODEC_FLAGS2_FAST.

CC: libav-stable@libav.org
2014-04-14 17:45:19 -04:00
Anton Khirnov
ea1806ce65 sgidec: fix buffer size check in expand_rle_row()
Right now it will spuriously fail if the linesize is exactly equal to
the data width.

CC:libav-stable@libav.org
2014-04-14 16:55:39 -04:00
Anton Khirnov
ff409c7d80 adx: check that the offset is not negative
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 5569146d48f06564e8fa393424782cceed510916)
2014-04-14 16:55:39 -04:00
Anton Khirnov
c9be276f8b mpegvideo: set reference/pict_type on generated reference frames
Otherwise the generic code will unref them, which can then result in
last_picture_ptr == current_picture_ptr, which causes deadlocks at least
in rv40.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:39 -04:00
Anton Khirnov
d8e89a3726 h264: reset data partitioning at the beginning of each decode call
Prevents using GetBitContexts with data from previous calls.

Fixes access to freed memory.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:39 -04:00
Anton Khirnov
7e8d27c851 h264: reset ref count if decoding the slice header fails
Otherwise the ER code might try to use some already freed references.

Fixes possible access to freed memory.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:39 -04:00
Anton Khirnov
7fd6c9fb49 h264: reset first_field if frame_start() fails for missing refs
In this case we may not have a current frame, while first_field being
set implies we do.

Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:38 -04:00
Anton Khirnov
7f604a048e h264: reject mismatching luma/chroma bit depths during sps parsing
There is no point in delaying the check and it avoids bugs with a
half-initialized context.

Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:38 -04:00
Anton Khirnov
7e513d85e8 h264: check that execute_decode_slices() is not called too many times
Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 9eef9eb3014b2ed9c3ff4aac510a9f04edb555cf)
2014-04-14 16:55:38 -04:00
Anton Khirnov
b6a0f5cde8 h264: do not use 422 functions for monochrome
Fixes invalid memory access.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:38 -04:00
Anton Khirnov
1147d39bca h264: reset data_partitioning if decoding the slice header for NAL_DPA fails
If it was set before then we can end up trying to decode a slice without
a valid slice header, which can lead to invalid memory access.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 58312b2472d3a44d7458865c459d59ef2e02bf1a)
2014-04-14 16:55:38 -04:00
Anton Khirnov
30e58e65e5 h264_refs: make sure not to write over the bounds of the default ref list
Fixes invalid writes.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:38 -04:00
Anton Khirnov
175b53d051 h264: limit allowed pred modes in ff_h264_check_intra_pred_mode() to 3
Higher modes are not allowed for 16x16/chroma, which is what this
function is used for. Otherwise this function would return 0 (vertical
prediction) for invalid higher modes, which could result in invalid
reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:38 -04:00
Anton Khirnov
af40847681 h264: check buffer size before accessing it
Fixes invalid reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:37 -04:00