23273 Commits

Author SHA1 Message Date
Michael Niedermayer
0ffa44340f avcodec/mjpegdec: check bits per pixel for changes similar to dimensions
Fixes out of array accesses
Fixes: asan_heap-oob_16668e9_2_asan_heap-oob_16668e9_346_miss_congeniality_pegasus_mjpg.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c378d6a6df8243f06c87962b873bd563e58cd39)

Conflicts:

	libavcodec/mjpegdec.c
(cherry picked from commit 94371a404c663c3dae3d542fa43951567ab67f82)

Conflicts:

	libavcodec/mjpegdec.c
2014-10-03 02:30:00 +02:00
Michael Niedermayer
0bd9b78fb6 avcodec/jpeglsdec: Check run value more completely in ls_decode_line()
previously it could have been by 1 too large
Fixes out of array access
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8c1e3.jls
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8nde0.jls
Fixes: asan_heap-oob_12240fa_1_asan_heap-oob_12240fa_448_t16e3.jls

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 06e7d58410a17dc72c30ee7f3145fcacc425f4f2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 02:29:04 +02:00
Michael Niedermayer
e9d85860d0 avcodec/ac3enc_template: fix out of array read
Found-by: Andreas Cadhalpun
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d85ebea3f3b68ebccfe308fa839fc30fa634e4de)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-27 20:58:44 +02:00
Gianluigi Tiesi
e28c27e25f avcodec/libilbc: support for latest git of libilbc
in the latest git commits of libilbc developers removed WebRtc_xxx typedefs

This commit uses int types instead,
it's safe to apply also for previous versions since
WebRtc_Word16 was always a typedef of int16_t and
WebRtc_UWord16 a typedef of uint16_t

Reviewed-by: Timothy Gu <timothygu99@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 59af5383c18c8cf3fe2a4b5cc1ebf2f3300bdfe5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-21 17:04:19 +02:00
James Almer
9b9048a3dd x86/dsputil: add emms to ff_scalarproduct_int16_mmxext()
Also undo the changes to ra144enc.c from previous commits.
Should fix ticket #3429

Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9e0e1f9067430de1655a7b28536b5afed48bded5)

Conflicts:

	libavcodec/ra144enc.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-20 14:14:30 +02:00
Michael Niedermayer
d49b57fe0f avcodec/mpegvideo: Set err on failure in ff_mpv_common_frame_size_change()
Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfce6f7efd28130bf0dd409b2367ca0f8c9b2417)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 15:33:07 +02:00
Michael Niedermayer
0f1882b9b2 avcodec/mpegvideo: check that the context is initialized in ff_mpv_common_frame_size_change()
The function otherwise would initialize the context without setting context_initialized
alternatively we could set context_initialized

Fixes valgrind anomalies related to ticket 3928

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0d0f7f0ba43f64312ae4a05d97afecf1b7b1330c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 15:33:06 +02:00
Michael Niedermayer
0fc229450f avcodec/mpegvideo: Use "goto fail" for all error paths in ff_mpv_common_frame_size_change()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2762323c37511fbbc98b164c07620b9ebc59ec68)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 15:33:05 +02:00
Michael Niedermayer
1482253790 avcodec/snow: check coeffs for validity
Fixes deadlock
Fixes integer overflow
Fixes Ticket 3892

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 596636a474ab201badaae269f3a2cef4824b8c1f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:41:13 +02:00
Michael Niedermayer
48443598ed avcodec/utils: add GBRP16 to avcodec_align_dimensions2()
Fixes Ticket3869

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3fe9e7be4c70c8fccdcd56fd19276e668cfb7de8)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:41:13 +02:00
Michael Niedermayer
30ffb80dca avcodec: fix aac/ac3 parser bitstream buffer size
Buffers containing copies of the AAC and AC3 header bits were not padded
before parsing, violating init_get_bits() buffer padding requirement,
leading to potential buffer read overflows.
This change adds FF_INPUT_BUFFER_PADDING_SIZE bytes to the bit buffer
for parsing the header in each of aac_parser.c and ac3_parser.c.

Based on patch by: Matt Wolenetz <wolenetz@chromium.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fccd85b9f30525f88692f53134eba41f1f2d90db)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:41:13 +02:00
Christophe Gisquet
18342848f7 proresenc_kostya: report buffer overflow
If the allocated size, despite best efforts, is too small, exit
with the appropriate error.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 52b81ff4635c077b2bc8b8d3637d933b6629d803)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-17 16:49:56 +02:00
Luca Barbato
61a51598e3 lavc: move put_bits_left in put_bits.h
(cherry picked from commit afe03092dd693d025d43e1620283d8d285c92772)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-17 16:49:56 +02:00
Michael Niedermayer
abc1fa7c5a avcodec/iff: check pixfmt for rgb8 / rgbn
Fixes out of array access

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3539d6c63a16e1b2874bb037a86f317449c58770)

Conflicts:

	libavcodec/iff.c
(cherry picked from commit 656f930160db48e0b7b25069c62abc340e7f0628)

Conflicts:

	libavcodec/iff.c
2014-08-10 22:31:02 +02:00
Anton Khirnov
f712d3669e cdgraphics: do not return 0 from the decode function
0 means no data consumed, so it can trigger an infinite loop in the
caller.

CC:libav-stable@libav.org
(cherry picked from commit c7d9b473e28238d4a4ef1b7e8b42c1cca256da36)

Conflicts:

	libavcodec/cdgraphics.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-07 02:05:19 +02:00
Michael Niedermayer
b4632cec74 avcodec/dvdsub_parser: print message if packet is smaller than the packet size field
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bcc898dd2643c883522ffa565be4b226ce798c78)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:37 +02:00
Michael Niedermayer
454f6c622c avcodec/dvdsub_parser: Check buf_size before reading 32bit packet size
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 81c1657a593b1c0f8e46fca00ead1d30ee1cd418)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:36 +02:00
Michael Niedermayer
f70ee7d14f avcodec/dvdsub_parser: never return 0 when the input isnt 0
Fixes a infinite loop
Fixes Ticket3804

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfdb30d2f1241de9354a8efdbf8252d0f1a6f933)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:34 +02:00
Michael Niedermayer
9d77a62936 avcodec/alsdec: Clear MPEG4AudioConfig so that no use of uninitialized memory is possible
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6e6bd5481cf42a9765c492c77754d4633092cece)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:43 +02:00
Michael Niedermayer
d13cf32099 avcodec/mss4: Fix () in MKVAL() macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cf7ff0146c76b93c32edf5230a28b9590acf5105)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
5626e11bd5 avcodec/mss34dsp: fix () in SOP* macros
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6e720c5c815e510188a0bda654662383f2c48050)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
8d1f38c4b1 avcodec/mlpdec: fix () in MSB_MASK() macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fa160af08b6f42f17e93124aef86e3f6eec70d51)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
ea7fbd0b16 avcodec/golomb-test: fix () in EXTEND() macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 97e6b5ee3a16fee7d130f19f4dcee030f14d91cf)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
b2b4df8d51 avcodec/aac: fix () in IS_CODEBOOK_UNSIGNED macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fa915d4193e13187773c500b80c7df6baeb22c3b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
5691d9a6c1 avcodec/libvorbisenc: dont add the duration to AV_NOPTS_VALUE
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 19e66c7232d96e4ae8f05b52da2b84dfaa4e4da3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
4b638ea42d avcodec/diracdec: move mc buffer allocation to per frame
Fixes out of array accesses for non default buffers with large strides

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4a30f08505a4e85718896ff233c97be41a9754ca)
(cherry picked from commit 9c9fc79d9237d28e33161cb2e75082d8ad232b2e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
0607c8af2b avcodec/mjpegdec: Fix undefined shift
Fixes CID1194388

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b4329605289e25bb071ec1c1182bf25fc83b09aa)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
85b00d798f iavcodec/vc1dec: Fix missing {}
Fixes part of Ticket3466
Found-by: Andrey_Karpov / PVS-Studio
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cb53beb81a5b9192c79de401f1e1e13fadddc429)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:21 +02:00
Michael Niedermayer
53c02d04e4 avcodec/g723_1: add assert to help static code analyzers
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1457f3fd90e17745791354fbb87899fc4803085a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:21 +02:00
Michael Niedermayer
90c7dfb907 avcodec/diracdec: fix undefined behavior with shifts
Fixes part of Ticket3466
Found-by: Andrey_Karpov / PVS-Studio
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b8598f6ce61ccda3f2ff0c730b009fb650e42986)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:21 +02:00
Michael Niedermayer
7b38ea4003 avcodec/mjpegen: Fix declared argument size
Fixes part of Ticket3466
Found-by: Andrey_Karpov / PVS-Studio
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 256f530603ef3838a712a4fcd737b46b7bce455e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Michael Niedermayer
98fe429ba5 avcodec/x86/idct_sse2_xvid: fix non C99 inline function
Found-by: Matt Oliver <protogonoi@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 46d5625f44185271862337d61cd246fd569c42a4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Anthoine Bourgeois
90cf672d7a avcodec/dirac_arith: Fix build with PIC and stack-check options
Fixes Ticket3540

The function dirac_get_arith_bit in libavcodec/dirac_arith.h can't be
built with PIC and check-stack because the asm code needs 6 registers
and PIC and check-stack options take 1 each and x86 is quite limited
in this area.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d8ab7f31dd819f7b3e0d460a2fa4261aaae87b98)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Michael Niedermayer
a61998e4f4 avcodec/msrle: check return code for success before use
The check is possibly redundant, but better to check for errors
that dont occur than to skip the check and crash

Fixes CID1197060

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 754f84663e8b3a88fa2e953b195d59230393fb8d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Michael Niedermayer
e5546d4871 avcodec/wma: use av_freep(), do not leave stale pointers in memory
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d167faafe9dfa0b82bebb267c3c4e5fa5286bd67)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Michael Niedermayer
e897eb108b avcodec/libx264: move where x264opts is applied down so it isnt overridden by avctx & defaults
fixes x264opts opengop=1

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 64b79141bdfdffaa9fda69eecce140473d0a9a18)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Michael Niedermayer
2af3a04164 avcodec/h264_mp4toannexb_bsf: prepend global headers before any in stream parameter sets
Fixes h264_mp4toannexb_bsf_failure.mkv

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 289b149cecb381522cc9ccdf382825330169c655)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Michael Niedermayer
92ca3578bf avcodec/x86/mpegvideoenc_template: fix integer overflow
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Martin Storsjö
401b3dafd9 adpcm: Write the proper predictor in trellis mode in IMA QT
The actual predictor value, set by the trellis code, never
was written back into the variable that was written into
the block header. This was accidentally removed in b304244b.

This significantly improves the audio quality of the trellis
case, which was plain broken since b304244b.

Encoding IMA QT with trellis still actually gives a slightly
worse quality than without trellis, since the trellis encoder
doesn't use the exact same way of rounding as in
adpcm_ima_qt_compress_sample and adpcm_ima_qt_expand_nibble.

Fixes part of Ticket3701

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fa8f060b75bf9074792a0f9ff4ed002652ef62b8)

Conflicts:
	tests/ref/acodec/adpcm-ima_qt-trellis
2014-06-10 00:24:30 +02:00
Martin Storsjö
71516ac175 adpcm: Fix trellis encoding of IMA QT
This was broken in 095be4fb - samples+ch (for the previous
non-planar case) equals &samples_p[ch][0]. The confusion
probably stemmed from the IMA WAV case where it originally
was &samples[avctx->channels + ch], which was correctly
changed into &samples_p[ch][1].

Fixes part of Ticket3701

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a32765c4252eb106a2ade543026ef6f59e699bfa)
2014-06-10 00:24:22 +02:00
Carl Eugen Hoyos
0983720cf2 avcodec/utvideodec: Increase vlc len
Fixes a regression since fb3e380 similar to ticket #2661,
reported by fluffrabbit at aol dot com.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 673716c54b39eba9579a38ad222130e3f9549167)

Conflicts:
	libavcodec/utvideodec.c
2014-06-10 00:21:05 +02:00
Michael Niedermayer
2c71734d9e avcodec/vorbisdec: try to workaround libvorbisenc bug
Fixes Ticket3590

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0a266cb55af9794fc5cff695d35cae4111e4334f)

Conflicts:
	libavcodec/vorbisdec.c
2014-05-01 18:32:13 +02:00
Michael Niedermayer
f07cebcd91 avcodec/h263dec: Fix use of uninitialized memory from the bitstream buffer
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-18 18:06:17 +01:00
Peter Ross
b1a960cc83 avcodec/adpcm: squelch 'mismatch in coded sample count' warning for AV_CODEC_ID_ADPCM_EA_R2/3
These ADPCM codecs include a per-frame flag that enables a raw 16-bit mode. Therefore
the the number of samples returned by get_nb_samples() is only ever approximate.

Fixes ticket #3460.

Signed-off-by: Peter Ross <pross@xvid.org>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7380201451a2edfb240cd356579c4c39a87cf5bd)
2014-03-17 16:06:20 +01:00
Peter Ross
620d80d572 avcodec/adpcm: ADPCM_IMA_DK3 packets are padded to 16-bit packet boundary
Fixes ticket #3461.

Signed-off-by: Peter Ross <pross@xvid.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d1bb17940dd242e56541775318636bcbe3eab73d)
2014-03-17 16:06:09 +01:00
Michael Niedermayer
e3ce3d373c avcodec/h264: be more tolerant on what pixel format changes trigger reinits
Fixes Ticket3260

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8e92ff25469f75f5c1fcbb9ba5721cea341ca34a)

Conflicts:
	libavcodec/h264.c
2014-03-17 16:06:04 +01:00
Michael Niedermayer
fb44a3e4a3 avcodec/vorbisdec: use the stored previous window type only when the actual previous is not known
Fixes Ticket3432

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5171ae781a240cac3860c20f9aefc6d1b2c61cac)
2014-03-13 07:35:17 +01:00
Michael Niedermayer
ddcccababe avcodec/utvideoenc: fix slice_bits size
Fixes assertion failure

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0a8c90202bb906747168a698b6837496f82c717c)

Conflicts:

	libavcodec/utvideoenc.c
(cherry picked from commit 57522ca79cc38c279123596d3288ddbf56fa8903)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-05 03:06:10 +01:00
Michael Niedermayer
b580bae53a avcodec/snow: split block clipping checks
Fixes out of array read
Fixes: d4476f68ca1c1c57afbc45806f581963-asan_heap-oob_2266b27_8607_cov_4044577381_snow_chroma_bug.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 61d59703c91869f4e5cdacd8d6be52f8b89d4ba4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-05 03:06:10 +01:00
Michael Niedermayer
9085cdd677 avcodec/ansi: fix integer overflow
Fixes out of array read
Fixes: 5f9698e86d92f19bb08d54ff0d57027f-signal_sigsegv_b30756_3795_cov_2693691257_ansi256.ans
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d42ec8433c687fcbccefa51a7716d81920218e4f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-05 03:06:10 +01:00