29184 Commits

Author SHA1 Message Date
Michael Niedermayer
57bdb3f3dd avcodec/mjpegdec: check bits per pixel for changes similar to dimensions
Fixes out of array accesses
Fixes: asan_heap-oob_16668e9_2_asan_heap-oob_16668e9_346_miss_congeniality_pegasus_mjpg.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c378d6a6df8243f06c87962b873bd563e58cd39)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
5e34dded10 avcodec/jpeglsdec: Check run value more completely in ls_decode_line()
previously it could have been by 1 too large
Fixes out of array access
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8c1e3.jls
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8nde0.jls
Fixes: asan_heap-oob_12240fa_1_asan_heap-oob_12240fa_448_t16e3.jls

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 06e7d58410a17dc72c30ee7f3145fcacc425f4f2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
ca2ccd85d7 avcodec/ac3enc_template: fix out of array read
Found-by: Andreas Cadhalpun
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d85ebea3f3b68ebccfe308fa839fc30fa634e4de)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Pascal Massimino
aa40bbb492 avcodec/webp: fix default palette color 0xff000000 -> 0x00000000
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e5b3112996c3da45aa03b39c5ade375d40d4407d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Gianluigi Tiesi
8e276fc96a avcodec/libilbc: support for latest git of libilbc
in the latest git commits of libilbc developers removed WebRtc_xxx typedefs

This commit uses int types instead,
it's safe to apply also for previous versions since
WebRtc_Word16 was always a typedef of int16_t and
WebRtc_UWord16 a typedef of uint16_t

Reviewed-by: Timothy Gu <timothygu99@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 59af5383c18c8cf3fe2a4b5cc1ebf2f3300bdfe5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Pascal Massimino
4a4e30a6d8 libavcodec/webp: treat out-of-bound palette index as translucent black
See https://code.google.com/p/webp/issues/detail?id=206
for a description of the problem/fix.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This patch makes the decoder follow the recommendation of the spec.
There is some disagreement (see "[FFmpeg-devel] [PATCH]: libavcodec/webp")
about what would be best to be written in the spec, so in case the spec
is changed again, this potentially would need to be amended or reverted
(cherry picked from commit 4fd21d58a72c38ab63c3a4483b420db260fa7b8d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
1c733a440a avcodec/x86/vp9lpf: Always include x86util.asm
Fixes executable stack

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 41d82b85ab0ee8bb2931c1f783e30c38c2fb5206)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
6952f6f39b avcodec/cabac_functions: fix "warning: UNCHECKED_BITSTREAM_READER is not defined, evaluates to 0"
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a67b6c95415438b4a6bfb6277f066f7c7b57b40f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
beb55b3981 avcodec/mpegvideo: Set err on failure in ff_mpv_common_frame_size_change()
Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfce6f7efd28130bf0dd409b2367ca0f8c9b2417)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
80aec733ad avcodec/mpegvideo: check that the context is initialized in ff_mpv_common_frame_size_change()
The function otherwise would initialize the context without setting context_initialized
alternatively we could set context_initialized

Fixes valgrind anomalies related to ticket 3928

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0d0f7f0ba43f64312ae4a05d97afecf1b7b1330c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
77bb6b5bcc avcodec/mpegvideo: Use "goto fail" for all error paths in ff_mpv_common_frame_size_change()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2762323c37511fbbc98b164c07620b9ebc59ec68)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
f68395f7fc avcodec/h264: Allow partial escaping
Fixes Ticket3923

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 033a5334badd8af48f13c6fd1e6827f8e3f2c2f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
f903147f2d avcodec/mjpegdec: fix green line at the bottom with upscale v
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f0d4f00f24d41441ebd63694df57ab203a764aa5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:58 +02:00
Michael Niedermayer
9a840d5e17 avcodec/mjpegdec: fix green vertical line at the right with upscale h
Fixes Ticket3891

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 63a52ca134dc4ee648a547c7c410d547f371b481)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:58 +02:00
Michael Niedermayer
e13e928baa avcodec/snow: check coeffs for validity
Fixes deadlock
Fixes integer overflow
Fixes Ticket 3892

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 596636a474ab201badaae269f3a2cef4824b8c1f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:58 +02:00
Michael Niedermayer
17a6ca7d31 avcodec/vc1dec: fix null pointer dereference
Fixes Ticket3860

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 30f680ee0a2707af9a649a0aa3fd951d18a25c05)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:16:38 +02:00
Michael Niedermayer
8a20224059 avcodec/utils: add GBRP16 to avcodec_align_dimensions2()
Fixes Ticket3869

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3fe9e7be4c70c8fccdcd56fd19276e668cfb7de8)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:16:37 +02:00
Michael Niedermayer
29ee8b72c4 avcodec: fix aac/ac3 parser bitstream buffer size
Buffers containing copies of the AAC and AC3 header bits were not padded
before parsing, violating init_get_bits() buffer padding requirement,
leading to potential buffer read overflows.
This change adds FF_INPUT_BUFFER_PADDING_SIZE bytes to the bit buffer
for parsing the header in each of aac_parser.c and ac3_parser.c.

Based on patch by: Matt Wolenetz <wolenetz@chromium.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fccd85b9f30525f88692f53134eba41f1f2d90db)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:16:37 +02:00
Christophe Gisquet
25864cf562 wavpackenc: proper buffer allocation
The allocation didn't account for headers, that can be easily 79 bytes.
As a result, buffers allocated for a few samples (e.g. 5 in the original
bug) could be undersized.

Fixed ticket #2881.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2ba58bec20b0039ccc40cfba59af6d56de16e8b1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 02:51:57 +02:00
Christophe Gisquet
f74206cb40 wavpack: report if there is no bits left
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 11a39bdf534a4ead634b4a593c66ebf756910b9b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 02:51:57 +02:00
Michael Niedermayer
3b6bde3b3d avcodec/h264_mb: fix grayscale only decoding with weighted prediction
Fixes Ticket3412

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 94f60b65446b37132d7bd644ab2c4627d9488295)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-17 15:09:05 +02:00
Michael Niedermayer
10c2d22ba1 avcodec/mjpegdec: Support AV_PIX_FMT_YUV420P16 with upscale_h
Fixes assertion failure
Fixes: test42f.jpg
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c7899a4834ee927f5629e4c02bfa225b846f016)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-14 16:53:36 +02:00
Christophe Gisquet
35738e5898 proresenc_kostya: properly account for alpha
The packet buffer allocation considered as dct-coded, while it is
actually run-coded and thus requires a larger buffer.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 117bc8e6ffc744fedcf77edf2fdb33c964b83370)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-14 15:44:54 +02:00
Christophe Gisquet
60bfa9154d proresenc_kostya: report buffer overflow
If the allocated size, despite best efforts, is too small, exit
with the appropriate error.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 52b81ff4635c077b2bc8b8d3637d933b6629d803)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-12 02:40:15 +02:00
Christophe Gisquet
9794727ccd proresenc_kostya: remove unneeded parameters
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bf10f09bccdcfdb41b9f5bbae01d55961bfd0693)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-12 02:40:14 +02:00
Michael Niedermayer
11420649d0 avcodec/snow: fix null pointer dereference in cleanup after allocation failure
Fixes: snowf.avi
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9a162146ca6cc12ef7ad4a15164349482885962c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-11 15:45:42 +02:00
Michael Niedermayer
dbf5d7e5cd avcodec/huffyuvdec: fix overread checks
Fixes: ffvhuff_f.avi
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ba47d519e537299179d20b9a599c5824589a3f7a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-11 15:32:00 +02:00
Michael Niedermayer
6badd558ce avcodec/get_bits: add BITS_LEFT() for finding the bits left with an opened reader
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 11512d70facf42fb490168d33a5986448cf36074)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-11 15:31:59 +02:00
Michael Niedermayer
a1fe3b4150 avcodec/vc1dec: do not crash when flushing without an allocated frame
Fixes Ticket3837
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6801eb0a0981f113f5f09ed4799d9ae805af62a3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-11 02:26:29 +02:00
Michael Niedermayer
fc57959fd5 avcodec/iff: Factorize av_pix_fmt_desc_get() out
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9bcdb98839f3d883867d87c6425e67ed7bcc7149)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-10 22:40:50 +02:00
Michael Niedermayer
83956309cc avcodec/iff: check pixfmt for rgb8 / rgbn
Fixes out of array access

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3539d6c63a16e1b2874bb037a86f317449c58770)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-10 22:40:50 +02:00
Felix Abecassis
6928193493 h264: fix interpretation of interleved stereo modes
Column and row frame packing arrangements were inverted.

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
(cherry picked from commit a7e541c9926d531a100ba0d36f4e56956dd84651)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-10 22:40:50 +02:00
Anton Khirnov
a72b7286e6 cdgraphics: do not return 0 from the decode function
0 means no data consumed, so it can trigger an infinite loop in the
caller.

CC:libav-stable@libav.org
(cherry picked from commit c7d9b473e28238d4a4ef1b7e8b42c1cca256da36)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-10 22:40:50 +02:00
Michael Niedermayer
fdfe94f4b1 avcodec/wavpackenc: Fix log2sample() result value
Found-by: CSA
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e706fe764049b3f1ccf10ba9f686426a4c007906)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-10 22:40:50 +02:00
Michael Niedermayer
638c3aca64 avcodec/dvdsub_parser: print message if packet is smaller than the packet size field
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bcc898dd2643c883522ffa565be4b226ce798c78)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:04 +02:00
Michael Niedermayer
76d59f1b34 avcodec/dvdsub_parser: Check buf_size before reading 32bit packet size
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 81c1657a593b1c0f8e46fca00ead1d30ee1cd418)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:03 +02:00
Michael Niedermayer
aa0a8ef50e avcodec/dvdsub_parser: never return 0 when the input isnt 0
Fixes a infinite loop
Fixes Ticket3804

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfdb30d2f1241de9354a8efdbf8252d0f1a6f933)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:02 +02:00
Michael Niedermayer
d6173ae341 avcodec/mpeg12dec: fix support for interlaced mpeg2 with missing last slice
Fixes Ticket3809

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f95298c913899207344d668a6d5624cb2d2e480c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-01 21:20:43 +02:00
Michael Niedermayer
594b1fa961 libavcodec/h264_parser: Increase parse_history, fix huge resolutions
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0782fb6bcb32fe3ab956a99af4cc472ff81da0c2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-31 16:50:32 +02:00
Benoit Fouet
7a1262fca3 h264_mp4toannexb_bsf: account for consecutive IDR pictures.
If there are consecutive IDR pictures, then SPS/PPS should be prepended
to all of them, not only the first one.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bf428bb3145c4f0eef32f8ef00de0ee222b3e414)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-31 16:50:31 +02:00
Chris \"Koying\" Browet
6d2219e9f9 avcodec/h264_mp4toannexb_bsf: fix issue when sps/pps are already in the bistream
(cherry picked from commit ad91bf854b559f1afc42fc06f8a7dfbb75def5c8)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-31 16:50:26 +02:00
Michael Niedermayer
c61ac696e5 avcodec/avdct: Add avcodec_dct_get_class()
This should have been in the initial commit of AVDCT

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2f717be22a933fc8f53b2cb2540ffb3358896467)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-30 22:39:08 +02:00
Vittorio Giovara
ab1ea597bd g2meet: allow size changes within original sizes
(cherry picked from commit 14b4e64eabc84c5a5e57c8ccc56bbeb95380823b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-30 22:32:01 +02:00
Christophe Gisquet
65259b4d68 x86: hevc_mc: replace one lea by add
Should have been in 036f11bdb565.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-27 17:32:01 +02:00
Michael Niedermayer
fcc6568a10 avcodec: add avdct
This provides a public sustainable API/ABI for DCT functions.
Only externally used dct functions are included.
The structure is extensible without ABI issues compared to the
existing dct contexts.

See Mailing list and IRC log of 2014-07-26/27

Reviewed-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 932ff7095696a90fda83a8c6343b57f475b40951)

Conflicts:

	doc/APIchanges
2014-07-27 17:28:32 +02:00
Michael Niedermayer
cf92cc8751 avcodec/hevc: clear HEVClcList[i] on allocation
Fixes fate failure with --enable-memory-poisoning && make THREAD_TYPE=slice THREADS=7  fate-hevc-conformance-ENTP_C_Qualcomm_1

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-16 01:34:57 +02:00
James Almer
ad24256e7e diracdec: remove unused dsputil context
Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-16 00:11:18 +02:00
Michael Niedermayer
880dbe43ca avcodec/hevc: treat current_sps like sps_list
This simplifies the management of current_sps
Fixes Ticket3458

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-15 21:49:56 +02:00
Michael Niedermayer
0fc2045d5f avcodec/hevc_ps: prevent stale pointer in malloc failure case
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-15 21:49:56 +02:00
Michael Niedermayer
5bda0467d2 avcodec/mpegvideo_enc: make edge for interlaced mpeg2 encoding smaller
Fixes segfault
the size can probably be reduced further for the mpeg2 case

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-15 19:56:24 +02:00