22645 Commits

Author SHA1 Message Date
Anton Khirnov
7e513d85e8 h264: check that execute_decode_slices() is not called too many times
Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 9eef9eb3014b2ed9c3ff4aac510a9f04edb555cf)
2014-04-14 16:55:38 -04:00
Anton Khirnov
b6a0f5cde8 h264: do not use 422 functions for monochrome
Fixes invalid memory access.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:38 -04:00
Anton Khirnov
1147d39bca h264: reset data_partitioning if decoding the slice header for NAL_DPA fails
If it was set before then we can end up trying to decode a slice without
a valid slice header, which can lead to invalid memory access.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 58312b2472d3a44d7458865c459d59ef2e02bf1a)
2014-04-14 16:55:38 -04:00
Anton Khirnov
30e58e65e5 h264_refs: make sure not to write over the bounds of the default ref list
Fixes invalid writes.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:38 -04:00
Anton Khirnov
175b53d051 h264: limit allowed pred modes in ff_h264_check_intra_pred_mode() to 3
Higher modes are not allowed for 16x16/chroma, which is what this
function is used for. Otherwise this function would return 0 (vertical
prediction) for invalid higher modes, which could result in invalid
reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:38 -04:00
Anton Khirnov
af40847681 h264: check buffer size before accessing it
Fixes invalid reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:37 -04:00
Janne Grunau
7a2254ab26 arm: hpeldsp: fix put_pixels8_y2_{,no_rnd_}armv6
The overread avoidance fix in cbddee1cca0ebd01e8c5aa694d31228eb4de4b41
broke the computation for the last row since it prevented the safe
reading from the height+1-th row.
2014-03-08 19:05:06 +01:00
Michael Niedermayer
d891fc9181 avcodec/utvideoenc: fix slice_bits size
Fixes assertion failure

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0a8c90202bb906747168a698b6837496f82c717c)

Conflicts:

	libavcodec/utvideoenc.c
(cherry picked from commit 57522ca79cc38c279123596d3288ddbf56fa8903)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-07 04:04:08 +01:00
Michael Niedermayer
50169747db avcodec/snow: split block clipping checks
Fixes out of array read
Fixes: d4476f68ca1c1c57afbc45806f581963-asan_heap-oob_2266b27_8607_cov_4044577381_snow_chroma_bug.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 61d59703c91869f4e5cdacd8d6be52f8b89d4ba4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-07 04:04:08 +01:00
Michael Niedermayer
ff5647b253 avcodec/ansi: fix integer overflow
Fixes out of array read
Fixes: 5f9698e86d92f19bb08d54ff0d57027f-signal_sigsegv_b30756_3795_cov_2693691257_ansi256.ans
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d42ec8433c687fcbccefa51a7716d81920218e4f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-07 04:04:07 +01:00
Michael Niedermayer
0c58c165ab avcodec/msrle: use av_image_get_linesize() to calculate the linesize
Fixes out of array access
Fixes: 14a74a0a2dc67ede543f0e35d834fbbe-asan_heap-oob_49572c_556_cov_215466444_44_001_engine_room.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c919e1ca2ecfc47d796382973ba0e48b8f6f92a2)

Conflicts:

	libavcodec/msrle.c
(cherry picked from commit bc1c8ec5e65098fd2ccd8456f667151dfc9cda42)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-07 04:04:07 +01:00
Michael Niedermayer
b0b02a2101 avcodec/wmalosslessdec: fix mclms_coeffs* array size
Fixes corruption of context
Fixes: 8835659dde6a4f7dcdf341de6a45c6c8-signal_sigsegv_1dce67b_4564_cov_2504444599_classical_22_16_1_14000_v3c_0_extend_0_29.wma
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ec9578d54d09b64bf112c2bf7a34b1ef3b93dbd3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-07 04:04:07 +01:00
Paul B Mahol
aeccfca268 avcodec/libopusenc: change default frame duration to 20 ms
20 ms is used by libopus encoder.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit 74906d3727ec3bd9b7b28dfa7a98ff6e8cf8b6d7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-07 04:04:07 +01:00
Michael Niedermayer
15d96c605b avcodec/jpeglsdec: check err value for ls_get_code_runterm()
Fixes infinite loop
Fixes Ticket3086

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cc0e47b55096361723b364afa43b79a3f5619cdc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-07 04:04:07 +01:00
Michael Niedermayer
8b0880ff1b avcodec/avpacket/av_packet_split_side_data: ensure that side data padding is initialized
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 240fd8c96f59ebe9dcfc4152a1086cd3f63400c0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1e48318802b3caa493a40c0584afc30cc866d9d0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-07 04:04:07 +01:00
Michael Niedermayer
f1f06710c4 Merge remote-tracking branch 'qatar/release/9' into release/1.1
* qatar/release/9:
  arm: hpeldsp: prevent overreads in armv6 asm

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-07 03:59:18 +01:00
Michael Niedermayer
993c2a256d Merge commit '798c715f4fa5cde37456af6202a32ee62cfb96d9' into release/1.1
* commit '798c715f4fa5cde37456af6202a32ee62cfb96d9':
  configure: enable PIC on s390(x)
  ituh263: reject b-frame with pp_time = 0
  lagarith: reallocate rgb_planes when needed
  truemotion1: check the header size
  shorten: pad the internal bitstream buffer
  samplefmt: avoid integer overflow in av_samples_get_buffer_size()

Conflicts:
	libavcodec/lagarith.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-07 03:54:51 +01:00
Michael Niedermayer
c9fef27deb Merge commit '8883b5f85bfe35509633bc590d19b6a1b495690e' into release/1.1
* commit '8883b5f85bfe35509633bc590d19b6a1b495690e':
  h264: Fix a typo from the previous commit
  h264: Lower bound check for slice offsets
  Add missing header to fix compilation after d2a0654
  Prepare for 9.12 RELEASE
  configure: Add missing dependency of Snow decoder on videodsp
  rpza: limit the number of blocks to the total remaining blocks in the frame

Conflicts:
	RELEASE

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-07 03:10:13 +01:00
Janne Grunau
460b948283 arm: hpeldsp: prevent overreads in armv6 asm
Based on a patch by Russel King <rmk+libav@arm.linux.org.uk>

Bug-Id: 646
CC: libav-stable@libav.org
2014-03-05 19:45:00 +01:00
Keiji Costantini
aa2a3ca27a ituh263: reject b-frame with pp_time = 0
Avoid a division by 0 in ff_mpeg4_set_one_direct_mv.

Sample-Id: 00000168-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
(cherry picked from commit 9514440337875e0c63b409abcd616b68c518283f)
(cherry picked from commit 5df52b0131d3d4d804ad6e221bc9a2cd8b201ef2)
2014-03-02 11:45:32 -05:00
Luca Barbato
8883b5f85b h264: Fix a typo from the previous commit
f777504f640260337974848c7d5d7a3f064bbb45 changed a - in +

CC: libav-stable@libav.org
(cherry picked from commit d922c5a5fbaf0b6c73bd8c81ae059bc6e406961c)
(cherry picked from commit 3ce77e04c2ca4b9e7fa6b94b51e8d7c5f188da86)
2014-02-28 23:05:53 -05:00
Anton Khirnov
b4d72f901c lagarith: reallocate rgb_planes when needed
Fixes invalid writes on pixel format changes.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 4c3e1956ee35fdcc5ffdb28782050164b4623c0b)
2014-02-28 23:05:53 -05:00
Vittorio Giovara
7c70cee29c h264: Lower bound check for slice offsets
And use the value from the specification.

Sample-Id: 00000451-google
Found-by: Mateusz j00ru Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit f777504f640260337974848c7d5d7a3f064bbb45)
(cherry picked from commit 5bd083d0216d9ee649039c84999fb61386536ac1)

Conflicts:
	libavcodec/h264.c
2014-02-28 23:05:53 -05:00
Anton Khirnov
13fd80837f truemotion1: check the header size
Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 2240e2078d53d3cfce8ff1dda64e58fa72038602)
2014-02-28 23:05:53 -05:00
Anton Khirnov
f1a7bfea41 shorten: pad the internal bitstream buffer
Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 1713eec29add37b654ec6bf262b843d139c1ffc6)
2014-02-28 23:05:53 -05:00
Anton Khirnov
8ba514117b Add missing header to fix compilation after d2a0654
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2014-02-28 23:05:08 -05:00
Anton Khirnov
d2a065437a rpza: limit the number of blocks to the total remaining blocks in the frame
Fixes invalid writes.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 77bb0004bbe18f1498cfecdc68db5f10808b6599)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-02-14 11:31:35 +01:00
Michael Niedermayer
08dde7567d Merge remote-tracking branch 'qatar/release/9' into release/1.1
* qatar/release/9:
  Update Changelog for 9.11
  oggparseogm: check timing variables
  mathematics: remove asserts from av_rescale_rnd()
  vc1: Always reset numref when parsing a new frame header.
  h264: reset num_reorder_frames if it is invalid

Conflicts:
	Changelog
	libavcodec/vc1.c
	libavutil/mathematics.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-04 06:41:15 +01:00
Michael Niedermayer
cb8180885f Merge commit '62ed6da016b789eee00e0fff517df4a254e12e5d' into release/1.1
* commit '62ed6da016b789eee00e0fff517df4a254e12e5d':
  h264: check that an IDR NAL only contains I slices
  mov: Free an earlier allocated array if allocating a new one

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-04 06:32:48 +01:00
Michael Niedermayer
e2781db62a Merge commit '44079902c49e526f464bb4eb855665e1af867e91' into release/1.1
* commit '44079902c49e526f464bb4eb855665e1af867e91':
  mov: Free intermediate arrays in the normal cleanup function
  segafilm: fix leaks if reading the header fails
  h264_cavlc: check the size of the intra PCM data.
  h263: Check init_get_bits return value
  cavsdec: check ff_get_buffer() return value

Conflicts:
	libavcodec/cavsdec.c
	libavcodec/h263dec.c
	libavformat/mov.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-04 06:23:46 +01:00
Michael Niedermayer
9ac7d8f85d Merge commit 'c85e5f13f6ac9c4c90125e7671d89009e57f9df9' into release/1.1
* commit 'c85e5f13f6ac9c4c90125e7671d89009e57f9df9':
  cavs: Check for negative cbp
  avi: DV in AVI must be considered single stream
  vmnc: Check the cursor dimensions
  vmnc: Port to bytestream2

Conflicts:
	libavcodec/cavsdec.c
	libavcodec/vmnc.c
	libavformat/avidec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-04 06:05:36 +01:00
Michael Niedermayer
fd856693de Merge commit 'f1476459b7013d306eb911573f1dc81e74ccd082' into release/1.1
* commit 'f1476459b7013d306eb911573f1dc81e74ccd082':
  vmnc: K&R formatting cosmetics
  flashsv: Check diff_start diff_height values
  dsputil/pngdsp: fix signed/unsigned type in end comparison

Conflicts:
	libavcodec/dsputil.c
	libavcodec/flashsv.c
	libavcodec/vmnc.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-04 05:43:42 +01:00
Michael Niedermayer
af74599e66 avcodec/vc1: reset fcm/field_mode in non advanced header parsing
Fixes NULL pointer dereference
Fixes: signal_sigsegv_1ab8bf4_2847_cov_4254117347_SA10091.vc1
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b51e9354772de446e8196dabf9aad1567b22f74d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-04 04:50:17 +01:00
Michael Niedermayer
74821341b9 avcodec/takdec: always check bits_per_raw_sample
Fixes out of array access
Fixes: asan_heap-oob_19c7a94_6470_cov_1453611734_luckynight-partial.tak
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f58eab151214d2d35ff0973f2b3e51c5eb372da4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-04 04:50:17 +01:00
Michael Niedermayer
7adf4a92a1 avcodec/vmnc: Check that rectangles are within the picture
Prevents out of array accesses with CODEC_FLAG_EMU_EDGE

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6ba02602aa7fc7d38db582e75b8b093fb3c1608d)

Conflicts:

	libavcodec/vmnc.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7c17207ab9acfaa934e8feb8fba90765c9d0b989)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-04 04:50:17 +01:00
Michael Niedermayer
e04f68f7c5 dnxhdenc: fix mb_rc size
Fixes out of array access with RC_VARIANCE set to 0

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f1caaa1c61310beba705957e6366f0392a0b005b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-04 04:50:17 +01:00
Anton Khirnov
62ed6da016 h264: check that an IDR NAL only contains I slices
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 8b2e5e42bb9d6a59ede5af2e6df4aaf7750d1195)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2014-02-01 23:51:46 -05:00
Michael Niedermayer
3cc8d9bc1f vc1: Always reset numref when parsing a new frame header.
Fixes an issue where the B-frame coding mode switches from interlaced
fields to interlaced frames, causing incorrect decisions in the motion
compensation code and resulting in visual artifacts.

CC: libav-stable@libav.org
Signed-off-by: Tim Walker <tdskywalker@gmail.com>
(cherry picked from commit dd2d0039b6405dc724e4fef0d5b8f49530eea3aa)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2014-02-01 23:51:46 -05:00
Anton Khirnov
299c5dcfb0 h264: reset num_reorder_frames if it is invalid
An invalid VUI is not considered a fatal error, so the SPS containing it
may still be used. Leaving an invalid value of num_reorder_frames there
can result in writing over the bounds of H264Context.delayed_pic.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 9ecabd7892ff073ae60ded3fc0a1290f5914ed5c)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:
	libavcodec/h264_ps.c
2014-02-01 23:51:46 -05:00
Luca Barbato
c85e5f13f6 cavs: Check for negative cbp
Sample-Id: 00000647-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2014-02-01 14:59:50 -05:00
Luca Barbato
f1476459b7 vmnc: K&R formatting cosmetics
Signed-off-by: Diego Biurrun <diego@biurrun.de>
2014-02-01 14:59:50 -05:00
Anton Khirnov
b5275ca1a8 h264_cavlc: check the size of the intra PCM data.
Fixes invalid reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-02-01 14:59:50 -05:00
Michael Niedermayer
d9c82cea11 h263: Check init_get_bits return value
And use init_get_bits8 to check for integer overflows while at it.

CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-02-01 14:59:50 -05:00
Luca Barbato
4b24eb1a03 vmnc: Check the cursor dimensions
And manage the reallocation failure path.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 5e992a4682d2c09eed3839c6cacf70db3b65c2f4)
2014-02-01 14:59:50 -05:00
Anton Khirnov
969028870c cavsdec: check ff_get_buffer() return value
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-02-01 14:59:50 -05:00
Luca Barbato
9f9e773881 vmnc: Port to bytestream2
Fix some buffer overreads.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2014-02-01 14:59:50 -05:00
Michael Niedermayer
10d48fe6d3 flashsv: Check diff_start diff_height values
Fix out of array accesses.

Found-by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Adresses: CVE-2013-7015
(cherry picked from commit 57070b1468edc6ac8cb3696c817f3c943975d4c1)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2014-02-01 13:56:58 -05:00
Michael Niedermayer
af9799790d dsputil/pngdsp: fix signed/unsigned type in end comparison
Fixes out of array accesses and integer overflows.

(cherry picked from commit d1916d13e28b87f4b1b214231149e12e1d536b4b)
Adresses: CVE-2013-7010, CVE-2013-7014

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2014-02-01 13:53:41 -05:00
Michael Niedermayer
6fa9741357 avcodec/aacdec: Dont fail if channels arent known yet
Fixes Ticket3312

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 676a395ab903cac623c5d6ddd0928c789e08a59e)

Conflicts:
	libavcodec/aacdec.c
2014-01-19 14:58:22 +01:00
Michael Niedermayer
bb26a88193 avcodec/mjpegdec: Dont treat the lack of a startcode differently from end of the bitstream
Fixes Ticket3303

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 31e703e899bee74c50efd8eb62c3d012ef5ab26d)
2014-01-19 14:57:56 +01:00