397 Commits

Author SHA1 Message Date
Michael Niedermayer
fa9873cce8 avcodec/mjpegdec: Check for end for both bytes in unescaping
Fixes assertion failure
Fixes: c40c779601b77dc6e19aaea0b04b9751/signal_sigabrt_7ffff6ae7cb7_5769_b94f6ec70caecb2d3d76b4771b109ac1.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 509c9e74e548139285f30ed8dcc9baf1d64359fa)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-02-01 02:12:22 +01:00
Michael Niedermayer
5c0d8a8387 avcodec/mjpegdec: Fix negative shift
Fixes: mjpeg_left_shift.avi

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d86d7b2486cd5c31db8e820d8a89554abf19567e)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-31 00:25:20 +01:00
Andreas Cadhalpun
46fcc2ba55 mjpegdec: extend check for incompatible values of s->rgb and s->ls
This can happen if s->ls changes from 0 to 1, but picture allocation is
skipped due to s->interlaced.

In that case ff_jpegls_decode_picture could be called even though the
s->picture_ptr frame has the wrong pixel format and thus a wrong
linesize, which results in a too small zero buffer being allocated.

This fixes an out-of-bounds read in ls_decode_line.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 7ea2db6eafa0a8a9497aab20be2cfc8742a59072)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-01-28 02:15:49 +01:00
Andreas Cadhalpun
073fcfe358 mjpegdec: consider chroma subsampling in size check
If the chroma components are subsampled, smaller buffers are allocated
for them. In that case the maximal block_offset for the chroma
components is not as large as for the luma component.

This fixes out of bounds writes causing segmentation faults or memory
corruption.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5adb5d9d894aa495e7bf9557b4c78350cbfc9d32)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-07 01:34:16 +01:00
Michael Niedermayer
5d9bee34f9 avcodec/mjpegdec: Reinitialize IDCT on BPP changes
Fixes misaligned access
Fixes: dc9262a469f6f315f74c087a7b3a7f35/signal_sigsegv_2e95bcd_9_9c0f9f4a9ba82aa9b3ab2b91ce4d5277.jpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cc35f6f4768ffe57cc4fcfa56ecb89aee409e3d5)

Conflicts:

	libavcodec/mjpegdec.c
(cherry picked from commit f82c4777ee7a319fe2aa36f413a61943313b4abc)
2015-12-06 12:40:49 +01:00
Michael Niedermayer
2f89546333 avcodec/mjpegdec: Check index in ljpeg_decode_yuv_scan() before using it
Fixes: 04715144ba237443010554be0d05343f/asan_heap-oob_1eafc76_1737_c685b48041a563461839e4e7ab97abb8.jpg
Fixes out of array access

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d24888ef19ba38b787b11d1ee091a3d94920c76a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 12:40:49 +01:00
Michael Niedermayer
7cdd319b01 avcodec/mjpegdec: Fix decoding RGBA RCT LJPEG
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 055e56e9f76da3298f1b59bf5ea46f570e844600)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 12:40:49 +01:00
Michael Niedermayer
748194b58b avcodec/mjpegdec: fix len computation in ff_mjpeg_decode_dqt()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 81cf9108563510dee24f73b2c5d94a7bd07ff747)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-01 23:25:20 +02:00
Michael Niedermayer
492818d724 avcodec/mjpegdec: Skip blocks which are outside the visible area
Fixes out of array accesses
Fixes: ffmpeg_mjpeg_crash.avi

Found-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 08509c8f86626815a3e9e68d600d1aacbb8df4bf)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-13 20:48:08 +01:00
Michael Niedermayer
0882212298 avcodec/mjpegdec: Check number of components for JPEG-LS
Fixes out of array accesses
Fixes: asan_heap-oob_1c1a4ea_1242_cov_2274415971_TESTcmyk.jpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fabbfaa095660982cc0bc63242c459561fa37037)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-13 20:48:08 +01:00
Michael Niedermayer
3531ff8db3 avcodec/mjpegdec: Check escape sequence validity
Fixes assertion failure
Fixes: asan_heap-oob_1c1a4ea_1242_cov_2274415971_TESTcmyk.jpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit afa92907f3c6a0c3bdad766ec8d938ee17ee1c9e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-13 20:48:08 +01:00
Michael Niedermayer
81754d8f8b Merge commit 'aa7a19b41774ce5f8a4e43f3692a4f9d90aa5c92' into release/2.4
* commit 'aa7a19b41774ce5f8a4e43f3692a4f9d90aa5c92':
  mjpegdec: check for pixel format changes

Conflicts:
	libavcodec/mjpegdec.c

See: 5c378d6a6df8243f06c87962b873bd563e58cd39
See: a2f680c7bc7642c687aeb4e14d00ac74833c7a09
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-20 11:50:12 +01:00
Anton Khirnov
aa7a19b417 mjpegdec: check for pixel format changes
Fixes possible invalid memory access.

Based on code by Michael Niedermayer <michaelni@gmx.at>

CC: libav-stable@libav.org
Bug-ID: CVE-2014-8541
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 809c3023b699c54c90511913d3b6140dd2436550)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-12-20 10:51:40 +01:00
Michael Niedermayer
18dba3d80d avcodec/mjpegdec: Fix integer overflow in shift
Fixes: signal_sigabrt_7ffff6ac7bb9_2683_cov_4120310995_m_ijpg.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 970a8f1c256f08d2f6414d573a54f2fa035c8e7a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:36 +01:00
Michael Niedermayer
8524009161 avcodec/mjpegdec: Fix context fields becoming inconsistent
Fixes out of array access
Fixes: asan_heap-oob_1ca4f85_2760_cov_144449187_miss_congeniality_pegasus_ljpg.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0eecf40935b22644e6cd74c586057237ecfd6844)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:36 +01:00
Michael Niedermayer
47f345de1d avcodec/mjpegdec: Check for pixfmtid 0x42111100 || 0x24111100 with more than 8 bits
These cases are not supported yet

Fixes assertion failure
Fixes: signal_sigabrt_7ffff6ac7bb9_1_cov_1553101927_00.jpg
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0bf416f2628137e5389050fa323c329692dd4ba6)

Conflicts:

	libavcodec/mjpegdec.c
2014-11-30 21:40:36 +01:00
Michael Niedermayer
6f5c505109 avcodec/mjpegdec: check bits per pixel for changes similar to dimensions
Fixes out of array accesses
Fixes: asan_heap-oob_16668e9_2_asan_heap-oob_16668e9_346_miss_congeniality_pegasus_mjpg.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c378d6a6df8243f06c87962b873bd563e58cd39)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:28 +02:00
Clément Bœsch
b96d864fd6 avcodec/mjpegdec: Fix chroma width rounding
Fixes vertical line at the right side
Fixes Ticket 3929

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-13 20:25:27 +02:00
Michael Niedermayer
1654ca7d4e avcodec/mjpegdec: fix rounding of chroma_height
Fixes green line at the bottom
Fixes Ticket3913

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-06 03:13:57 +02:00
Michael Niedermayer
f0d4f00f24 avcodec/mjpegdec: fix green line at the bottom with upscale v
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-31 15:33:30 +02:00
Michael Niedermayer
63a52ca134 avcodec/mjpegdec: fix green vertical line at the right with upscale h
Fixes Ticket3891

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-31 15:16:36 +02:00
Michael Niedermayer
5c7899a483 avcodec/mjpegdec: Support AV_PIX_FMT_YUV420P16 with upscale_h
Fixes assertion failure
Fixes: test42f.jpg
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-14 16:20:55 +02:00
Przemysław Sobala
c68098ba4a avcodec/mjpegdec: add pix_fmt: 0x14121200
Fixes: _15801_F.jpg

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-22 18:07:44 +02:00
Michael Niedermayer
ef7e8425e8 avcodec/mjpegdec: factorize some parts of the pix_fmt_id switch()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-01 23:28:18 +02:00
Michael Niedermayer
784e1cf76b avcodec/mjpegdec: handle luma upscale detection generically
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-01 23:26:19 +02:00
Michael Niedermayer
64d98dadc7 avcodec/mjpegdec: set upscale_h/upscale_v using generic code instead of hardcoding a list
Some code is left to handle corner cases

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-01 22:54:57 +02:00
Michael Niedermayer
7558e55345 avcodec/mjpegdec: Support pix_fmt_id==0x11222200
Fixes: 4858286_300.jpg

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-01 22:12:20 +02:00
Michael Niedermayer
cd417d947e avcodec/mjpegdec: fix width for non chroma in rescaling
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-01 22:09:19 +02:00
Michael Niedermayer
4e09300ffa mjpegdec: Support pix_fmt_id == 0x22112200
Fixes 4780490_300.jpg

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-01 21:47:54 +02:00
Michael Niedermayer
581b5f0b9b Merge commit 'e3fcb14347466095839c2a3c47ebecff02da891e'
* commit 'e3fcb14347466095839c2a3c47ebecff02da891e':
  dsputil: Split off IDCT bits into their own context

Conflicts:
	configure
	libavcodec/aic.c
	libavcodec/arm/Makefile
	libavcodec/arm/dsputil_init_arm.c
	libavcodec/arm/dsputil_init_armv6.c
	libavcodec/asvdec.c
	libavcodec/dnxhdenc.c
	libavcodec/dsputil.c
	libavcodec/dvdec.c
	libavcodec/dxva2_mpeg2.c
	libavcodec/intrax8.c
	libavcodec/mdec.c
	libavcodec/mjpegdec.c
	libavcodec/mjpegenc_common.h
	libavcodec/mpegvideo.c
	libavcodec/ppc/dsputil_altivec.h
	libavcodec/ppc/dsputil_ppc.c
	libavcodec/ppc/idctdsp.c
	libavcodec/x86/Makefile
	libavcodec/x86/dsputil_init.c
	libavcodec/x86/dsputil_mmx.c
	libavcodec/x86/dsputil_x86.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-01 15:22:11 +02:00
Diego Biurrun
e3fcb14347 dsputil: Split off IDCT bits into their own context 2014-06-30 07:58:46 -07:00
Derek Buitenhuis
2deb614272 mjpegdec: Properly set the context colorspace info
The JPEG spec requires it to be this.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2014-06-24 20:42:40 +01:00
Derek Buitenhuis
c11043aca7 mjpegdec: Properly set the context colorspace info
The JPEG spec requires it to be this.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2014-06-24 20:40:44 +01:00
Michael Niedermayer
0dceefc5fa Merge commit '9e500efdbe0deeff1602500ebc229a0a6b6bb1a2'
* commit '9e500efdbe0deeff1602500ebc229a0a6b6bb1a2':
  Add av_image_check_sar() and use it to validate SAR

Conflicts:
	libavcodec/dpx.c
	libavcodec/dvdec.c
	libavcodec/ffv1dec.c
	libavcodec/utils.c
	libavutil/version.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-20 22:20:28 +02:00
Justin Ruggles
9e500efdbe Add av_image_check_sar() and use it to validate SAR 2014-06-20 10:39:33 -04:00
Michael Niedermayer
2b05db4f81 Merge commit 'e74433a8e6fc00c8dbde293c97a3e45384c2c1d9'
* commit 'e74433a8e6fc00c8dbde293c97a3e45384c2c1d9':
  dsputil: Split clear_block*/fill_block* off into a separate context

Conflicts:
	configure
	libavcodec/asvdec.c
	libavcodec/dnxhddec.c
	libavcodec/dnxhdenc.c
	libavcodec/dsputil.h
	libavcodec/eamad.c
	libavcodec/intrax8.c
	libavcodec/mjpegdec.c
	libavcodec/ppc/dsputil_ppc.c
	libavcodec/vc1dec.c
	libavcodec/x86/dsputil_init.c
	libavcodec/x86/dsputil_mmx.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-19 04:54:38 +02:00
Diego Biurrun
e74433a8e6 dsputil: Split clear_block*/fill_block* off into a separate context 2014-06-18 14:07:23 -07:00
Michael Niedermayer
0545ef7116 avcodec/mjpegdec: Improve intel jpeg flip heuristic
Fixes Ticket3698

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-06 01:32:22 +02:00
Michael Niedermayer
149be91374 avcodec/mjpegdec: request a AMV sample with non mod 16 height
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-01 04:00:47 +02:00
Michael Niedermayer
ec33f59fed avcodec/mjpegdec: Support pix_fmt_id== 0x42111100
Fixes: 538782_300.jpg
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-05-28 16:36:04 +02:00
Michael Niedermayer
aff352be63 avcodec/mjpegdec: zero gb to silence warning about it being possibly uninitialized
The code is not speed relevant, also its more robust if the pointers are NULL instead of random.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-05-08 17:37:40 +02:00
Michael Niedermayer
8532566e80 avcodec/mjpegdec: use av_mallocz_array()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-30 18:26:28 +02:00
Michael Niedermayer
e31727bd53 avcodec/mjpegdec: make type of shift unsigned to avoid undefined behavior
Found-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-28 00:22:38 +02:00
Michael Niedermayer
b432960528 avcodec/mjpegdec: Fix undefined shift
Fixes CID1194388

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-27 05:32:56 +02:00
Carl Eugen Hoyos
ced0d6c14d Use correct msvc type specifiers for ptrdiff_t and size_t.
The Windows runtime aborts if it finds %t or %z.
Fixes ticket #3472.

Reviewed-by: Ronald Bultje
2014-04-24 18:01:30 +02:00
Justin Ruggles
591c0c26b5 mjpeg: cosmetics: indentation
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2014-04-18 12:41:17 -04:00
Justin Ruggles
39ef000e19 mjpeg: Do not fail jpeg decoding on bad EXIF data.
It is not required to correctly decode the image.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2014-04-18 12:41:10 -04:00
Thilo Borgmann
8c161d7884 lavc/exif: Make EXIF IFD decoding part of private API/ABI.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-05 03:31:56 +02:00
Michael Niedermayer
254f653b24 avcodec/jpeglsdec: add PAL8 support
Fixes Ticket3478

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-01 13:43:44 +02:00
Michael Niedermayer
d56c373391 avcodec/mjpegdec: fix cmyk 420 with adobe_transform == 2
Fixes Ticket3516

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-01 01:14:26 +02:00