410 Commits

Author SHA1 Message Date
Michael Niedermayer
342b6d3718 avcodec/mjpegdec: Do not try to detect last scan but apply idct after all scans for progressive jpeg
Fixes: IMG-20160418-WA0002.jpg

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit deaf58abf236e09fc9b97db29f1edd064e4b5ad4)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-05-02 23:59:14 +02:00
Michael Niedermayer
778ee32990 avcodec/mjpegdec: Fix decoding slightly odd progressive jpeg
Fixes: ebd58db6-dc86-11e5-91c2-59daeddf50c7.jpg

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c6f4720b8664e6e22eb5b3da6bb48ed5b113f746)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-29 14:23:40 +02:00
Michael Niedermayer
517c856d7f avcodec/mjpegdec: Check for end for both bytes in unescaping
Fixes assertion failure
Fixes: c40c779601b77dc6e19aaea0b04b9751/signal_sigabrt_7ffff6ae7cb7_5769_b94f6ec70caecb2d3d76b4771b109ac1.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 509c9e74e548139285f30ed8dcc9baf1d64359fa)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-31 00:24:28 +01:00
Michael Niedermayer
bb1263185b avcodec/mjpegdec: Fix negative shift
Fixes: mjpeg_left_shift.avi

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d86d7b2486cd5c31db8e820d8a89554abf19567e)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:23:55 +01:00
Andreas Cadhalpun
a5fb5a18e3 mjpegdec: extend check for incompatible values of s->rgb and s->ls
This can happen if s->ls changes from 0 to 1, but picture allocation is
skipped due to s->interlaced.

In that case ff_jpegls_decode_picture could be called even though the
s->picture_ptr frame has the wrong pixel format and thus a wrong
linesize, which results in a too small zero buffer being allocated.

This fixes an out-of-bounds read in ls_decode_line.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 7ea2db6eafa0a8a9497aab20be2cfc8742a59072)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:23:54 +01:00
Andreas Cadhalpun
a90967013b mjpegdec: consider chroma subsampling in size check
If the chroma components are subsampled, smaller buffers are allocated
for them. In that case the maximal block_offset for the chroma
components is not as large as for the luma component.

This fixes out of bounds writes causing segmentation faults or memory
corruption.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5adb5d9d894aa495e7bf9557b4c78350cbfc9d32)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-17 21:51:33 +01:00
Michael Niedermayer
dd6274d904 avcodec/mjpegdec: Reinitialize IDCT on BPP changes
Fixes misaligned access
Fixes: dc9262a469f6f315f74c087a7b3a7f35/signal_sigsegv_2e95bcd_9_9c0f9f4a9ba82aa9b3ab2b91ce4d5277.jpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cc35f6f4768ffe57cc4fcfa56ecb89aee409e3d5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-19 03:51:38 +01:00
Michael Niedermayer
4fec2df719 avcodec/mjpegdec: Check index in ljpeg_decode_yuv_scan() before using it
Fixes: 04715144ba237443010554be0d05343f/asan_heap-oob_1eafc76_1737_c685b48041a563461839e4e7ab97abb8.jpg
Fixes out of array access

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d24888ef19ba38b787b11d1ee091a3d94920c76a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-19 03:51:38 +01:00
Michael Niedermayer
4ef257f702 avcodec/mjpegdec: Fix decoding RGBA RCT LJPEG
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 055e56e9f76da3298f1b59bf5ea46f570e844600)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-19 03:51:38 +01:00
Michael Niedermayer
0a5b242422 avcodec/mjpegdec: fix len computation in ff_mjpeg_decode_dqt()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 81cf9108563510dee24f73b2c5d94a7bd07ff747)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-05-15 10:04:51 +02:00
Michael Niedermayer
08509c8f86 avcodec/mjpegdec: Skip blocks which are outside the visible area
Fixes out of array accesses
Fixes: ffmpeg_mjpeg_crash.avi

Found-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-11 03:53:30 +01:00
Michael Niedermayer
fabbfaa095 avcodec/mjpegdec: Check number of components for JPEG-LS
Fixes out of array accesses
Fixes: asan_heap-oob_1c1a4ea_1242_cov_2274415971_TESTcmyk.jpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-04 20:49:12 +01:00
Michael Niedermayer
afa92907f3 avcodec/mjpegdec: Check escape sequence validity
Fixes assertion failure
Fixes: asan_heap-oob_1c1a4ea_1242_cov_2274415971_TESTcmyk.jpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-04 20:14:18 +01:00
Michael Niedermayer
6c68522a2a Merge commit '809c3023b699c54c90511913d3b6140dd2436550'
* commit '809c3023b699c54c90511913d3b6140dd2436550':
  mjpegdec: check for pixel format changes

Conflicts:
	libavcodec/mjpegdec.c

See: 5c378d6a6df8243f06c87962b873bd563e58cd39
See: a2f680c7bc7642c687aeb4e14d00ac74833c7a09
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-19 12:49:23 +01:00
Anton Khirnov
809c3023b6 mjpegdec: check for pixel format changes
Fixes possible invalid memory access.

Based on code by Michael Niedermayer <michaelni@gmx.at>

CC: libav-stable@libav.org
Bug-ID: CVE-2014-8541
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
2014-12-19 08:01:46 +01:00
Michael Niedermayer
970a8f1c25 avcodec/mjpegdec: Fix integer overflow in shift
Fixes: signal_sigabrt_7ffff6ac7bb9_2683_cov_4120310995_m_ijpg.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-27 19:30:31 +01:00
Michael Niedermayer
0eecf40935 avcodec/mjpegdec: Fix context fields becoming inconsistent
Fixes out of array access
Fixes: asan_heap-oob_1ca4f85_2760_cov_144449187_miss_congeniality_pegasus_ljpg.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-25 13:54:20 +01:00
Michael Niedermayer
172d22a071 avcodec/mjpegdec: Add YUVA420 formats to *scale asserts
Fixes assertion failure
Fixes: signal_sigabrt_7ffff6ac7bb9_2042_cov_2593130068_ef1f8a057bb6056674fad92f6b8c0acd.jpg
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-25 01:42:39 +01:00
Michael Niedermayer
03a17f2bbf avcodec/mjpegdec: Print the number of bits in the unsupported pixel format error
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-25 01:34:32 +01:00
Michael Niedermayer
0bf416f262 avcodec/mjpegdec: Check for pixfmtid 0x42111100 || 0x24111100 with more than 8 bits
These cases are not supported yet

Fixes assertion failure
Fixes: signal_sigabrt_7ffff6ac7bb9_1_cov_1553101927_00.jpg
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-25 01:23:21 +01:00
Michael Niedermayer
2f6550bb9a avcodec/mjpegdec: fix pixfmtid 0x14111100
Fixes part of Ticket 2004
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-22 19:38:43 +01:00
Michael Niedermayer
4243415741 avcodec/mjpegdec: Support some subsampled GBR variants
Fixes Ticket4045

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-18 05:10:21 +01:00
Michael Niedermayer
960c573cc5 avcodec/mjpegdec: support pix fmt id 0x22111111
Fixes: 4163724_300.jpg

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-07 15:37:11 +01:00
Michael Niedermayer
c53a1507aa Merge commit '199d9f995da53fe2507821f6d96bbc692574e1a9'
* commit '199d9f995da53fe2507821f6d96bbc692574e1a9':
  mjpegdec: fix undefined shift

Conflicts:
	libavcodec/mjpegdec.c

See: b4329605289e25bb071ec1c1182bf25fc83b09aa
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-06 18:28:49 +01:00
Vittorio Giovara
199d9f995d mjpegdec: fix undefined shift
Add a comment to explain the code.

CC: libav-stable@libav.org
Bug-Id: CID 1194388
2014-11-06 10:44:46 -05:00
Michael Niedermayer
c5ffd7aee5 avcodec/mjpegdec: use FF_CEIL_RSHIFT() for width
No testcase known

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-22 02:28:54 +02:00
Michael Niedermayer
d5a3a20d1e avcodec/mjpegdec: simplify chroma_height calculation
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-21 21:36:17 +02:00
Michael Niedermayer
059c842818 avcodec/mjpegdec: Support 24111100 pix fmt id
Fixes 129533924_640.jpg

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-21 16:41:47 +02:00
Michael Niedermayer
0db1f2c2c7 avcodec/mjpegdec: sanity check bits
Fixes undefined shift
Fixes: asan_heap-oob_16668e9_2_asan_heap-oob_16668e9_346_miss_congeniality_pegasus_mjpg.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-12 00:28:07 +02:00
Michael Niedermayer
5c378d6a6d avcodec/mjpegdec: check bits per pixel for changes similar to dimensions
Fixes out of array accesses
Fixes: asan_heap-oob_16668e9_2_asan_heap-oob_16668e9_346_miss_congeniality_pegasus_mjpg.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 02:18:13 +02:00
Clément Bœsch
b96d864fd6 avcodec/mjpegdec: Fix chroma width rounding
Fixes vertical line at the right side
Fixes Ticket 3929

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-13 20:25:27 +02:00
Michael Niedermayer
1654ca7d4e avcodec/mjpegdec: fix rounding of chroma_height
Fixes green line at the bottom
Fixes Ticket3913

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-06 03:13:57 +02:00
Michael Niedermayer
f0d4f00f24 avcodec/mjpegdec: fix green line at the bottom with upscale v
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-31 15:33:30 +02:00
Michael Niedermayer
63a52ca134 avcodec/mjpegdec: fix green vertical line at the right with upscale h
Fixes Ticket3891

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-31 15:16:36 +02:00
Michael Niedermayer
5c7899a483 avcodec/mjpegdec: Support AV_PIX_FMT_YUV420P16 with upscale_h
Fixes assertion failure
Fixes: test42f.jpg
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-14 16:20:55 +02:00
Przemysław Sobala
c68098ba4a avcodec/mjpegdec: add pix_fmt: 0x14121200
Fixes: _15801_F.jpg

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-22 18:07:44 +02:00
Michael Niedermayer
ef7e8425e8 avcodec/mjpegdec: factorize some parts of the pix_fmt_id switch()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-01 23:28:18 +02:00
Michael Niedermayer
784e1cf76b avcodec/mjpegdec: handle luma upscale detection generically
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-01 23:26:19 +02:00
Michael Niedermayer
64d98dadc7 avcodec/mjpegdec: set upscale_h/upscale_v using generic code instead of hardcoding a list
Some code is left to handle corner cases

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-01 22:54:57 +02:00
Michael Niedermayer
7558e55345 avcodec/mjpegdec: Support pix_fmt_id==0x11222200
Fixes: 4858286_300.jpg

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-01 22:12:20 +02:00
Michael Niedermayer
cd417d947e avcodec/mjpegdec: fix width for non chroma in rescaling
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-01 22:09:19 +02:00
Michael Niedermayer
4e09300ffa mjpegdec: Support pix_fmt_id == 0x22112200
Fixes 4780490_300.jpg

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-01 21:47:54 +02:00
Michael Niedermayer
581b5f0b9b Merge commit 'e3fcb14347466095839c2a3c47ebecff02da891e'
* commit 'e3fcb14347466095839c2a3c47ebecff02da891e':
  dsputil: Split off IDCT bits into their own context

Conflicts:
	configure
	libavcodec/aic.c
	libavcodec/arm/Makefile
	libavcodec/arm/dsputil_init_arm.c
	libavcodec/arm/dsputil_init_armv6.c
	libavcodec/asvdec.c
	libavcodec/dnxhdenc.c
	libavcodec/dsputil.c
	libavcodec/dvdec.c
	libavcodec/dxva2_mpeg2.c
	libavcodec/intrax8.c
	libavcodec/mdec.c
	libavcodec/mjpegdec.c
	libavcodec/mjpegenc_common.h
	libavcodec/mpegvideo.c
	libavcodec/ppc/dsputil_altivec.h
	libavcodec/ppc/dsputil_ppc.c
	libavcodec/ppc/idctdsp.c
	libavcodec/x86/Makefile
	libavcodec/x86/dsputil_init.c
	libavcodec/x86/dsputil_mmx.c
	libavcodec/x86/dsputil_x86.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-01 15:22:11 +02:00
Diego Biurrun
e3fcb14347 dsputil: Split off IDCT bits into their own context 2014-06-30 07:58:46 -07:00
Derek Buitenhuis
2deb614272 mjpegdec: Properly set the context colorspace info
The JPEG spec requires it to be this.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2014-06-24 20:42:40 +01:00
Derek Buitenhuis
c11043aca7 mjpegdec: Properly set the context colorspace info
The JPEG spec requires it to be this.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2014-06-24 20:40:44 +01:00
Michael Niedermayer
0dceefc5fa Merge commit '9e500efdbe0deeff1602500ebc229a0a6b6bb1a2'
* commit '9e500efdbe0deeff1602500ebc229a0a6b6bb1a2':
  Add av_image_check_sar() and use it to validate SAR

Conflicts:
	libavcodec/dpx.c
	libavcodec/dvdec.c
	libavcodec/ffv1dec.c
	libavcodec/utils.c
	libavutil/version.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-20 22:20:28 +02:00
Justin Ruggles
9e500efdbe Add av_image_check_sar() and use it to validate SAR 2014-06-20 10:39:33 -04:00
Michael Niedermayer
2b05db4f81 Merge commit 'e74433a8e6fc00c8dbde293c97a3e45384c2c1d9'
* commit 'e74433a8e6fc00c8dbde293c97a3e45384c2c1d9':
  dsputil: Split clear_block*/fill_block* off into a separate context

Conflicts:
	configure
	libavcodec/asvdec.c
	libavcodec/dnxhddec.c
	libavcodec/dnxhdenc.c
	libavcodec/dsputil.h
	libavcodec/eamad.c
	libavcodec/intrax8.c
	libavcodec/mjpegdec.c
	libavcodec/ppc/dsputil_ppc.c
	libavcodec/vc1dec.c
	libavcodec/x86/dsputil_init.c
	libavcodec/x86/dsputil_mmx.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-19 04:54:38 +02:00
Diego Biurrun
e74433a8e6 dsputil: Split clear_block*/fill_block* off into a separate context 2014-06-18 14:07:23 -07:00