144 Commits

Author SHA1 Message Date
Michael Niedermayer
7a9f4f2816 avcodec/jpeg2000dec: Check that coords match before applying ICT
This avoid potential out of array accesses

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 12ba1b2b4d5592c0e27b0fcc83db929e8d6a8eee)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-10 02:13:12 +02:00
Michael Niedermayer
32f568f82c avcodec/jpeg2000dec: fix boolean operator
Fixes CID1271791 #7-6

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f8f155a18ac454e7ff3312e0e0c3a70eb4359143)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-10 02:13:11 +02:00
Michael Niedermayer
ca5d6c615e avcodec/jpeg2000dec: fix error detection in pix_fmt_match()
Fixes out of array accesses with CODEC_FLAG_EMU_EDGE

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8001e9f7d17e90b4b0898ba64e3b8bbd716c513c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-21 17:05:16 +01:00
Michael Niedermayer
bd9dcb411d avcodec/jpeg2000dec: Check precno before using it in JPEG2000_PGOD_CPRL
Fixes out of array reads
Fixes: asan_heap-oob_f0de57_6823_mjp2.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3d5a5e86be2a65e33c34ab3ad7923f54e8e49c1d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:47 +01:00
Michael Niedermayer
b545d11d49 avcodec/jpeg2000dec: non zero image offsets are not supported
Fixes out of array accesses
Fixes Ticket3080
Found-by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 780669ef7c23c00836a24921fcc6b03be2b8ca4a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:47 +01:00
Michael Niedermayer
47f8497837 avcodec/jpeg2000dec: fix context consistency with too large lowres
Fixes out of array accesses
Fixes Ticket2898

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a1b9004b768bef606ee98d417bceb9392ceb788d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:46 +01:00
Michael Niedermayer
93f26b7992 avcodec/jpeg2000dec: prevent out of array accesses in pixel addressing
Fixes Ticket2921

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fe448cd28d674c3eff3072552eae366d0b659ce9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:46 +01:00
Michael Niedermayer
f684bbf224 avcodec/jpeg2000dec: check transform equality in MCT
Fixes null pointer dereference
Fixes Ticket2843

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ac3b01a9c0607961f4540fe62040833870f5deb1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:46 +01:00
Michael Niedermayer
e2eb0d2326 avcodec/jpeg2000dec: Check cdx/y values more carefully
Some invalid values where not handled correctly in the later pixel
format matching code.
Fixes out of array accesses
Fixes Ticket2848

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8bb11c3ca77b52e05a9ed1496a65f8a76e6e2d8f)

Conflicts:

	libavcodec/jpeg2000dec.c
2013-08-29 02:13:25 +02:00
Michael Niedermayer
3e817d91ef jpeg2000: check log2_cblk dimensions
Fixes out of array access
Fixes Ticket2895

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9a271a9368eaabf99e6c2046103acb33957e63b7)
2013-08-25 15:57:13 +02:00
Michael Niedermayer
d73ce6cb56 jpeg2000dec: Support non subsampled 9-16bit planar pixel formats
This applies changes similar to fc6de70c44be05eb0368ab519bfb790431d8dee5
to the >8bit codepath

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1434df3b93fde086be729d174ffbbee1e25792b3)
2013-07-28 04:09:21 +02:00
Michael Niedermayer
9a6d3eee59 jpeg2000dec: silence unused variable warning
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit db33010483a024f1343bfb516415fe9ef77e82d9)
2013-07-28 04:09:11 +02:00
Michael Niedermayer
8b221d60fa jpeg2000dec: Support non subsampled 8bit planar pixel formats
Fixes file2.jp2

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fc6de70c44be05eb0368ab519bfb790431d8dee5)
2013-07-28 04:08:10 +02:00
Michael Niedermayer
9da9b36435 jpeg2000dec: parse CDEF
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Conflicts:

	libavcodec/jpeg2000dec.c

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 99de97cabf35c035b628943b865ab991583a6ea5)
2013-07-28 04:07:36 +02:00
Carl Eugen Hoyos
09b33f9a82 Fix pix_fmt detection in the native jpeg2000 decoder.
Based on b7a928b by Michael Bradshaw.
Fixes ticket #2683.

Reviewed-by: Nicolas Bertrand
(cherry picked from commit b39a6bbe7f43710d8e7163757c5aeef596878712)
2013-07-28 04:07:20 +02:00
Michael Niedermayer
fa6b6dad3d jpeg2000: fix overflow in dequantization
Fixes decoding of file generated with:
ffmpeg -f lavfi -i smptehdbars=hd720 -pix_fmt rgb48 /tmp/o.jp2

Reviewed-by: Nicolas BERTRAND <nicoinattendu@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f57119b8e58cb5437c3ab40d797293ecb9b4a894)
2013-07-28 04:07:10 +02:00
Nicolas Bertrand
e0d88cfd18 jpeg2000: Initialize only once mqc arrays
Increases encoding and decoding speed

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dd1382ac9534bd9b5ec7833eed9ab6f383e68a50)
2013-07-28 04:07:02 +02:00
Michael Niedermayer
5eb23e6b0a Merge commit '22e18ea39e371030cc78973d1b46aae45a7ea215'
* commit '22e18ea39e371030cc78973d1b46aae45a7ea215':
  jpeg2000: Optimize output sample conversion
  jpeg2000: Reset s->numX/Ytiles on tile deallocation
  jpeg2000: Optimize dequantization
  jpeg2000: Do not assume a single tile
  jpeg2000: Speed up jpeg2000_decode_tile()
  jpeg2000: Use EBCOT's CAUSAL and BYPASS mode in decode_cblk()

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 16:36:05 +02:00
Michael Niedermayer
19ff6f55e8 Merge commit 'd57c737ac30cf079a342fa649fd4888c2d059dd9'
* commit 'd57c737ac30cf079a342fa649fd4888c2d059dd9':
  jpeg2000: Simplify jpeg2000_decode_packets()
  jpeg2000: Do not crash on NULL node in tag_tree_decode
  jpeg2000: Refactor SOT marker parsing

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 16:26:01 +02:00
Michael Niedermayer
1abb875c98 Merge commit 'daeb4e3042f2ecae2d41aaa4cae0bed932539788'
* commit 'daeb4e3042f2ecae2d41aaa4cae0bed932539788':
  jpeg2000: Proper cleanup on failure in decode_frame()

Conflicts:
	libavcodec/jpeg2000.c

See: 9d56ccf5

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 16:11:01 +02:00
Michael Niedermayer
ee7da5bacc Merge commit '589e5b52f634f6b2d307a167a19eef7e7328cb08'
* commit '589e5b52f634f6b2d307a167a19eef7e7328cb08':
  jpeg2000: Use the correct sizeof in memset for T1 data
  jpeg2000: Simplify init_tile()
  jpeg2000: Update pixel format support

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 15:45:03 +02:00
Michael Niedermayer
11cf4b7247 Merge commit 'f9581f1414ec8e3e7d0868793cda4f2e4fa113ea'
* commit 'f9581f1414ec8e3e7d0868793cda4f2e4fa113ea':
  jpeg2000: Improve reduced resolution decoding
  jpeg2000: Compute quantization for 'scalar derived' in the correct case.
  jpeg2000: Calculate code-block coord in ff_jpeg2000_init_component()

Conflicts:
	libavcodec/jpeg2000.c
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 15:20:37 +02:00
Michael Niedermayer
c7a092cb2d Merge commit '4cbd5ed11b0161a6a192174934b241e0cc0599a9'
* commit '4cbd5ed11b0161a6a192174934b241e0cc0599a9':
  jpeg2000: Fix compute precedence error in lut_gain index
  jpeg2000: Initialize code blocks structures in precincts to 0
  jpeg2000: Check that we have enough components for MCT

Conflicts:
	libavcodec/jpeg2000.c
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 15:03:08 +02:00
Michael Niedermayer
9dfa5ea5dc Merge commit '86a2602e16171ddc10c3ec655c23b16ffa595059'
* commit '86a2602e16171ddc10c3ec655c23b16ffa595059':
  jpeg2000: Drop unused and writeonly fields
  jpeg2000: Correctly calculate sgnd

Conflicts:
	libavcodec/jpeg2000.h
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 14:46:10 +02:00
Michael Niedermayer
a4a16821d7 Merge commit 'fd54dd028bc9f7bfb80ebf823a533dc84b73f936'
* commit 'fd54dd028bc9f7bfb80ebf823a533dc84b73f936':
  jpeg2000: check len before parsing header
  jpeg2000: Check component number in get_coc() and get_qcc()

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 14:38:37 +02:00
Michael Niedermayer
2cad1ea29e Merge commit '17e5d614a8647d51b9795cb8bccf97ee33ca2d58'
* commit '17e5d614a8647d51b9795cb8bccf97ee33ca2d58':
  jpeg2000: Check zero bit-plane validity

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 14:30:17 +02:00
Michael Niedermayer
db5c93a1a9 Merge commit '7e201d575dc4385eb67314b0419d4d77185e65f4'
* commit '7e201d575dc4385eb67314b0419d4d77185e65f4':
  jpeg2000: Validate block lengthinc

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 14:28:51 +02:00
Michael Niedermayer
f0358dc1d3 Merge commit '278a923c51187d37445d88a6b21082036ec9568d'
* commit '278a923c51187d37445d88a6b21082036ec9568d':
  jpeg2000: Validate SIZ parsing

Conflicts:
	libavcodec/jpeg2000dec.c

This commit is not exactly merged due to bugs in it

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 14:19:35 +02:00
Michael Niedermayer
2e85737b0e Merge commit 'd3cb302b88503c3111e25add196622110c056188'
* commit 'd3cb302b88503c3111e25add196622110c056188':
  jpeg2000: Validate SOT parsing

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 13:28:35 +02:00
Michael Niedermayer
57bc64e235 Merge commit '1a3598aae768465a8efc8475b6df5a8261bc62fc'
* commit '1a3598aae768465a8efc8475b6df5a8261bc62fc':
  jpeg2000: Use bytestream2

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 12:54:01 +02:00
Michael Niedermayer
26f6acc66b Merge commit '5efadcb8cdef6fc5ca5f1f72e45ef3b23016b1c4'
* commit '5efadcb8cdef6fc5ca5f1f72e45ef3b23016b1c4':
  jpeg2000: Clean up return paths and error messages

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 12:16:24 +02:00
Michael Niedermayer
b0448e5123 Merge commit 'be3271009ef893fbed9640e8edbd771685fd957b'
* commit 'be3271009ef893fbed9640e8edbd771685fd957b':
  jpeg2000: Define the maximum decomposition levels

Conflicts:
	libavcodec/jpeg2000.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 12:02:11 +02:00
Michael Niedermayer
3f0364327c Merge commit 'fbcc03db8f3919e2ea46d6ad11a00906b1c0ef0e'
* commit 'fbcc03db8f3919e2ea46d6ad11a00906b1c0ef0e':
  jpeg2000: Check code-block size

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 11:55:45 +02:00
Michael Niedermayer
37a970bbb6 Merge commit '5650e331a733711b809bc27f968500f0f5530126'
* commit '5650e331a733711b809bc27f968500f0f5530126':
  jpeg2000: Validate resolution levels

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 11:20:39 +02:00
Michael Niedermayer
22e18ea39e jpeg2000: Optimize output sample conversion
67935 -> 29984 kcycles

Reviewed-by: Nicolas BERTRAND <nicoinattendu@gmail.com>

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:48 +02:00
Michael Niedermayer
f0552e63a6 jpeg2000: Reset s->numX/Ytiles on tile deallocation
Keep the structure fields more consistent after cleanup.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:48 +02:00
Michael Niedermayer
d57c737ac3 jpeg2000: Simplify jpeg2000_decode_packets()
Raise PATCHWELCOME error in case of non-implemented progression order.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Signed-off-by: Nicolas Bertrand <nicoinattendu@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:47 +02:00
Luca Barbato
daeb4e3042 jpeg2000: Proper cleanup on failure in decode_frame()
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:47 +02:00
Michael Niedermayer
e11099db20 jpeg2000: Optimize dequantization
Float:   4700 -> 2700 cycles
Integer: 4400 -> 2800 cycles

(sandybridge  i7)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:47 +02:00
Michael Niedermayer
09d5929f37 jpeg2000: Do not crash on NULL node in tag_tree_decode
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:47 +02:00
Michael Niedermayer
c1dcbc590d jpeg2000: Do not assume a single tile
In preparation of supporting multiple tiles.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:47 +02:00
Michael Niedermayer
aa16bbaf9b jpeg2000: Refactor SOT marker parsing
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:47 +02:00
Michael Niedermayer
53d5d89c1b jpeg2000: Speed up jpeg2000_decode_tile()
Skip processing bands with dimension set to 0.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:47 +02:00
Michael Niedermayer
64f6570c6e jpeg2000: Use EBCOT's CAUSAL and BYPASS mode in decode_cblk()
Speed it up a bit.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:47 +02:00
Michael Niedermayer
589e5b52f6 jpeg2000: Use the correct sizeof in memset for T1 data
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:46 +02:00
Michael Niedermayer
f9581f1414 jpeg2000: Improve reduced resolution decoding
Correctly scale down the component coordinates and
clean up some redundant code.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:46 +02:00
Michael Niedermayer
5b73916d34 jpeg2000: Simplify init_tile()
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:46 +02:00
Michael Niedermayer
33e665a6ab jpeg2000: Update pixel format support
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:46 +02:00
Michael Niedermayer
5bf208f659 jpeg2000: Use separate fields for int and float codepaths
Split stepsize and data into int and float variants.
Eliminates a number of casts and simplifies spotting errors.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:46 +02:00
Michael Niedermayer
a458b91cf4 jpeg2000: Split int/float codepaths depending on the DWT
DWT53 is always int, DWT97 is always float.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:46 +02:00