Commit Graph

262 Commits

Author SHA1 Message Date
Michael Niedermayer
8d024c5107 Merge commit 'cc8163e1a3601a56f722a4720516e860bf1c6198'
* commit 'cc8163e1a3601a56f722a4720516e860bf1c6198':
  avcodec: more correct printf specifiers

Conflicts:
	libavcodec/4xm.c
	libavcodec/alsdec.c
	libavcodec/dfa.c
	libavcodec/h264_ps.c
	libavcodec/jpeg2000dec.c
	libavcodec/lagarith.c
	libavcodec/mpeg12dec.c
	libavcodec/rv10.c
	libavcodec/svq3.c
	libavcodec/wmaprodec.c
	libavcodec/xwddec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-22 18:43:40 +01:00
Diego Biurrun
cc8163e1a3 avcodec: more correct printf specifiers 2014-03-22 14:08:20 +01:00
Lukasz Marek
aeb2905fb7 lavc/jpeg2000dec: silent warning discards qualifiers
Signed-off-by: Lukasz Marek <lukasz.m.luki@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-08 02:54:37 +01:00
Michael Niedermayer
8001e9f7d1 avcodec/jpeg2000dec: fix error detection in pix_fmt_match()
Fixes out of array accesses with CODEC_FLAG_EMU_EDGE

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-20 18:46:38 +01:00
Michael Niedermayer
723e7b86ea avcodec/jpeg2000dec: calculate planar and pixelsize from pixel format descriptor
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-20 18:46:37 +01:00
Michael Niedermayer
55fa898969 avcodec/jpeg2000dec: zero Jpeg2000QuantStyle structure before use in get_qcd()
Fixes: msan_uninit-mem_7f50b84aac30_6823_mjp2.mov
Fixes use of uninitialized memory
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-28 22:08:49 +01:00
Michael Niedermayer
3d5a5e86be avcodec/jpeg2000dec: Check precno before using it in JPEG2000_PGOD_CPRL
Fixes out of array reads
Fixes: asan_heap-oob_f0de57_6823_mjp2.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-24 04:02:24 +01:00
Michael Niedermayer
780669ef7c avcodec/jpeg2000dec: non zero image offsets are not supported
Fixes out of array accesses
Fixes Ticket3080
Found-by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-26 02:19:13 +02:00
Michael Niedermayer
a1b9004b76 avcodec/jpeg2000dec: fix context consistency with too large lowres
Fixes out of array accesses
Fixes Ticket2898

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-15 14:30:17 +02:00
Michael Niedermayer
fe448cd28d avcodec/jpeg2000dec: prevent out of array accesses in pixel addressing
Fixes Ticket2921

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-13 21:48:36 +02:00
Michael Niedermayer
ac3b01a9c0 avcodec/jpeg2000dec: check transform equality in MCT
Fixes null pointer dereference
Fixes Ticket2843

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-13 19:23:24 +02:00
Michael Niedermayer
e96c80154f Merge commit '1018a92219a38a812cf97761c6b3a5e66a400f4b'
* commit '1018a92219a38a812cf97761c6b3a5e66a400f4b':
  jpeg2000: Check block length

See: 914ab4cd1c
See: 582f53349e
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-10 10:32:04 +02:00
Luca Barbato
1018a92219 jpeg2000: Check block length
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2013-10-09 22:25:19 +02:00
Michael Niedermayer
fcd19d6c8d avcodec/jpeg2000dec: Zero code and quant style
This prevents future frames from using half initialized/inconsistent
structures
Fixes assertion failure
Fixes Ticket2897

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-03 19:03:56 +02:00
Michael Niedermayer
2dd38a1d48 avcodec/jpeg2000dec: move "missing SIZ" check up
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-18 10:04:43 +02:00
Michael Niedermayer
8ae14bf5ce Merge commit '18a245a2aea31b92bc435f2962a71e060d83b5f7'
* commit '18a245a2aea31b92bc435f2962a71e060d83b5f7':
  jpeg2000dec: Add more checks when parsing headers

The check cannot be true in FFmpeg, but does no harm.
I will move it in a subsequent commit so it can detect missing SIZ

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-18 10:03:35 +02:00
Martin Storsjö
18a245a2ae jpeg2000dec: Add more checks when parsing headers
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
2013-09-17 15:57:27 +03:00
Clément Bœsch
04427182bc avcodec: typo fix sepera* → separa* 2013-09-15 21:32:42 +02:00
Michael Niedermayer
e428632c1a avcodec/jpeg2000dec: print invalid cdx/y values
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-15 15:37:16 +02:00
Carl Eugen Hoyos
f99f7f4025 Set bits_per_raw_sample when reading j2k frames. 2013-09-12 09:23:32 +02:00
Michael Niedermayer
4ced30908f avcodec/jpeg2000dec: make SOC finding code more robust
Fixes file from Ticket860

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-05 01:39:20 +02:00
Michael Niedermayer
9a271a9368 jpeg2000: check log2_cblk dimensions
Fixes out of array access
Fixes Ticket2895

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-24 03:42:54 +02:00
Michael Niedermayer
8bb11c3ca7 avcodec/jpeg2000dec: Check cdx/y values more carefully
Some invalid values where not handled correctly in the later pixel
format matching code.
Fixes out of array accesses
Fixes Ticket2848

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-23 01:02:03 +02:00
Michael Niedermayer
63c0e9077e avcodec/jpeg2000dec: fix near null pointer dereference
Fixes Ticket2840

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-18 18:44:12 +02:00
Michael Niedermayer
5cd57e8758 avcodec/jpeg2000dec: check sample sepration for validity
Fixes division by 0
Fixes Ticket2841

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-07 15:09:35 +02:00
Michael Niedermayer
bff371e34c jpeg2000dec: simplify jpeg2000_read_bitstream_packets()
23 lines less
less error prone as theres no more need to update *_size on every read

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-02 03:55:07 +02:00
Michael Niedermayer
1434df3b93 jpeg2000dec: Support non subsampled 9-16bit planar pixel formats
This applies changes similar to fc6de70c44
to the >8bit codepath

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-21 04:16:53 +02:00
Michael Niedermayer
db33010483 jpeg2000dec: silence unused variable warning
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-19 17:54:57 +02:00
Michael Niedermayer
fc6de70c44 jpeg2000dec: Support non subsampled 8bit planar pixel formats
Fixes file2.jp2

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-18 21:50:32 +02:00
Michael Niedermayer
99de97cabf jpeg2000dec: parse CDEF
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Conflicts:

	libavcodec/jpeg2000dec.c

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-18 21:43:42 +02:00
Carl Eugen Hoyos
b39a6bbe7f Fix pix_fmt detection in the native jpeg2000 decoder.
Based on b7a928b by Michael Bradshaw.
Fixes ticket #2683.

Reviewed-by: Nicolas Bertrand
2013-07-18 13:51:24 +02:00
Nicolas Bertrand
f56fe04de3 jpeg2000: Initialize only once mqc arrays
Improves decoding speed.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-13 07:54:12 +02:00
Michael Niedermayer
f57119b8e5 jpeg2000: fix overflow in dequantization
Fixes decoding of file generated with:
ffmpeg -f lavfi -i smptehdbars=hd720 -pix_fmt rgb48 /tmp/o.jp2

Reviewed-by: Nicolas BERTRAND <nicoinattendu@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-13 12:44:37 +02:00
Nicolas Bertrand
dd1382ac95 jpeg2000: Initialize only once mqc arrays
Increases encoding and decoding speed

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-12 23:43:09 +02:00
Michael Niedermayer
5eb23e6b0a Merge commit '22e18ea39e371030cc78973d1b46aae45a7ea215'
* commit '22e18ea39e371030cc78973d1b46aae45a7ea215':
  jpeg2000: Optimize output sample conversion
  jpeg2000: Reset s->numX/Ytiles on tile deallocation
  jpeg2000: Optimize dequantization
  jpeg2000: Do not assume a single tile
  jpeg2000: Speed up jpeg2000_decode_tile()
  jpeg2000: Use EBCOT's CAUSAL and BYPASS mode in decode_cblk()

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 16:36:05 +02:00
Michael Niedermayer
19ff6f55e8 Merge commit 'd57c737ac30cf079a342fa649fd4888c2d059dd9'
* commit 'd57c737ac30cf079a342fa649fd4888c2d059dd9':
  jpeg2000: Simplify jpeg2000_decode_packets()
  jpeg2000: Do not crash on NULL node in tag_tree_decode
  jpeg2000: Refactor SOT marker parsing

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 16:26:01 +02:00
Michael Niedermayer
1abb875c98 Merge commit 'daeb4e3042f2ecae2d41aaa4cae0bed932539788'
* commit 'daeb4e3042f2ecae2d41aaa4cae0bed932539788':
  jpeg2000: Proper cleanup on failure in decode_frame()

Conflicts:
	libavcodec/jpeg2000.c

See: 9d56ccf5

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 16:11:01 +02:00
Michael Niedermayer
ee7da5bacc Merge commit '589e5b52f634f6b2d307a167a19eef7e7328cb08'
* commit '589e5b52f634f6b2d307a167a19eef7e7328cb08':
  jpeg2000: Use the correct sizeof in memset for T1 data
  jpeg2000: Simplify init_tile()
  jpeg2000: Update pixel format support

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 15:45:03 +02:00
Michael Niedermayer
11cf4b7247 Merge commit 'f9581f1414ec8e3e7d0868793cda4f2e4fa113ea'
* commit 'f9581f1414ec8e3e7d0868793cda4f2e4fa113ea':
  jpeg2000: Improve reduced resolution decoding
  jpeg2000: Compute quantization for 'scalar derived' in the correct case.
  jpeg2000: Calculate code-block coord in ff_jpeg2000_init_component()

Conflicts:
	libavcodec/jpeg2000.c
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 15:20:37 +02:00
Michael Niedermayer
c7a092cb2d Merge commit '4cbd5ed11b0161a6a192174934b241e0cc0599a9'
* commit '4cbd5ed11b0161a6a192174934b241e0cc0599a9':
  jpeg2000: Fix compute precedence error in lut_gain index
  jpeg2000: Initialize code blocks structures in precincts to 0
  jpeg2000: Check that we have enough components for MCT

Conflicts:
	libavcodec/jpeg2000.c
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 15:03:08 +02:00
Michael Niedermayer
9dfa5ea5dc Merge commit '86a2602e16171ddc10c3ec655c23b16ffa595059'
* commit '86a2602e16171ddc10c3ec655c23b16ffa595059':
  jpeg2000: Drop unused and writeonly fields
  jpeg2000: Correctly calculate sgnd

Conflicts:
	libavcodec/jpeg2000.h
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 14:46:10 +02:00
Michael Niedermayer
a4a16821d7 Merge commit 'fd54dd028bc9f7bfb80ebf823a533dc84b73f936'
* commit 'fd54dd028bc9f7bfb80ebf823a533dc84b73f936':
  jpeg2000: check len before parsing header
  jpeg2000: Check component number in get_coc() and get_qcc()

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 14:38:37 +02:00
Michael Niedermayer
2cad1ea29e Merge commit '17e5d614a8647d51b9795cb8bccf97ee33ca2d58'
* commit '17e5d614a8647d51b9795cb8bccf97ee33ca2d58':
  jpeg2000: Check zero bit-plane validity

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 14:30:17 +02:00
Michael Niedermayer
db5c93a1a9 Merge commit '7e201d575dc4385eb67314b0419d4d77185e65f4'
* commit '7e201d575dc4385eb67314b0419d4d77185e65f4':
  jpeg2000: Validate block lengthinc

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 14:28:51 +02:00
Michael Niedermayer
f0358dc1d3 Merge commit '278a923c51187d37445d88a6b21082036ec9568d'
* commit '278a923c51187d37445d88a6b21082036ec9568d':
  jpeg2000: Validate SIZ parsing

Conflicts:
	libavcodec/jpeg2000dec.c

This commit is not exactly merged due to bugs in it

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 14:19:35 +02:00
Michael Niedermayer
2e85737b0e Merge commit 'd3cb302b88503c3111e25add196622110c056188'
* commit 'd3cb302b88503c3111e25add196622110c056188':
  jpeg2000: Validate SOT parsing

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 13:28:35 +02:00
Michael Niedermayer
57bc64e235 Merge commit '1a3598aae768465a8efc8475b6df5a8261bc62fc'
* commit '1a3598aae768465a8efc8475b6df5a8261bc62fc':
  jpeg2000: Use bytestream2

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 12:54:01 +02:00
Michael Niedermayer
26f6acc66b Merge commit '5efadcb8cdef6fc5ca5f1f72e45ef3b23016b1c4'
* commit '5efadcb8cdef6fc5ca5f1f72e45ef3b23016b1c4':
  jpeg2000: Clean up return paths and error messages

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 12:16:24 +02:00
Michael Niedermayer
b0448e5123 Merge commit 'be3271009ef893fbed9640e8edbd771685fd957b'
* commit 'be3271009ef893fbed9640e8edbd771685fd957b':
  jpeg2000: Define the maximum decomposition levels

Conflicts:
	libavcodec/jpeg2000.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 12:02:11 +02:00
Michael Niedermayer
3f0364327c Merge commit 'fbcc03db8f3919e2ea46d6ad11a00906b1c0ef0e'
* commit 'fbcc03db8f3919e2ea46d6ad11a00906b1c0ef0e':
  jpeg2000: Check code-block size

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 11:55:45 +02:00
Michael Niedermayer
37a970bbb6 Merge commit '5650e331a733711b809bc27f968500f0f5530126'
* commit '5650e331a733711b809bc27f968500f0f5530126':
  jpeg2000: Validate resolution levels

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-03 11:20:39 +02:00
Michael Niedermayer
95a57d26d8 jpeg2000: Use the matching coding style struct for transform selection
Fixes a null pointer dereference.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:48 +02:00
Michael Niedermayer
22e18ea39e jpeg2000: Optimize output sample conversion
67935 -> 29984 kcycles

Reviewed-by: Nicolas BERTRAND <nicoinattendu@gmail.com>

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:48 +02:00
Michael Niedermayer
f0552e63a6 jpeg2000: Reset s->numX/Ytiles on tile deallocation
Keep the structure fields more consistent after cleanup.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:48 +02:00
Michael Niedermayer
d57c737ac3 jpeg2000: Simplify jpeg2000_decode_packets()
Raise PATCHWELCOME error in case of non-implemented progression order.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Signed-off-by: Nicolas Bertrand <nicoinattendu@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:47 +02:00
Luca Barbato
daeb4e3042 jpeg2000: Proper cleanup on failure in decode_frame()
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:47 +02:00
Michael Niedermayer
e11099db20 jpeg2000: Optimize dequantization
Float:   4700 -> 2700 cycles
Integer: 4400 -> 2800 cycles

(sandybridge  i7)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:47 +02:00
Michael Niedermayer
09d5929f37 jpeg2000: Do not crash on NULL node in tag_tree_decode
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:47 +02:00
Michael Niedermayer
c1dcbc590d jpeg2000: Do not assume a single tile
In preparation of supporting multiple tiles.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:47 +02:00
Michael Niedermayer
aa16bbaf9b jpeg2000: Refactor SOT marker parsing
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:47 +02:00
Michael Niedermayer
53d5d89c1b jpeg2000: Speed up jpeg2000_decode_tile()
Skip processing bands with dimension set to 0.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:47 +02:00
Michael Niedermayer
64f6570c6e jpeg2000: Use EBCOT's CAUSAL and BYPASS mode in decode_cblk()
Speed it up a bit.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:47 +02:00
Michael Niedermayer
589e5b52f6 jpeg2000: Use the correct sizeof in memset for T1 data
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:46 +02:00
Michael Niedermayer
f9581f1414 jpeg2000: Improve reduced resolution decoding
Correctly scale down the component coordinates and
clean up some redundant code.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:46 +02:00
Michael Niedermayer
5b73916d34 jpeg2000: Simplify init_tile()
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:46 +02:00
Michael Niedermayer
33e665a6ab jpeg2000: Update pixel format support
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:46 +02:00
Michael Niedermayer
5bf208f659 jpeg2000: Use separate fields for int and float codepaths
Split stepsize and data into int and float variants.
Eliminates a number of casts and simplifies spotting errors.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:46 +02:00
Michael Niedermayer
a458b91cf4 jpeg2000: Split int/float codepaths depending on the DWT
DWT53 is always int, DWT97 is always float.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:46 +02:00
Michael Niedermayer
4e11b15534 jpeg2000: Calculate code-block coord in ff_jpeg2000_init_component()
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Signed-off-by: Nicolas Bertrand <nicoinattendu@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:45 +02:00
Michael Niedermayer
cf04af2086 jpeg2000: Check that we have enough components for MCT
Avoid overread.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:45 +02:00
Michael Niedermayer
b564784a20 jpeg2000: Check that there is a SOT before SOD
Avoid overreads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:45 +02:00
Michael Niedermayer
2c3901b2c3 jpeg2000: Remove unneeded returns
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:45 +02:00
Michael Niedermayer
ef35d6dbc6 jpeg2000: Propagate error code from get_cox()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:45 +02:00
Michael Niedermayer
86a2602e16 jpeg2000: Drop unused and writeonly fields
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:44 +02:00
Michael Niedermayer
fd54dd028b jpeg2000: check len before parsing header
Avoid overread.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:44 +02:00
Michael Niedermayer
17e5d614a8 jpeg2000: Check zero bit-plane validity
Prevent integer overflows.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:44 +02:00
Luca Barbato
7e201d575d jpeg2000: Validate block lengthinc
Currently we are using an array with a static data size.

Similar to a patch with the same purpose by Michael Niedermayer.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:44 +02:00
Michael Niedermayer
278a923c51 jpeg2000: Validate SIZ parsing
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:44 +02:00
Michael Niedermayer
d3cb302b88 jpeg2000: Validate SOT parsing
Avoid some overreads.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:44 +02:00
Michael Niedermayer
ac921338a4 jpeg2000: Correctly calculate sgnd
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:44 +02:00
Michael Niedermayer
eae63e3c15 jpeg2000: Check component number in get_coc() and get_qcc()
Avoid overreads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Nicolas Bertrand <nicoinattendu@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:44 +02:00
Michael Niedermayer
1a3598aae7 jpeg2000: Use bytestream2
Prevent a number of overreads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:43 +02:00
Luca Barbato
5efadcb8cd jpeg2000: Clean up return paths and error messages
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:43 +02:00
Luca Barbato
be3271009e jpeg2000: Define the maximum decomposition levels
And define the resolution levels according.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:43 +02:00
Michael Niedermayer
fbcc03db8f jpeg2000: Check code-block size
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Signed-off-by: Nicolas Bertrand <nicoinattendu@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:43 +02:00
Luca Barbato
5650e331a7 jpeg2000: Validate resolution levels
There are 32 maximum decomposition levels, thus 33 resolution levels.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-02 20:05:43 +02:00
Carl Eugen Hoyos
0915b531bc Rename "AVClass class" as "AVClass component_class".
The aix header math.h defines "extern int class()" for C.
2013-06-30 13:47:45 +02:00
Michael Niedermayer
32fc8d6db6 avcodec/jpeg2000dec: check that tp_end is after the start
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-21 00:58:41 +02:00
Michael Niedermayer
582f53349e jpeg2000dec: make cblk->length(inc)/data use consistent
Fixes Ticket2612

Reviewed-by: Nicolas BERTRAND <nicoinattendu@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-17 20:08:51 +02:00
Michael Niedermayer
3b86174290 avcodec/jpeg2000dec: move bpno check to a earlier place
This prevents additional integer overflows

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-16 15:27:39 +02:00
Michael Niedermayer
69e4d8e6a4 avcodec/jpeg2000dec: check len before parsing header
Fixes out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-14 19:34:00 +02:00
Michael Niedermayer
5deb96c564 avcodec/jpeg2000dec: Check Psot
Fixes out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-14 19:34:00 +02:00
Michael Niedermayer
e850a064ce jpeg2000dec: Check compno in get_coc()
Fixes out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-14 01:55:43 +02:00
Michael Niedermayer
129edcb50f jpeg2000dec: dont set ncomponents to an invalid value
Fixes out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-14 00:36:45 +02:00
Michael Niedermayer
8e887ca1fe jpeg2000dec: Check bpno in decode_cblk()
Fixes integer overflow in fate-redcode-demux

Reviewed-by: Nicolas BERTRAND <nicoinattendu@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-10 17:13:46 +02:00
Michael Niedermayer
7c3a296b4f jpeg2000dec: fix av_log()
Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-06 21:52:49 +02:00
Michael Niedermayer
21d0f75f29 jpeg2000dec: check that we have enough components for MCT
Fixes out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-06 16:06:48 +02:00
Michael Niedermayer
c49d94487c jpeg2000dec: Use the matching coding style struct for transform selection
Fixes a null pointer dereference

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-06 15:38:05 +02:00
Michael Niedermayer
b26bcd08e6 jpeg2000dec: Check that theres a SOT before SOD
Fixes out of array access

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-06 10:51:07 +02:00
Michael Niedermayer
c51654fbc0 jpeg2000dec: Support 4 tile parts
Fixes out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-06 03:10:42 +02:00
Michael Niedermayer
c59ce1c98e get_qcc: Check compno
Fixes out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-06 02:26:22 +02:00
Michael Niedermayer
bbc19010ed jpeg2000dec: return error for invalid cdxy values
Fixes division by 0

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-05 23:31:19 +02:00
Michael Niedermayer
66c4d54413 jpeg2000dec: Propagate error code from get_cox() correctly
Without this the context state could become inconsistent

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-05 18:34:07 +02:00
Michael Niedermayer
bbf43c70dd jpeg2000dec: assert that curtileno is valid when used
Fixes CID1026755
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-03 14:59:52 +02:00
Michael Niedermayer
6507d86f07 jpeg2000dec; optimize output sample convert a bit
67935 -> 29984 kcycles

Reviewed-by: Nicolas BERTRAND <nicoinattendu@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-03 13:00:50 +02:00
Paul B Mahol
d2021f74ed jpeg2000dec: remove redundant error log message
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2013-06-03 00:20:08 +00:00
Michael Niedermayer
9054e72529 jpeg2000: make sure s->numXtiles / s->numYtiles are reset when tile is deallocated
This keeps the structure fields more consistent after cleanup

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-01 14:41:46 +02:00
Michael Niedermayer
dc60061bb6 jpeg2000dec: optimize dequantization_int()
4400->2800 cycles

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-01 11:48:13 +02:00
Michael Niedermayer
6cbd0241f2 jpeg2000dec: optimize dequantization_float()
4700 -> 2700 cycles (sandybridge  i7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-01 11:48:09 +02:00
Michael Niedermayer
f77467a11d jpeg2000dec: fix indention
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-31 13:33:13 +02:00
Michael Niedermayer
c1415cfefb jpeg2000dec: print more detailed cdx/y debug info
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-31 13:32:33 +02:00
Michael Niedermayer
9d56ccf5af j2k/jpeg2000dec: merge
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-31 03:10:08 +02:00
Michael Niedermayer
fd7e119037 jpeg2000dec: merge simplification of jpeg2000_decode_packets() from j2k
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-31 03:10:08 +02:00
Michael Niedermayer
192050d7a7 jpeg2000dec: merge sgnd fix from j2k
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-31 03:10:07 +02:00
Michael Niedermayer
5dbbb762e5 jpeg2000dec: merge struct field types from j2k
Fixes AVOption type mismatch

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-31 03:10:07 +02:00
Michael Niedermayer
cdb86136f7 j2k/jpeg2000: merge some of the tilepart related code
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-31 02:22:11 +02:00
Michael Niedermayer
f67f2681da jpeg2000deci/j2kdec: fix sizeof types
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-31 02:22:10 +02:00
Michael Niedermayer
20a2d5ec11 jpeg2000dec: fix ff_mqc_initdec() and data setup order
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-31 02:22:10 +02:00
Michael Niedermayer
7a2b634220 jpeg2000dec: simplify init_tile() / merge from j2k
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-31 02:22:10 +02:00
Michael Niedermayer
3f714d1ffd jpeg2000/j2k: merge non functional changes
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-30 18:38:04 +02:00
Michael Niedermayer
9ccc9f30f7 jpeg2000: drop red-blue swap
This should no longer be needed as the pix fmt was updated

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-30 18:37:56 +02:00
Michael Niedermayer
310f9dd698 j2k/jpeg2000: merge pix_fmt setting code
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-30 16:45:30 +02:00
Michael Niedermayer
8bedbb82ce j2k/jpeg2000: split data pointer in int & float.
This fixes a TODO item and unifies both decoders structures
It also fixes undefined behavior due to aliasing violations

I choose 2 fields instead of a union because mistakely using the
wrong type with a union will lead to hard to debug "wrong output"
while with 2 fields mistakely using the wrong type will crash
with a null pointer derefernce which is much easier to debug

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-30 14:18:53 +02:00
Michael Niedermayer
83fd377c94 j2k/jpeg2000: merge float DWT and related code
the float conditions are largely changed to check the DWT
instead of the bitexact flag, which previously was wrong
(DWT53 is always int)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-29 00:29:25 +02:00
Michael Niedermayer
c37d735c1c jpeg2000dec: mct_decode: remove unused return
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-28 23:49:11 +02:00
Michael Niedermayer
fe7136ef82 j2k/jpeg2000: split stepsize in float & int variables
This is more clear and less prone to mistakes.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-28 22:16:14 +02:00
Michael Niedermayer
aa0d11ed0b jpeg2000dec: remove unneeded returns
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-28 22:16:14 +02:00
Michael Niedermayer
8cf57efdd7 j2k/jpeg2000: merge lowres code
This also fixes lowres use with ffplay

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-28 00:30:06 +02:00
Michael Niedermayer
8c2e201c4f j2k/jpeg2000: drop xi/yi0/1 from Jpeg2000Prec
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-27 15:53:42 +02:00
Michael Niedermayer
14652c080d j2k/jpeg2000: restructure cblk coord
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-27 15:13:18 +02:00
Michael Niedermayer
bd89b2b22a j2k/jpeg2000: log2_prec size cleanup
This also changes the cblk size storage method to what jpeg2000 uses

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-26 22:29:11 +02:00
Michael Niedermayer
cb39dfb870 j2k/jpeg2000: check cblk size
This is based on j2k FCD which is ambigous thus the limit
might be tighter.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-26 20:15:50 +02:00
Michael Niedermayer
de90bd6c3f j2k/jpeg2000: merge cosmetics and whitespace
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-26 20:15:50 +02:00
Michael Niedermayer
914ab4cd1c j2kdec & jpeg2000dec: check lengthinc against array size
Fixes out of array reads

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-22 04:18:38 +02:00
Michael Niedermayer
17e66c9ff4 j2k & jpeg2000dec: reset numX/Ytiles if tiles failed allocation
Fixes null pointer dereference

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-22 03:18:55 +02:00
Michael Niedermayer
53f0442464 avcodec/jpeg2000dec: Port tile size check from j2kdec
Fixes potential integer overflow

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-22 03:16:52 +02:00
Michael Niedermayer
73ffab4195 jpeg2000dec: check TPsot
Fixes out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-22 02:53:21 +02:00
Michael Niedermayer
fec8783883 jpeg2000dec: read and check curtileno
Fixes out of array accesses
Code ported from j2kdec

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-22 02:29:49 +02:00
Michael Niedermayer
f399b33c2b jpeg2000dec & j2kdec: Fix return type of get_sot()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-22 02:14:04 +02:00
Michael Niedermayer
4a9a6a1f35 vcodec/jpeg2000dec: fix av_log() type
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-22 02:05:50 +02:00
Michael Niedermayer
0b42631641 avcodec/jpeg2000dec: merge bytestream2 reading from j2kdec
This fixes at least one out of array access

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-22 01:24:47 +02:00
Michael Niedermayer
c649ecb3b2 avcodec/jpeg2000dec: get_qcx, fix stack and heap overwrites
This is likely remotely exploitable
Fix ported from j2kdec
No uptodate FFmpeg release should be affected by this

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-22 00:07:45 +02:00
Michael Niedermayer
069ede2981 jpeg2000dec: Check ncomponents and tile dimensions
Fixes various problems

Code ported from j2kdec

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-21 21:44:34 +02:00
Michael Niedermayer
75835abe91 Merge commit '7c57a582a03fb473091a88737ab92b9f2a5bb87a'
* commit '7c57a582a03fb473091a88737ab92b9f2a5bb87a':
  jpeg2000dec: don't use deprecated PIX_FMT values

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-15 10:39:00 +02:00
Michael Niedermayer
85de84c3e0 Merge commit '7cbc4cb4424430586ce2d5c0dba81174c6d1db01'
* commit '7cbc4cb4424430586ce2d5c0dba81174c6d1db01':
  fate: update the mmf reference
  mmf: Write metadata into an Optional Data chunk
  jpeg2000dec: Use correct printf length modifier for pointer differences

Conflicts:
	libavformat/mmf.c
	tests/ref/lavf/mmf

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-15 10:15:15 +02:00
Anton Khirnov
7c57a582a0 jpeg2000dec: don't use deprecated PIX_FMT values 2013-05-15 07:46:39 +02:00
Diego Biurrun
a349458e67 jpeg2000dec: Use correct printf length modifier for pointer differences
Also eliminate an unnecessary cast. Fixes the following warning:
jpeg2000dec.c:1200:20: warning: format ‘%lX’ expects argument of type ‘long unsigned int’, but argument 5 has type ‘long long unsigned int’
2013-05-14 16:46:56 +02:00
Michael Niedermayer
1bad40ef27 Merge commit '70f9661542a581dfe93b636b1c55b5558e4a4e3c'
* commit '70f9661542a581dfe93b636b1c55b5558e4a4e3c':
  jpeg2000dec: initialize static data just once

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-08 11:46:41 +02:00
Michael Niedermayer
aecb3421f6 Merge commit '45a1694fde251ee56e23aec073d73172c00aa470'
* commit '45a1694fde251ee56e23aec073d73172c00aa470':
  jpeg2000dec: cleanup properly after decoding errors

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-08 11:41:54 +02:00
Michael Niedermayer
974420a701 jpeg2000dec: fix memleaks in error cases in jpeg2000_decode_frame()
Reviewed-by: Nicolas BERTRAND <nicoinattendu@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-06 22:56:22 +02:00