Fixes out of array access
Fixes: j2k-poc.bin
Found-by: Lucas Leong <wmliang.tw@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0aada30510d809bccfd539a90ea37b61188f2cb4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Sample rate of 11025 takes 16 bits but previous code would pick only 8.
Fixes assertion failure.
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit 3e7d6849120d61bb354376d52786c26f20e20835)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Cleaner solution is welcome!
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a53fbda9dc92273054a103db7539d2bb6e9632b2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array access
Fixes: aaa479088e6fb40b04837b3119f47b04/asan_heap-oob_e38c68_8576_9d653078b2470700e2834636f12ff557.tga
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 03d83ba34b2070878909eae18dfac0f519503777)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3ef5de0f19774e2c3dd9b08ba2e8ab7241a4862a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array access
Fixes: 83aedfb29af669c4d6e10f1bfad974d2/asan_heap-oob_1ab42fe_4984_9f6ec14462f8d8a00ea24b320572a963.tif
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 89f464e9c229006e16f6bb5403c5529fdd0a9edd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
src and dst are only 8-bit-aligned, so accessing them as uint16_t causes
SIGBUS crashes on architectures like sparc.
This fixes ubsan runtime error: load of misaligned address for type
'const uint16_t', which requires 2 byte alignment
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 44ac13eed49593f4f8efdb72ab0d5b48e05aa305)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This causes non-unique elements in floor_setup->data.t1.list, which
makes the stream undecodable according to the specification.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit e7a7b3135a4e5ba4bd2e144444d95a7563f53e9b)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
It causes the angle channel number to equal the magnitude channel
number, which makes the stream undecodable according to the
specification.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit b4b13848dec5420fa5dd9e1a7d4dfae5de1932d5)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The size of decoding map can differ from one calculated
internally, producing artifacts while decoding video.
Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit c293ef258cbb2c058e23651a26edf46e3bc05050)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Fixes a warning with -Wformat-extra-args
(cherry picked from commit 17e7fdf61a04f52c499e2d06eab2cf2d22343aa9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array accesses
Fixes: 0cf176e6d3ab9fe924f39738e513f547/asan_generic_4a54aa_3431_aaa28be1cb32e307a9890cad06f84fba.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8f4c3e4b92212d98f5b9ca2dee13e076effe9589)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array access
Fixes: 0cf176e6d3ab9fe924f39738e513f547/asan_generic_4a54aa_3431_aaa28be1cb32e307a9890cad06f84fba.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7ec9c5ce8a753175244da971fed9f1e25aef7971)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes assertion failure
Fixes out of memory access
Fixes: test_casex.ivf
Found-by: Tyson Smith <twsmith@mozilla.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7c97946d6131b31340954a3f603b6bf92590a9a5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
They are stored in a byte, thus more than 255 is not possible
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 59c915a403af32c4ff5126625b0cc7e38f4beff9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes index out of bounds error
Fixes: aac_index_out_of_bounds.wmv
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 590863876d1478547640304a31c15809c3618090)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: dvdec_left_shift.avi
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 93ac72a98dff592ffc174cfb36a8975dfbf145ae)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: mjpeg_left_shift.avi
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d86d7b2486cd5c31db8e820d8a89554abf19567e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: mss2_left_shift.wmv
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e273dade78943e22b71d0ddb67cd0d737fc26edf)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: test_case-mdc.264 (b47be15a120979f5a1a945c938cbef33)
Found-by: Tyson Smith <twsmith@mozilla.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 13f266b50cc7554028d22480b7e4383968e64a63)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Found-by: Tyson Smith <twsmith@mozilla.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5b4da8a38a5ed211df9504c85ce401c30af86b97)
Conflicts:
libavcodec/motion_est.h
Fixes mozilla bug 1230423
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7cc01c25727a96eaaa0c177234b626e47c8ea491)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This causes a overall slowdown of 0.1 % (tested with mpeg4 single thread encoding of matrixbench at QP=3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cccb0ffccc3723acc7aab3a859b24743596dd9c0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This can happen if s->ls changes from 0 to 1, but picture allocation is
skipped due to s->interlaced.
In that case ff_jpegls_decode_picture could be called even though the
s->picture_ptr frame has the wrong pixel format and thus a wrong
linesize, which results in a too small zero buffer being allocated.
This fixes an out-of-bounds read in ls_decode_line.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 7ea2db6eafa0a8a9497aab20be2cfc8742a59072)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This is used to check if the input buffer is large enough, so if this
overflows it can cause a false negative leading to a segmentation fault
in bytestream2_get_bufferu.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9d38f06d05efbb9d6196c27668eb943e934943ae)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This macro unconditionally used out[-1], which causes an out of bounds
read, if out is the very beginning of the buffer.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 90b99a81071d10e6b5efe86a4602d54d4f45bbcb)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
More don't fit into the integer output.
Also use get_bits_long, since get_bits only supports reading up to 25
bits, while get_bits_long supports the full integer range.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 4d5c3b02e9d2c9a630ca433fabca43285879e0b8)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Otherwise the too samll buffer is directly used in the frame, causing
segmentation faults, when trying to use the frame.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes mozilla bug 1233606
Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b92b4775a0d07cacfdd2b4be6511f3cb362c977b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes ubsan warning
Fixes Mozilla bug 1230276
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c8ea57664fe3ad611c9ecd234670544ddff7ca55)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This also suppresses a ubsan warning
Fixes Mozilla bug 1230247
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ef8f6464a55db730cab8c48a1a51fa4e6ca12107)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The change was not correct and broke H264
This reverts commit cd83f899c94f691b045697d12efa21f83eb2329f.
(cherry picked from commit 95b59bfb9d9e47de8438183a035e02667946f27c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
If that is the case, the loop setting predictor_state in
sonic_decode_frame causes out of bounds reads of int_samples, which has
only frame_size number of elements.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9637c2531f7eb040ad1c3cb46cb40a63dfc77b80)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Due to this typo max_center can be too large, causing nlsf to be set to
too large values, which in turn can cause nlsf[i - 1] + min_delta[i] to
overflow to a negative value, which is not allowed for nlsf and can
cause an out of bounds read in silk_lsf2lpc.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit f61d44b74aaae1d306d8a0d38b7b3d4292c89ced)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Also correct the check to reject log < 7, because UPDATE_CACHE only
guarantees 25 meaningful bits.
This fixes undefined behavior:
runtime error: shift exponent is negative
Testing with START/STOP timers in get_ue_golomb, one for the first
branch (A) and one for the second (B), shows that there is practically no
slowdown, e.g. for the cavs decoder:
With the check in the B branch:
629 decicycles in get_ue_golomb B, 4194260 runs, 44 skips
433 decicycles in get_ue_golomb A,268434102 runs, 1354 skips
Without the check:
624 decicycles in get_ue_golomb B, 4194273 runs, 31 skips
433 decicycles in get_ue_golomb A,268434203 runs, 1253 skips
Since the B branch is executed far less often than the A branch, this
change is negligible, even more so for the h264 decoder, where the ratio
B/A is a lot smaller.
Fixes: mozilla bug 1230239
Fixes: fbeb8b2c7c996e9b91c6b1af319d7ebc/asan_heap-oob_195450f_2743_e8856ece4579ea486670be2b236099a0.bit
Found-by: Tyson Smith
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 22e960ad478e568f4094971a58c6ad8f549c0180)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If the input contains too many too large values, the imdct can overflow.
Even if it didn't, the output would be larger than the valid range of 29
bits.
Note that this is a very delicate limit: Allowing values up to 1<<25
does not prevent input larger than 1<<29 from arriving at
sbr_sum_square, while limiting values to 1<<23 breaks the
fate-aac-fixed-al_sbr_hq_cm_48_5.1 test.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit fdc94db37e89165964fdf34f1cd7632e44108bd0)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a9c20e922cee435c9ad2dc78f6c50651f353329c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes a division by zero in the aac_fixed decoder.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ff8816f7172b94028131ee2426ba35e875d973ae)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes out-of-bounds reads in avoid_clipping.
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5b0da6999fdd0135b6f269d3691d74720f773c85)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If minq is negative, the range of sf_idx can be larger than
SCALE_MAX_DIFF allows, causing assertion failures later in
encode_scale_factors.
Reviewed-by: Claudio Freire <klaussfreire@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 7a4652dd5da0502ff21c183b5ca7d76b1cfd6c51)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The transpose_4x4H is wrong which cost me much time to find this bug. The orders of r2 and r3 are wrong,
this bug waste me much time while I make aarch64 arm instruction which used the function.
(cherry picked from commit c18176bd551b4616757080376707637e30547fd0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
If the chroma components are subsampled, smaller buffers are allocated
for them. In that case the maximal block_offset for the chroma
components is not as large as for the luma component.
This fixes out of bounds writes causing segmentation faults or memory
corruption.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5adb5d9d894aa495e7bf9557b4c78350cbfc9d32)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
