The tile 0 cannot depend on a previous one.
Prevent an out of array bound load in ff_hevc_cabac_init().
Fixes: asan_heap-oob_e3a924_1630_DBLK_A_MAIN10_VIXS_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Guillaume Martres <smarter@ubuntu.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 48a5b155433ed7af20fb0a5c20ca131958727727)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The src buffer should only contain values in the interval
[0, (1 << BIT_DEPTH) - 1]. Since shift = (BIT_DEPTH - 5), src[x] >> shift
must be in the interval [0, 31], so no clip is needed.
This removes the code that was changed in 5856bca360c5bc3e340a357d91b1f993c80a7bea
as the clip that was repositioned in that commit is removed
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b00a8b4d194f1bf23343f3f42138affa1fe26641)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes: asan_stack-oob_eae8e3_7333_WPP_B_ericsson_MAIN10_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
This is a more proper fix than 5856bca360c5bc3e340a357d91b1f993c80a7bea
The reconstructed picture should always be clipped (see section 8.6.5),
previously we did not clip coding units where
cu_transquant_bypass_flag == 1
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c9fe0caf7a1abde7ca0b1a359f551103064867b1)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Prevent an out of array bound read.
Fixes: asan_stack-oob_eae8e3_7333_WPP_B_ericsson_MAIN10_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5856bca360c5bc3e340a357d91b1f993c80a7bea)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes use of uninitialized memory and out of stack array read
Fixes: signal_sigsegv_ecc526_7846_WPP_C_ericsson_MAIN_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0999f1613bc48ed9d6578a3ad7bcd17610e07fbf)
Conflicts:
libavcodec/hevc.c
Fixes use of uninitialized memory
Fixes: msan_uninit-mem_7f8b64436530_7895_quicktime_newcodec_applelosslessaudiocodec.m4a
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e11983bda073f8c63f60509ee753da9fba20ed10)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes use of uninitialized memory
partly fixes: msan_uninit-mem_7f7834b6a530_6473_luckynight-partial.wma
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6b18a6839b43ea78e70cd3e35f781d1c955bda73)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes use of uninitialized memory
partly fixes: msan_uninit-mem_7f7834b6a530_6473_luckynight-partial.wma
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ae3856dcaf9c5ef339969c95a72bcaf7c4bba9ec)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 03fff09b32171e0c76d104c02ebf578c7f4fe21d)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Avoids len from becoming negative and causing assertion failure
Fixes: signal_sigabrt_7ffff7126425_5140_fd44dc63fa7bdd12ee34fc602231ef02.jpg
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6060234d43dcf0b5200cdd7dbd2f1542146827eb)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array read
Fixes: asan_heap-uaf_ae6067_5415_g2m4.wmv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6d9dad6a7cb5d544d540abf941fedbd34c14d2bd)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array read
Fixes assertion failure
Fixes asan_static-oob_16431c0_8036_rio_bravo_mono_64_spx.ac3
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4782c4284fa3856a9b6910fe5ff6e4fb1c65b58c)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array accesses
Fixes asan_static-oob_eb9812_5961_iv41.avi
This reverts the merge of c9ef6b09326a24010bf86d6b0d19cfa42df4d546
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit c3d5cd1ebfba8fe36a0da7fad47df7fdf9c4ccd0)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes qp fields becoming out of range
Fixes: asan_static-oob_e393a3_6998_WPP_A_ericsson_MAIN10_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4ced5d7780fea2ea49444d6686d26f26b3a2160f)
Conflicts:
libavcodec/hevc_filter.c
This matches how its done for SPS/PPS.
An alternative to this is to check it when its used.
Fixes null pointer dereference
Fixes: signal_sigsegv_e30a43_1437_CIP_A_Panasonic_3.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d66bab0a69ac1860e78dd951ad8db1a507e75642)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c90f31146e8b1407a4a5808d0d904d85baeed5d4)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3c8b085764ed4b036df4a8908a0781dc6d73ee11)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b8f4410ff60b3a973cd13351d00a1d88eaddfb71)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8443b27072a076abb28d7f2f60bc90e1d5c285df)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1458f0647ca0c882cc1c29892ac130a1056a1f47)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c81234651f761a44a3e72829fd494211e237069c)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fca7943850ecdc1e67a0275b488768be01867f75)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This ensures that theres just one AVFrame allocation function and libs dont
produce multiple AVFrame variants after a minor lib update
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5abdda214df53f009434f19b9eb8e1375f2924d9)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
AVFrames cannot be copied literally, their definition is in
avutil and their extended_data can point to their data[]
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4c1b4ae1baf77df7150fa8cbcece8057a261e47d)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'd4f1188d1a662fed5347e70016da49e01563e8a8':
dv: use AVFrame API properly
Conflicts:
libavcodec/dvdec.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 94a849b8b6c3e4a90361485b2e12a9a5c35833a3)
Conflicts:
libavcodec/dv.h
libavcodec/dvdec.c
libavcodec/dvenc.c
Author of the merged code: Anton Khirnov
* commit 'd351ef47d0e0ccb7de96b37f137c16b2885580ac':
pthread_frame: use the AVFrame API properly.
Conflicts:
libavcodec/pthread_frame.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 45fd4ec9ef2b3a7074c49cdddac6e7dcc127a874)
Conflicts:
libavcodec/pthread_frame.c
Author of the merged code: Anton Khirnov
* commit 'b605b123ef1d3bac0e7c221d8d7fa74cd8c7253c':
mxpegdec: use the AVFrame API properly.
Merged-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8947f47fdfaf7f3a907a334fc65dc724f2fdd23f)
Author of the merged code: Anton Khirnov
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'afa21a12bf084f905187615706b0a8d92bc98661':
p*menc: use the AVFrame API properly.
Conflicts:
libavcodec/Makefile
libavcodec/pamenc.c
libavcodec/pnmenc.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 37945584bfb29f187e38531c90bb02a32014e48d)
Author of the merged code: Anton Khirnov
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'e2274aa555f023e4f4e4819bf29b2d7e0adec7d5':
mjpegdec: use the AVFrame API properly.
Conflicts:
libavcodec/mjpegdec.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 639303867640d1880fad675472bc47e9c95f96c7)
Author of the merged code: Anton Khirnov
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master:
mpegvideo_enc: use the AVFrame API properly.
ffv1: use the AVFrame API properly.
jpegls: use the AVFrame API properly.
huffyuv: use the AVFrame API properly.
Conflicts:
libavcodec/ffv1.c
libavcodec/ffv1.h
libavcodec/ffv1dec.c
libavcodec/ffv1enc.c
Changes to ffv1 are more redone than merged due to them being based on
an ancient codebase and a good part of that having being done already
as well.
Merged-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit be1e6e7503b2f10b0176201418eb97912cee093f)
Conflicts:
libavcodec/ffv1enc.c
libavcodec/mpegvideo.h
libavcodec/mpegvideo_enc.c
Author of the merged code: Anton Khirnov
* commit 'd48c20630214a4effcc920e93a5044bee4e2002e':
qtrleenc: use the AVFrame API properly.
ulti: use the AVFrame API properly.
vc1: use the AVFrame API properly.
flashsv: use the AVFrame API properly.
Conflicts:
libavcodec/flashsv.c
libavcodec/qtrleenc.c
libavcodec/ulti.c
libavcodec/vc1dec.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 92cbd775687204f9750a09c69f97702719036aab)
Conflicts:
libavcodec/flashsv.c
Author of the merged code: Anton Khirnov
* commit 'ffe04c330335add4c6d70ab0bb98e6b3f4f7abfa':
libxvid: use the AVFrame API properly.
pcxenc: use the AVFrame API properly.
roqvideo: remove unused variables
libschroedingerenc: use the AVFrame API properly.
Conflicts:
libavcodec/pcxenc.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f4f7888bab7061f08c54356c285adaba24383dc0)
Author of the merged code: Anton Khirnov
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit '97168b204a0b6b79bb6c5f0d40efdf7fc2262476':
eatgv: use the AVFrame API properly.
libxavs: use the AVFrame API properly.
nuv: use the AVFrame API properly.
flashsvenc: use the AVFrame API properly.
Conflicts:
libavcodec/eatgv.c
libavcodec/nuv.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a0c0629dd963b00f989172f0c599353b6b288c37)
Conflicts:
libavcodec/eatgv.c
Author of the merged code: Anton Khirnov
* commit '57e7b3a89f5a0879ad039e8f04273b48649799a8':
dnxhdenc: use the AVFrame API properly.
libx264: use the AVFrame API properly.
svq1enc: use the AVFrame API properly.
gif: use the AVFrame API properly.
Conflicts:
libavcodec/gif.c
libavcodec/svq1enc.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5b0c70c2499e20529d517b712910d6f4f72e9485)
Author of the merged code: Anton Khirnov
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit '45bde93eefa78c1bdb0936109fbd2e2fb27fbfe7':
sunrastenc: use the AVFrame API properly.
targaenc: use the AVFrame API properly.
tiffenc: use the AVFrame API properly.
pngenc: use the AVFrame API properly.
Conflicts:
libavcodec/pngenc.c
libavcodec/sunrastenc.c
libavcodec/targaenc.c
libavcodec/tiffenc.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3ea168edeb7a20eae1fccf7da66ac7b8c8c791ba)
Author of the merged code: Anton Khirnov
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit '0ea430c75b8d90449d2878ad84669a2da2ad3cbc':
lclenc: use the AVFrame API properly.
Conflicts:
libavcodec/lclenc.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 85b7b0c519f8d9491b4c0340329a605cc97c8984)
Author of the merged code: Anton Khirnov
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'e4155f15b35c4272a235f5521d2dc6c2aabdd462':
eamad: use the AVFrame API properly.
dpxenc: use the AVFrame API properly.
bmpenc: use the AVFrame API properly.
sgienc: use the AVFrame API properly.
Conflicts:
libavcodec/bmpenc.c
libavcodec/dpxenc.c
libavcodec/eamad.c
libavcodec/sgienc.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9ad477d9098b5281cede0bd8525ca90b0e52436d)
Conflicts:
libavcodec/eamad.c
Author of the merged code: Anton Khirnov
