This limits ABI issues in case libavcodec is linked to a libavutil with larger AVFrame
Which can happen if they are shiped in seperate binary packages and libavutil is upgraded
A cleaner alternative would be to replace them by pointers but this would likely cause
a small speedloss
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fc567ac49e17151f00f31b59030cd10f952612ef)
Conflicts:
libavcodec/h264.h
(cherry picked from commit 618d062bd553a7d7fad194d4236913f2b0a0251e)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The actual predictor value, set by the trellis code, never
was written back into the variable that was written into
the block header. This was accidentally removed in b304244b.
This significantly improves the audio quality of the trellis
case, which was plain broken since b304244b.
Encoding IMA QT with trellis still actually gives a slightly
worse quality than without trellis, since the trellis encoder
doesn't use the exact same way of rounding as in
adpcm_ima_qt_compress_sample and adpcm_ima_qt_expand_nibble.
Fixes part of Ticket3701
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fa8f060b75bf9074792a0f9ff4ed002652ef62b8)
Conflicts:
tests/ref/acodec/adpcm-ima_qt-trellis
This was broken in 095be4fb - samples+ch (for the previous
non-planar case) equals &samples_p[ch][0]. The confusion
probably stemmed from the IMA WAV case where it originally
was &samples[avctx->channels + ch], which was correctly
changed into &samples_p[ch][1].
Fixes part of Ticket3701
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a32765c4252eb106a2ade543026ef6f59e699bfa)
Fixes a regression since fb3e380 similar to ticket #2661,
reported by fluffrabbit at aol dot com.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 673716c54b39eba9579a38ad222130e3f9549167)
This is a bit tricky, we allocate a correctly sized AVFrame but then only
copy the compile time AVFrame size, this is to ensure that user applications
which do not use the correct av frame API dont end with out of array reads.
Note, applications using the correct API have set extended_data and the
changed code will never be executed for them.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8ab80707841a73ca7708e1e1aa97f3513fff3d35)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array accesses
This should not affect any release
Fixes: 8ab69af9e5a7a7e20fe04cdd25c0d6e7-asan_heap-oob_e72b82_5505_cov_2278389485_g2m4.wmv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6b53c1aa822e9c92be52a462dd0aef1c2010ce73)
Conflicts:
libavcodec/g2meet.c
These ADPCM codecs include a per-frame flag that enables a raw 16-bit mode. Therefore
the the number of samples returned by get_nb_samples() is only ever approximate.
Fixes ticket #3460.
Signed-off-by: Peter Ross <pross@xvid.org>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7380201451a2edfb240cd356579c4c39a87cf5bd)
Fixes ticket #3461.
Signed-off-by: Peter Ross <pross@xvid.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d1bb17940dd242e56541775318636bcbe3eab73d)
avcodec_flush_buffers() must release all internally held references
according to its documentation, for which all the threads need to be
flushed.
Bug-Id: vlc/9665
(cherry picked from commit d1f9563d502037239185c11578cc614bdf0c5870)
Conflicts:
libavcodec/pthread_frame.c
Fixes use of uninitialized memory
Fixes: 93728afd9aa074ba14a09bfd93a632fd-asan_static-oob_124a17d_1445_cov_1021181966_DBLK_D_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 64278039e55ffc88d231a8d760ecc257a120760a)
Conflicts:
libavcodec/hevc_filter.c
Fixes out of array read
Fixes: 08e48e9daae7d8f8ab6dbe3919e797e5-asan_heap-oob_157461c_5295_cov_1266798650_firefing.mpg
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3edc3b159503d512c919b3d5902f7026e961823a)
Conflicts:
libavcodec/mpeg4videodec.c
Fixes out of array read
Fixes: 1cb91c36c4e55463f14aacb9bdf55b38-asan_heap-oob_106cbce_5617_cov_11212800_h264_mmx_chroma_intra_lf.mp4
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8c55ff393340998faae887dfac19e7ef128e1e58)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array read
Fixes: caa65cc01655505705129b677189f036-signal_sigsegv_fdcc43_2681_cov_3043376737_PPH422I5_Panasonic_A.264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 91253839e14cce9793ee93f184cef609ca8195d5)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array access
Fixes: abd3c041acbcb816be113455d138166b-asan_heap-oob_b11634_3707_cov_1707137151_als_05_2ch48k16b.mp4
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 18f94df8af04f2c02a25a7dec512289feff6517f)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This ensures the previous one is matching the curent and not just any
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1a3ed056c523b4670e192301be15dbc521ec8353)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This allows us to detect which areas have failed to decode
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a18f11158216c22f4a69e44f8cbb59b300a7f10c)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes use of uninitialized memory
Fixes out of array read
Fixes assertion failure
Fixes part of cb307d24befbd109c6f054008d6777b5/asan_static-oob_124a175_1445_cov_2355279992_DBLK_D_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6ef57f4d9a0920c82237facb0d1f3856b17da9dc)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes inconsistencies
Fixes use of uninitilaized memory
Fixes part of cb307d24befbd109c6f054008d6777b5/asan_static-oob_124a175_1445_cov_2355279992_DBLK_D_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 56985d26d7054079cbe8865532c4a2cff123f596)
Conflicts:
libavcodec/hevc.c
This avoids them being cleared before the full initialization finished
Fixes out of array read
Fixes: asan_heap-oob_f0c5e6_7071_cov_1605985132_mov_h264_aac__Demo_FlagOfOurFathers.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8a3b85f3a7952c54a2c36ba1797f7e0cde9f85aa)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Such changes are forbidden in H.264 and lead to race conditions
Fixes out of array read
Fixes: signal_sigsegv_f9796a_1613_cov_3114610371_FM1_BT_B.h264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e708424b70bef8641e8a090ec4d9e8c4490db87e)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array read
Fixes: asan_static-oob_1efed25_1887_cov_2013541199_HeyYa_RA10_AAC_192K_30s.rm
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6e42ccb9dbc13836cd52cda594f819d17af9afa2)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes use of uninitialized memory
Fixes out of array read
Fixes: asan_static-oob_123cee5_2630_cov_1869071233_PICSIZE_A_Bossen_1.bin
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 96c4ba2392b9cd55a5e84cb28db5c0c7e53cd390)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 388b4cf86ed5ec27d35eb5069769db12a4e31af0)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This case could occur when cuting and concatenating bitstreams
Fixes out of array read
Fixes: asan_heap-oob_1b33fdd_2849_cov_478905890_SA10143.vc1
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 85d51d8e327c666ac963acf25cf6a6763e6c6671)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array read
Fixes: d4476f68ca1c1c57afbc45806f581963-asan_heap-oob_2266b27_8607_cov_4044577381_snow_chroma_bug.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 61d59703c91869f4e5cdacd8d6be52f8b89d4ba4)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array read
Fixes: 5f9698e86d92f19bb08d54ff0d57027f-signal_sigsegv_b30756_3795_cov_2693691257_ansi256.ans
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d42ec8433c687fcbccefa51a7716d81920218e4f)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array access
Fixes: 14a74a0a2dc67ede543f0e35d834fbbe-asan_heap-oob_49572c_556_cov_215466444_44_001_engine_room.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c919e1ca2ecfc47d796382973ba0e48b8f6f92a2)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes inconsistency and out of array accesses
Fixes: 10cdd7e63e7f66e3e66273939e0863dd-asan_heap-oob_1a4ff32_7078_cov_4056274555_mov_h264_aac__mp4box_frag.mp4
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e5c7229999182ad1cef13b9eca050dba7a5a08da)
Conflicts:
libavcodec/utils.c
Fixes integer overflow
Fixes out of array accesses
Fixes 2f65e7dbd02a12f426a423bd7bf880b4-signal_sigsegv_127c952_2793_cov_2517424539_RPLM_A_qualcomm_4.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d1e6602665d5ec1b7e211ab27b298c26139f82cc)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array access
Fixes: asan_heap-oob_19c7a94_6470_cov_1453611734_luckynight-partial.tak
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f58eab151214d2d35ff0973f2b3e51c5eb372da4)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array accesses with CODEC_FLAG_EMU_EDGE
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8001e9f7d17e90b4b0898ba64e3b8bbd716c513c)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Prevents out of array accesses with CODEC_FLAG_EMU_EDGE
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6ba02602aa7fc7d38db582e75b8b093fb3c1608d)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes assertion failure
Fixes part of msan_uninit-mem_7fa0d8c8bd58_8417_sh3.ape
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d5128fce38646d3f64c55feda42084888ba0e87e)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array access with RC_VARIANCE set to 0
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f1caaa1c61310beba705957e6366f0392a0b005b)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
