Fixes out of array access
Fixes: 08664a2a7921ef48172f26495c7455be/asan_heap-oob_23036c6_3301_523388ef84285a0270caf67a43247b59.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d85aa76115)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 007c4a36608ebdf27ee260ad60a81184/asan_heap-oob_32076b4_2243_116b1cb29d91cc4974d6680e3d10bd91.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ef9f7bbfa4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Personally, I need the decoder to back out if get_format() returns no
usable pixel format. This didn't work because the error code was not
propagated down the call chain. This in turn happened because the
variable declaration removed in this patch shadowed the variable, whose
value is returned at the end of the function. Consequently, failures of
decode_nal_unit() were ignored in this place.
Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cc5e4bb484)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes CID1239099 part 2
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1c6ae98d4a)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes CID1239099 part 1
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3e9d5e16ad)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
hevc seems to be the only place where the C implementation
of the av_clip function is explicitly selected, precluding
platform-specific optimizations
Signed-off-by: Peter Meerwald <pmeerw@pmeerw.net>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
The buffer pointers would be otherwise overwritten, causing a
leak on e.g. PERSIST_RPARAM_A_RExt_Sony_1.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Use edge emu buffers
And enable the code unconditionally
Speed difference without USE_SAO_SMALL_BUFFER and with the new code:
Decicycles: 26772->26220 (BO32), 83803->80942 (BO64)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
cherry picked from commit 5d9f79edef2c11b915bdac3a025b59a32082f409
SAO edge filter uses pre-SAO pixel data on the left and top of the ctb, so
this data must be kept available. This was done previously by having 2
copies of the frame, one before and one after SAO.
This commit reduces the storage to just that, instead of the previous whole
frame.
Commit message taken from patch by Christophe Gisquet <christophe.gisquet@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit '443b71928b2f36362e805c037751e6c3c79ea4e8':
hevc: unref the current frame if frame_start() fails
Conflicts:
libavcodec/hevc.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
This avoids use of uninitialized memory
Fixes: asan_static-oob_17aa046_582_cov_212287884_DBLK_G_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'cd975d5658a1cbe99939df75db59d5ae9fbcb4e0':
hevc: Spin the mv_mpv_mode calls in a stand alone function
Conflicts:
libavcodec/hevc.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit '16c01fb4347312b6d29a6498dad627665b96a20e':
hevc: remove an unused function parameter
Conflicts:
libavcodec/hevc.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
