Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3051e7fa712dfe2136f19b7157211453895f2a3c)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes CID1257799
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c64b2d480b4a35d4face9928b4265a0fda3f3dd9)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes CID1297594 part 1
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0f3e6959bfa67d12cd5a173b86eb15abd7d9e4d5)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes CID1271783
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ade8a46154cb45c88b1cb5c616eaa6320c941187)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c131a9fead5bf63215b6e1172b3c5c183cf90b85)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes CID1271810
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cdd25f9a3df3905543a5546cf6076d2eaf895736)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes CID1239055
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 294469416d8193a28710d802bb0c46e5fa09fad7)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes CID1210526
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d201becfc0d89c6a5dfe44e96f1044fbc2aadb70)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2d15588124ab1d4c0612cab66f02a716f1509211)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c4c6aea397f62421bf8ef0449b2b465a53e4ab4d)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9f0b898e8258a9a51b290f4c145388d62080f868)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes: CID1239152
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a9bf628bfdad142763880a3d1ccb6058040dda57)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes CID1239110
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a6a45774d045007f8262cd7c614804390e53122e)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Personally, I need the decoder to back out if get_format() returns no
usable pixel format. This didn't work because the error code was not
propagated down the call chain. This in turn happened because the
variable declaration removed in this patch shadowed the variable, whose
value is returned at the end of the function. Consequently, failures of
decode_nal_unit() were ignored in this place.
Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cc5e4bb48476a89cc8ce0c41bc2bd2e8fda9b37c)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 139e1c8009df7729a53eaaae7036ca01071aced5)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes CID1239106
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 22f15f5735389e992ec9aed43b0680e75746b3a1)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This avoids potential accesses over the end
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 93cfa7d1692c25cff045f99ba1af2c9e5772c45e)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 81cf9108563510dee24f73b2c5d94a7bd07ff747)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This fixes nothing but maybe helps coverity which does not see that this is failing later
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 65e5032955cb5022f0f39160aa3839f0799456bd)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes CID1239099 part 2
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1c6ae98d4a9ff9ea607df87908393eda4ebdf4e8)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes CID1239099 part 1
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3e9d5e16ad9799f6b6faae4f21120d23146b84c9)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes CID1271794
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 93b0ee21a2f534f6d3b812686f3acde110e94f18)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes CID1239154
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8a62b80ce6c8e87e7937f9a5d68f83882c1c8da2)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes Ticket4533
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7f45f7fc7f4977e3a0697dfa0771015b4b985e24)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d43cd6b08ed555c303478e3133717fbb2236be6e)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 042260cde4ecf716438c5fc92d15ad5f037ee2e1)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
this quiets a warning:
Failed to set VP8E_SET_NOISE_SENSITIVITY codec control: Unspecified
internal error
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: James Zern <jzern@google.com>
(cherry picked from commit 238ec505e263e7d0363798a05c1c28f494a6300a)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
See sample vp90-2-14-resize-fp-tiles-16-8.webm from the vp9 test vector
set to reproduce the issue.
(cherry picked from commit 3de13d5212b6a0ff146dc0b50e34ee7dbc822eef)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e06dde52d75ac216c98c8479f7f8f0d4566024af)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
See sample vp90-2-14-resize-fp-tiles-16-8-4-2-1.webm from the vp9 test
vector set which reproduces the issue. This probably costs a few cycles,
but I don't think there's an easy way to workaround that.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d02d04a18f300ebe97319ca6e91fc943cb14f58b)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Since commit 676a395a aac->frame->data is not necessarily allocated at
the end of aac_decode_frame_int if avctx->channels is 0.
In this case a bogus frame without any data, but non-zero nb_samples is
returned.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ec38a1ba404b8cb8d71ccee2b8dcd6f3fcbde273)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 584cc1ade10a3297ef9c107ef3a2081c04024156)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
In init_planes p->xblen and p->yblen are set to:
p->xblen = s->plane[0].xblen >> s->chroma_x_shift;
p->yblen = s->plane[0].yblen >> s->chroma_y_shift;
These are later used as block_w and block_h arguments of
s->vdsp.emulated_edge_mc. If one of them is 0 it triggers an av_assert2
in emulated_edge_mc:
av_assert2(start_x < end_x && block_w > 0);
av_assert2(start_y < end_y && block_h > 0);
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 75fc81c8318505aa7946e05a9bee08d47241fc66)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
s->ref_pics[i] is later used as ref argument of interpolate_refplane,
where it is dereferenced.
If it is NULL, it causes a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d93181ef3eacdb862d93448f31c97765a523d1db)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If bytes is large enough, bytes*8 can overflow and become negative.
In that case 'bufsize -= bytes*8' causes bufsize to increase instead of
decrease.
This leads to a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9e66b39aa87eb653a6e5d15f70b792ccbf719de7)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
buf_idx + data_unit_size can overflow, causing the '> buf_size' check to
wrongly fail.
This causes a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 984f50deb2d48f6844d65e10991b996a6d29e87c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
AV_PIX_FMT_MONOBLACK has the AV_PIX_FMT_FLAG_BITSTREAM flag, i.e.
linesize can be smaller than width.
Since x_offset is only check against the width, this can lead to
x_offset * bpp >= image_linesize.
In this case ptr could be set to a position outside the image_buf in
png_handle_row, leading to memory corruption and thus crashes.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 372aa0777aaacf726de7cd7dd0e6797026a124ee)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
During the loop ret can get changed. Since it is not set on all failure
paths, decode_frame_common can return 0 even though an error occurred.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f760be4d312bb6e78f80d39b9d0062253332e08)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This check was removed in commit 08aec6f6, but
s->last_picture.f->data[0] is still used in handle_p_frame_apng
unconditionally.
This fixes a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 287dbb0771d558b336e377d0594e26c0a6291755)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
s->decoded_buffer is allocated with a min_size of:
2 * FFALIGN(blockstodecode, 8) * sizeof(*s->decoded_buffer)
Then it is assigned to s->decoded[0] (and s->decoded_buffer + FFALIGN(blockstodecode, 8)
to s->decoded[1]) and passed as out buffer to decode_array_0000.
In this function 64 elements of the out buffer are written
unconditionally and outside the array if blockstodecode is too small.
This causes memory corruption, leading to segmentation faults or other
crashes.
Thus change decode_array_0000 to write at most blockstodecode elements
of the out buffer.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 699341d647f7af785fb8ceed67604467b0b9ab12)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Otherwise range_start_decoding is not necessarily run and thus
ctx->rc.range still 0 in range_dec_normalize leading to an infinite
loop.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 464c49155ce7ffc88ed39eb2511e7a75565c24be)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If rice_limit is 0, k can be 0 in decode_scalar, which calls show_bits(gb, k).
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4b657a1b1eedcf38bcf36e89a2f4be6f76b5ce09)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
For max_order = 0 the clipping range is invalid. (amin = 2, amax = 1)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 58d605ee9b3277289278dc40e022311f8e083833)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
