Commit Graph

96 Commits

Author SHA1 Message Date
Ronald S. Bultje
2254b559cb swscale: make filterPos 32bit.
Fixes overflows for large image sizes.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-03-06 10:47:41 -08:00
Ronald S. Bultje
791de61bbb swscale: fix another integer overflow.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-02-29 11:19:03 -08:00
Ronald S. Bultje
2bb628f870 swscale: fix another integer overflow at large dimensions/rescales. 2012-02-24 15:39:55 -08:00
Ronald S. Bultje
19a65b5be4 swscale: fix overflows in filterPos[] calculation for large sizes.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-02-23 10:30:07 -08:00
Ronald S. Bultje
1254022ea7 swscale: fix filtersize clipping.
if srcW<=2, clip(x, 1, srcW-2) still allows srcW to be < 1.
2012-02-22 16:54:20 -08:00
Ronald S. Bultje
dae2ce361a swscale: enforce a minimum filtersize.
At very small dimensions, this calculation could lead to zero-sized
filters, which leads to uninitialized output, zero-sized allocations,
loop overflows in SIMD that uses do{..}while(i++<filtersize); instead
of for(i=0;i<filtersize;i++){..} and several other similar failures.
Therefore, require a minimum filtersize of 1.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-02-11 10:06:50 -08:00
Alex Converse
7181c4edee cosmetics: Remove extra newlines at EOF 2012-01-27 17:19:09 -08:00
Alex Converse
08628b6afb Revert "sws/pixfmt/pixdesc: add support for yuva444p"
This reverts commit fc115c80b7.

Tests are broken.
2012-01-26 18:06:57 -08:00
Paul B Mahol
fc115c80b7 sws/pixfmt/pixdesc: add support for yuva444p
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
2012-01-27 09:24:08 +08:00
Diego Biurrun
e44c11e9fa cosmetics: Move static and inline attributes to more standard places.
Fixes several "‘static’ is not at beginning of declaration" warnings.
2012-01-14 17:32:36 +01:00
Paul B Mahol
5cad970971 swscale: RGB4444 and BGR444 input
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
2012-01-08 20:19:41 -08:00
Ronald S. Bultje
9ea3501d87 swscale: don't show full-chroma-int warning for non-RGB output.
Non-RGB output always uses full chroma interpolation.
2012-01-03 20:01:21 -08:00
Ronald S. Bultje
d49352c7cc swscale: fix overflows in vertical scaling at top/bottom edges.
This fixes integer multiplication overflows in RGB48 output
(vertical) scaling as detected by IOC. What happens is that for
certain types of filters (lanczos, spline, bicubic), the
intermediate sum of coefficients in the middle of a filter can
be larger than the fixed-point equivalent of 1.0, even if the
final sum is 1.0. This is fine and we support that.

However, at frame edges, initFilter() will merge the coefficients
for the off-screen pixels into the top or bottom pixel, such as
to emulate edge extension. This means that suddenly, a single
coefficient can be larger than the fixed-point equivalent of
1.0, which the vertical scaling routines do not support.

Therefore, remove the merging of coefficients for edges for
the vertical scaling filter, and instead add edge detection
to the scaler itself so that it copies the pointers (not data)
for the edges (i.e. it uses line[0] for line[-1] as well), so
that a single coefficient is never larger than the fixed-point
equivalent of 1.0.
2011-12-18 08:27:43 -08:00
Reinhard Tartler
5089ce1b5a swscale: #include "libavutil/mathematics.h"
this file uses the M_PI macro since
4e74187db2, so include the correct header
directly.

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2011-12-01 19:12:26 +01:00
Martin Storsjö
f32dfad9dc swscale: Readd #define _SVID_SOURCE
This was removed erroneously in
046f081b46. This define still is
necessary for getting MAP_ANONYMOUS defined on linux/glibc,
despite the define reshuffling done in that commit.

Without MAP_ANONYMOUS defined, the mprotect calls for setting the
generated mmx2 scaler code pages executable are left out, causing
crashes if that codepath is chosen.

This patch fixes scaling from 192x144 to 320x240 with
-sws_flags fast_bilinear, which crashes on linux at the
moment.

Signed-off-by: Martin Storsjö <martin@martin.st>
2011-11-25 19:59:15 +02:00
Ronald S. Bultje
185655c601 swscale: add support for planar RGB input. 2011-11-24 10:40:05 -08:00
Sean McGovern
124e56454d swscale: add padding to conversion buffer.
Altivec does unaligned reads from this buffer in
hscale_altivec_real(), and can thus read up to 16 bytes beyond
the end of the buffer. Therefore, add an extra 16 bytes of
padding at the end of the conversion buffer.

This fixes fate-lavfi-pixfmts_scale on AltiVec-enabled builds
under valgrind.

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
2011-11-11 07:44:35 -08:00
Ronald S. Bultje
1deb08fcb6 swscale: align vertical filtersize by 2 on x86.
The vertical scaler handles 2 rows at a time and thus requires
alignment by 2, or else it'll read invalid memory and result in
corrupt output.
2011-11-05 07:06:38 -07:00
Ronald S. Bultje
dc49bf1270 sws/pixfmt/pixdesc: add support for yuv420p9le/be. 2011-10-21 00:58:01 -07:00
Ronald S. Bultje
8305041e13 swscale: prevent overflow in coefficient calculation. 2011-10-21 00:14:11 -07:00
Ronald Bultje
d1d421cbc0 swscale: prevent overflow during initialization
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
2011-10-18 10:29:49 +02:00
Anton Khirnov
04de1569cd sws: support yuv444p9/10 output. 2011-10-12 08:27:30 +02:00
Ronald S. Bultje
ea540401d6 swscale: fix byte overreads in SSE-optimized hscale().
SSE-optimized hScale() scales up to 4 pixels at once, so we need to
allocate up to 3 padding pixels to prevent overreads. This fixes
valgrind errors in various swscale-tests on fate.
2011-09-15 07:30:46 -07:00
Ronald S. Bultje
28c1115a91 swscale: use 15-bit intermediates for 9/10-bit scaling. 2011-08-12 11:54:25 -07:00
Stefano Sabatini
6a92551234 lsws: remove optimization debug logs in sws_init_context()
The logged information is possibly false, and it tends to be outdated
after each change since the logging code needs to be manually updated.

Simplify and prevent confusing wrong debug messages.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2011-08-05 00:45:06 +02:00
Stefano Sabatini
347167ecf5 lsws: use array for storing the supported in/out information
Also remove the unnecessary isSupportedIn/Out macros.

Make the code more compact/readable, and simplify the access to
lsws-specific pixel format information.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2011-08-05 00:44:11 +02:00
Ronald S. Bultje
62ee0e6a97 Revert "swscale: use 15-bit intermediates for 9/10-bit scaling."
This reverts commit ac0fb59348. It
causes valgrind errors which I'll want to investigate before
resubmitting this.
2011-08-02 12:27:43 -07:00
Ronald S. Bultje
ac0fb59348 swscale: use 15-bit intermediates for 9/10-bit scaling. 2011-08-02 10:34:02 -07:00
Joseph Artsimovich
eedb1f2034 swscale: mark YUV422P10(LE,BE) as supported for output
Signed-off-by: Mans Rullgard <mans@mansr.com>
2011-07-21 02:10:31 +01:00
Ronald S. Bultje
baba2eedac swscale: extend mmx padding.
Fixes a crash when forcing libc to strictly adhere to malloc sizes.
2011-07-14 13:27:06 -07:00
Diego Biurrun
96c1e6d40d doxygen: Make sure parameter names match between .c and .h files. 2011-07-14 04:09:49 +02:00
Ronald S. Bultje
5c391a161a swscale: rename uv_off/uv_off2 to uv_off_px/byte. 2011-07-08 15:15:31 -07:00
Ronald S. Bultje
42d622fab3 swscale: fix 16-bit scaling when output is 8-bits.
We would use the second half of the U plane buffer, rather than the
V plane buffer, to output the V plane pixels.
2011-07-08 15:15:30 -07:00
Ronald S. Bultje
8f440223f6 swscale: disable full_chroma_int when converting to non-24/32bpp RGB.
This functionality is only implemented for RGB24/32 and causes crashes
otherwise.
2011-07-02 08:33:52 -07:00
Ronald S. Bultje
8a8d0ce208 swscale: for >8bit scaling, read in native bit-depth.
For 9/10bit, it means we don't have to upscale to 16bit before
actual scaling or pixel format conversion, and thus a performance
gain.
2011-07-01 09:08:26 -07:00
Ronald S. Bultje
ef1ee362b3 swscale: implement >8bit scaling support.
This means that precision is retained when scaling between sample
formats with >8 bits per component (48bit RGB, 16bit grayscale,
9/10/16bit YUV).
2011-06-29 09:45:52 -07:00
Ronald S. Bultje
dff5a83532 swscale: re-add support for non-native endianness.
This works through some non-obvious hacks in utils.c.
2011-06-27 18:10:41 -07:00
Janne Grunau
3636e791ec swscale: use SwsContext for av_log when available
Signed-off-by: Diego Biurrun <diego@biurrun.de>
2011-06-15 13:20:20 +02:00
Ronald S. Bultje
103278f7b0 libavutil/swscale: YUV444P10/YUV444P9 support.
Also add missing glue code for recently added YUV422P10 formats
to swscale.

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
2011-06-10 15:12:24 -04:00
Ronald S. Bultje
6d4d483eee swscale: de-macro'ify RGB15/16/32 input functions.
Inline functions are easier to read, maintain, modify and test,
which justifies the slightly increased source size. This patch
also adds support for non-native endianness RGB15/16 and fixes
isSupportedOutput() to no longer claim that we support writing
non-native RGB565/555/444.
2011-06-09 16:47:45 -04:00
Ronald S. Bultje
808d8ff6bb swscale: allocate larger buffer to handle altivec overreads.
Altivec sws code intentionally overreads buffers for better performance,
so we need to allocate larger buffers to handle that.
2011-06-02 11:25:56 -07:00
Ronald S. Bultje
009f829dde swscale: fix crash in bilinear scaling. 2011-05-28 08:59:56 -04:00
Ronald S. Bultje
389e2000eb swscale: delay allocation of formatConvBuffer().
That means it won't be allocated when not needed. Alongside
this, it fixes valgrind/fate-detected memory leaks.
2011-05-27 13:22:27 -04:00
Ronald S. Bultje
0f4eb8b043 swscale: remove VOF/VOFW. 2011-05-26 09:31:27 -04:00
Ronald S. Bultje
b4a224c5e4 swscale: split chroma buffers into separate U/V planes.
Preparatory step to implement support for sizes > VOFW.
2011-05-26 09:31:27 -04:00
Ronald S. Bultje
69645c021a swscale: replace formatConvBuffer[VOF] by allocated array.
This allows to convert between formats of arbitrary width,
regardless of the value of VOF/VOFW.
2011-05-26 09:31:02 -04:00
Diego Biurrun
0093199184 swscale: Remove commented-out printf cruft. 2011-05-26 03:45:58 +02:00
Ronald S. Bultje
c4fd283a46 swscale: use emms_c(). 2011-05-24 14:48:49 -04:00
Ronald S. Bultje
e66149e714 swscale: force --enable-runtime-cpudetect and remove SWS_CPU_CAPS_*. 2011-05-24 10:03:26 -04:00
Diego Biurrun
046f081b46 configure: Do not unconditionally add -D_POSIX_C_SOURCE to CPPFLAGS.
Adding _POSIX_C_SOURCE to CPPFLAGS globally produces all sorts of problems
since it causes certain system functions to be hidden on some (BSD) systems.
The solution is to only add the flag on systems that really require it, i.e.
glibc-based ones.

This change makes BSD systems compile out-of-the-box without the need for
adding specific flags manually.  It also allows dropping a number of flags
set manually on a file-per-file basis, but were only present to work around
breakage introduced by the presence of _POSIX_C_SOURCE.

Also add _XOPEN_SOURCE to CPPFLAGS for glibc systems.  We use XSI extensions
in several places already, so it is preferable to define it globally instead
of littering source files with individual #defines only needed for glibc.
2011-05-12 11:41:59 +02:00