1679 Commits

Author SHA1 Message Date
Michael Niedermayer
852ef62b85 swscale/utils: clear formatConvBuffer on allocation
Fixes use of uninitialized memory
Fixes: asan_heap-oob_35ca682_1474_cov_3230122439_aletrek_tga_16bit.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 007498fc1a639ecee2cda1892cbcff66c7c8c951)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-10 02:13:09 +02:00
Michael Niedermayer
3cca4c770e swscale/utils: More carefully merge and clear coefficients outside the input
Fixes out of array read
Fixes: asan_heap-oob_35ca682_1474_cov_3230122439_aletrek_tga_16bit.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1895d414aaacece3b57d7bf19502305e9a064fae)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-10 02:13:09 +02:00
Michael Niedermayer
0d728ed792 swscale/utils: Limit filter shifting so as not to read from prior the array
Fixes out of array read
Fixes: asan_heap-oob_1fb2f9b_3780_cov_3984375136_usf.mkv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 692b22626ec9a9585f667c124a186b1a9796e432)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-10 02:13:08 +02:00
Vittorio Giovara
3d5c48937b swscale: check memory allocations
Bug-Id: CID 1257779
(cherry picked from commit 1dd797e3c9f179f957316a0becbec048b42df8aa)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-10 02:13:08 +02:00
Michael Niedermayer
8b130c4aab swscale: increase yuv2rgb table headroom
Fixes out of array access
Fixes: case2_bad_read_yuv2rgbx32.mp4
Found-by: Michal Zalewski <lcamtuf@coredump.cx>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-10 02:13:07 +02:00
Michael Niedermayer
a0976c15e6 swscale/x86/rgb2rgb_template: fix crash with tiny size and nv12 output
Fixes Ticket4151

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8524558858b7e14bc50afa10233e0194f591ab9d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-10 02:13:07 +02:00
Michael Niedermayer
ff6ee4571c swscale/x86/rgb2rgb_template: handle the first 2 lines with C in rgb24toyv12_*()
This avoids out of array accesses
Should fix Ticket3451

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4388e78a0f022c8572996f9ab568a39b5f716f9d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-10 02:13:06 +02:00
Michael Niedermayer
0e18480fc3 sws: dont use the optimized 410->420 unscaled conversion when height%4
Fixes Ticket3594
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 421b21ca8a02a346ba03cea3bb2ecc33f791fc30)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
690a3c4fb7 swscale/swscale: fix srcStride/srcSlice typo
Fixes part of Ticket3466
Found by: Andrey_Karpov / PVS-Studio
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 14fa7fc6a81d5e59e05243cdc92108eab1b138ac)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:18:40 +02:00
Michael Niedermayer
218c853774 swscale/x86/swscale_template: loose hardcoded dstw_offset
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f6759d9ad4a8b71e6f212ca4f1e7da9fa56d3298)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:18:40 +02:00
Michael Niedermayer
36fc9bff08 swscale/swscale: fix integer overflow
Should fix fate failure with clang ftrapv

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c9c0451224fd7bc38b4e135e99f114f80c1ae67f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:18:39 +02:00
Michael Niedermayer
9933e06595 swscale/x86/swscale: fix missing xmm clobbers in yuv2yuvX_sse3()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6c47a4e972485e5f0c812159373f703c6f1d089f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:18:39 +02:00
Michael Niedermayer
bd339d4882 swscale/utils: check chroma width for fast bilinear scaler
Fixes artifacts where fast bilinear was used for downscaling chroma

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 037fc3b054b10aee0f11fdbe835e5dffa8e95b37)
2013-12-16 02:24:13 +01:00
Michael Niedermayer
9de71b0eb2 swscale/utils: remove useless ()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 554e913fd7acc9da02ddac2c5ce9487f7f633c92)
2013-12-16 02:24:12 +01:00
Michael Niedermayer
8f9bc6f2ce swscale/input: fix 16bit gbrp input
Fixes Ticket2793

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a4b55bbb6f8ba055bba1493f6872792503e47563)
2013-07-19 14:50:16 +02:00
Michael Niedermayer
1ba01d3d46 swscale: treat gray as a fullrage (0-255) format
Fixes Ticket2684

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-08 15:21:48 +02:00
Michael Niedermayer
034b31df2c swscale: Fix PAL8 input with alpha
Fixes Ticket2158

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-26 15:14:00 +02:00
Hendrik Leppkes
7cdf574c22 swscale: fix filter alignment reduction without inline asm
Fixes scaling in MSVC/ICL, where inline asm is not present, but the
YASM scaling code is.

Reviewed-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-21 21:01:11 +02:00
Carl Eugen Hoyos
7800b09ece Check more completely for invalid pix_fmts in planarRgb16ToRgb16Wrapper()
This makes the conversion function more robust.
2013-06-11 01:14:38 +02:00
Michael Niedermayer
0047da071e swscale/swscale_unscaled: add ()
Fixes order of operations
Fixes CID1030350

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-10 03:34:33 +02:00
Carl Eugen Hoyos
d5978c8678 Add unscaled converter from GBRP > 8bit to RGBx48/64.
Fixes ticket #2633.

Tested-by: Michael Cinquin
2013-06-06 23:57:50 +02:00
Michael Niedermayer
45a73d2b47 Merge commit 'c011ceef78eae66039efc66d9551a7146e08838a'
* commit 'c011ceef78eae66039efc66d9551a7146e08838a':
  swscale: ppc: Remove commented-out define cruft
  nsvdec: Remove commented-out debug cruft
  cpu: Restructure code to avoid pointless ret variable indirection

Conflicts:
	libavutil/cpu.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-03 11:53:05 +02:00
Diego Biurrun
c011ceef78 swscale: ppc: Remove commented-out define cruft 2013-06-02 18:11:59 +02:00
Michael Niedermayer
a90baa63c3 add YUVJ411P
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-21 12:56:11 +02:00
Paul B Mahol
b154d203b1 swscale/output: silence warning about undefined 'ASSERT_LEVEL'
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2013-05-19 14:06:16 +00:00
Paul B Mahol
f4b2d3598b swscale/output: remove unused header
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2013-05-19 14:04:14 +00:00
Michael Niedermayer
e1efacf60b sws/output: yuv2rgb_write() assert that alpha has the correct value in the tables
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-18 15:14:19 +02:00
Michael Niedermayer
5918b7ac41 sws/output: init A1/A2 so that rgba64 ends with 0xffff in the absence of alpha input
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-18 01:20:30 +02:00
Michael Niedermayer
c7c71f95f8 replace remaining PIX_FMT_* flags with AV_PIX_FMT_FLAG_*
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-15 11:23:14 +02:00
Michael Niedermayer
ff4680922f Merge remote-tracking branch 'qatar/master'
* qatar/master:
  pixdesc: rename PIX_FMT_* flags to AV_PIX_FMT_FLAG_*

Conflicts:
	doc/APIchanges
	libavcodec/avpicture.c
	libavcodec/ffv1dec.c
	libavcodec/ffv1enc.c
	libavcodec/imgconvert.c
	libavcodec/tiffenc.c
	libavfilter/vf_pixdesctest.c
	libavfilter/vf_scale.c
	libavutil/imgutils.c
	libavutil/pixdesc.c
	libavutil/version.h
	libswscale/swscale_internal.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-15 11:10:09 +02:00
Anton Khirnov
e6c4ac7b5f pixdesc: rename PIX_FMT_* flags to AV_PIX_FMT_FLAG_* 2013-05-15 07:46:51 +02:00
Michael Niedermayer
e751481cd8 swscale/output: silence compiler warnings about uninitialized variables
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-14 18:58:02 +02:00
Michael Niedermayer
145e644aee sws/output: remove offset/coef for alpha in rgba64
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-12 22:01:18 +02:00
Paul B Mahol
e1746d057b swscale: RGBA64 output
Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-12 21:47:19 +02:00
Michael Niedermayer
91f4a44ff4 sws/packed_16bpc_bswap: fix handling of negative stride 2013-05-12 17:26:39 +02:00
Michael Niedermayer
63725d158d Revert "sws: support endianness conversion for AV_PIX_FMT_RGBA64"
This reverts commit 23f7b14f2851d1d1a969c12b7e337826f0c74a43.

This commit broke fate
2013-05-12 15:05:25 +02:00
Paul B Mahol
23f7b14f28 sws: support endianness conversion for AV_PIX_FMT_RGBA64
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2013-05-12 11:44:20 +00:00
Michael Niedermayer
b2cf655d6e sws: add casts to silence pointer type mismatch warnings
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-10 20:36:12 +02:00
Michael Niedermayer
7ded017983 sws/output/yuv2gbrp_full_X_c(): silence warning about uninitialized variable
clang did not show this one, so i missed it in the last batch

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-10 20:36:12 +02:00
Michael Niedermayer
8ef453ff83 sws/output: silence warnings about uninitialized use of variables
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-10 01:33:57 +02:00
Clément Bœsch
570d63eef3 lavu: add FF_CEIL_RSHIFT and use it in various places. 2013-05-09 16:59:42 +02:00
Paul B Mahol
443181c0a9 libswscale: simplify isAnyRGB macro
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2013-05-08 12:48:05 +00:00
Paul B Mahol
5c057433cc libswscale: GBRAP input & output and GBRAP16 input support
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2013-05-08 12:48:05 +00:00
Clément Bœsch
c3dcd3a255 sws: factorize rgb 9+ planar funcs declarations. 2013-05-07 15:23:27 +02:00
Michael Niedermayer
dce12f6948 Merge commit '9cfa21c8c2b81146aab574f55c93f27e5c2d435c'
* commit '9cfa21c8c2b81146aab574f55c93f27e5c2d435c':
  swscale: support endianness conversion for AV_PIX_FMT_XYZ12

Conflicts:
	libswscale/swscale_unscaled.c
	libswscale/utils.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-07 11:13:46 +02:00
Michael Niedermayer
2060d94409 Merge commit 'c1eb3e7fecdc270e03a700d61ef941600a6af491'
* commit 'c1eb3e7fecdc270e03a700d61ef941600a6af491':
  swscale: add support for endianness only conversion

Conflicts:
	libswscale/utils.c
	libswscale/version.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-07 11:04:07 +02:00
Janne Grunau
9cfa21c8c2 swscale: support endianness conversion for AV_PIX_FMT_XYZ12 2013-05-06 21:48:14 +02:00
Michael Niedermayer
b3e9fd1fe5 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  swscale: Use alpha from the right row in yuva2rgba_c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-06 21:48:03 +02:00
Michael Niedermayer
050b48f397 Merge commit '6c5eac9a603a91a707d030d96ba97e2ef6b3ce50'
* commit '6c5eac9a603a91a707d030d96ba97e2ef6b3ce50':
  swscale: Remove unnecessary backslashes
  jpeg2000: Fix uninitialized data errors pointed out by valgrind/memcheck

Conflicts:
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-06 21:23:29 +02:00
Janne Grunau
c1eb3e7fec swscale: add support for endianness only conversion
Use bitfields in FormatEntry array to avoid wasting an int for each flag.
2013-05-06 18:55:58 +02:00