Commit Graph

1750 Commits

Author SHA1 Message Date
Michael Niedermayer
9734a7a1de avcodec/h264: Check mode before considering mixed mode intra prediction
Fixes out of array read
Fixes: asan_heap-oob_e476fc_2_asan_heap-oob_1333ec6_61_CAMACI3_Sony_C.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 14:53:13 +02:00
Michael Niedermayer
2cd7c99498 h264: reset ret to avoid propagating minor failures
Unbreak 772d150a6e.

CC: libav-stable@libav.org
Bug-Id: 750 / 905753
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
2014-09-29 12:33:55 +01:00
Michael Niedermayer
3e56ae67d3 avcodec/h264: Use FF_ALLOCZ_ARRAY_OR_GOTO()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-10 22:37:35 +02:00
Michael Niedermayer
033a5334ba avcodec/h264: Allow partial escaping
Fixes Ticket3923

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 01:58:18 +02:00
Michael Niedermayer
e0237208b4 avcodec/h264: Do not get stuck on IDR inter frames
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-31 04:09:06 +02:00
Michael Niedermayer
ef768ab976 avcodec/h264: Move h264_vdpau_class under ifdef to avoid unused variable warning
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-28 16:03:28 +02:00
Michael Niedermayer
949057c958 avcodec/h264: do proper cleanup in ff_h264_alloc_tables() in case DPB alloc fails
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-23 03:47:14 +02:00
Christophe Gisquet
585047bb7d h264: do not return on sidedata allocation failure
Not having allocated it is not a good reason to leave the object
in an undetermined state. Though a particular setting like the
AV_EF_* flags could be useful to control that behaviour.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-23 01:01:57 +02:00
Michael Niedermayer
fb33bff990 Merge commit 'f929ab0569ff31ed5a59b0b0adb7ce09df3fca39'
* commit 'f929ab0569ff31ed5a59b0b0adb7ce09df3fca39':
  cosmetics: Write NULL pointer equality checks more compactly

Conflicts:
	cmdutils.c
	ffmpeg_opt.c
	ffplay.c
	libavcodec/dvbsub.c
	libavcodec/dvdsubdec.c
	libavcodec/dvdsubenc.c
	libavcodec/dxa.c
	libavcodec/libxvid_rc.c
	libavcodec/mpegvideo.c
	libavcodec/mpegvideo_enc.c
	libavcodec/rv10.c
	libavcodec/tiffenc.c
	libavcodec/utils.c
	libavcodec/vc1dec.c
	libavcodec/zmbv.c
	libavdevice/v4l2.c
	libavformat/matroskadec.c
	libavformat/movenc.c
	libavformat/sdp.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-15 21:00:50 +02:00
Gabriel Dume
f929ab0569 cosmetics: Write NULL pointer equality checks more compactly
Signed-off-by: Diego Biurrun <diego@biurrun.de>
2014-08-15 03:18:18 -07:00
Michael Niedermayer
2bf87dcc2d Merge commit 'a7e541c9926d531a100ba0d36f4e56956dd84651'
* commit 'a7e541c9926d531a100ba0d36f4e56956dd84651':
  h264: fix interpretation of interleved stereo modes

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-07 20:22:16 +02:00
Felix Abecassis
a7e541c992 h264: fix interpretation of interleved stereo modes
Column and row frame packing arrangements were inverted.

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
2014-08-07 11:31:47 +01:00
Benoit Fouet
66af2a01d2 h264: remove useless assignment.
source index, as well as dest one, is unconditionnaly set afterwards,
before being effectively used.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-01 12:20:01 +02:00
Michael Niedermayer
601c238854 avcodec/h264: support AV_PKT_DATA_NEW_EXTRADATA
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-23 17:40:21 +02:00
Michael Niedermayer
1bf371b24e avcodec/h264: factor is_extra() out
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-23 17:40:21 +02:00
Michael Niedermayer
3a2d1465c8 Merge commit '2d60444331fca1910510038dd3817bea885c2367'
* commit '2d60444331fca1910510038dd3817bea885c2367':
  dsputil: Split motion estimation compare bits off into their own context

Conflicts:
	configure
	libavcodec/Makefile
	libavcodec/arm/Makefile
	libavcodec/dvenc.c
	libavcodec/error_resilience.c
	libavcodec/h264.h
	libavcodec/h264_slice.c
	libavcodec/me_cmp.c
	libavcodec/me_cmp.h
	libavcodec/motion_est.c
	libavcodec/motion_est_template.c
	libavcodec/mpeg4videoenc.c
	libavcodec/mpegvideo.c
	libavcodec/mpegvideo_enc.c
	libavcodec/x86/Makefile
	libavcodec/x86/me_cmp_init.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-17 23:27:40 +02:00
Diego Biurrun
2d60444331 dsputil: Split motion estimation compare bits off into their own context 2014-07-17 09:07:10 -07:00
Michael Niedermayer
5fce29ef29 Merge commit '2db953f84671997e936f91140ffb5143c1537844'
* commit '2db953f84671997e936f91140ffb5143c1537844':
  h264: K&R formatting cosmetics

Conflicts:
	libavcodec/h264.c
	libavcodec/h264_direct.c
	libavcodec/h264_slice.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-17 14:00:01 +02:00
Luca Barbato
2db953f846 h264: K&R formatting cosmetics
Signed-off-by: Diego Biurrun <diego@biurrun.de>
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
2014-07-17 03:05:24 -07:00
Michael Niedermayer
cd0dc88751 Merge commit '18e3d61e9e3b52c177aa7a1f2a054a8a753e1b09'
* commit '18e3d61e9e3b52c177aa7a1f2a054a8a753e1b09':
  h264: parse display orientation SEI message

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-10 00:07:04 +02:00
Vittorio Giovara
18e3d61e9e h264: parse display orientation SEI message
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
2014-07-09 10:17:42 -04:00
Michael Niedermayer
47048aa30b avcodec/h264: do not leave ret random on minor failures, causing major failure
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-24 01:54:39 +02:00
Michael Niedermayer
82e4340f1e Merge commit '772d150a6e82542c06b0c251e73dd299d98d1027'
* commit '772d150a6e82542c06b0c251e73dd299d98d1027':
  h264: error out from decode_nal_units() when AV_EF_EXPLODE is set

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-24 01:46:30 +02:00
Vittorio Giovara
772d150a6e h264: error out from decode_nal_units() when AV_EF_EXPLODE is set
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
2014-06-23 18:58:57 -04:00
Michael Niedermayer
5bf5e6b1c0 avcodec/h264: Use named identifier for single thread fall-back return from ff_h264_decode_slice_header()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-19 19:49:26 +02:00
Michael Niedermayer
7eae8cd870 avcodec/h264: fix frame skip code
Fixes Ticket3475

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-16 13:28:52 +02:00
Michael Niedermayer
9140d37af2 avcodec/h264: use the correct level for droping non intra frames
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-16 13:18:43 +02:00
Michael Niedermayer
19c9d1e8e7 avcodec/h264: in the absence of recovery points, be more tolerant on accepting plain I frames
Fixes: Ticket3652
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-04 04:25:14 +02:00
Michael Niedermayer
7ef01a7878 avcodec/h264: mark recovery_cnt==0 frames as keyframes
Fixes Ticket3063

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-05-03 16:01:45 +02:00
Michael Niedermayer
a75ba1e116 avcodec/h264/find_start_code: factorize addition out
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-07 04:18:21 +02:00
Michael Niedermayer
0ecb3075c1 Merge commit 'e10fd08aa7fbe8645545ad2e8721f0ed03c8e06a'
* commit 'e10fd08aa7fbe8645545ad2e8721f0ed03c8e06a':
  h264: Refactor decode_nal_units

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-07 03:59:22 +02:00
Luca Barbato
e10fd08aa7 h264: Refactor decode_nal_units 2014-04-06 23:33:17 +02:00
Michael Niedermayer
72bff8da47 avcodec: Make ff_print_debug_info2() independant of Picture struct
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-27 22:43:05 +01:00
Michael Niedermayer
ffd77f94a2 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  h264: Split h264 slice decoding from nal decoding

Conflicts:
	libavcodec/Makefile
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-24 01:33:02 +01:00
Michael Niedermayer
0626211b56 Merge commit 'e9a77f4bc084e5174baba3a8bc869ce4d102c4f0'
* commit 'e9a77f4bc084e5174baba3a8bc869ce4d102c4f0':
  h264: move macroblock decoding into its own file

Conflicts:
	libavcodec/Makefile
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-24 00:46:20 +01:00
Luca Barbato
f51d0f39c0 h264: Split h264 slice decoding from nal decoding 2014-03-23 22:41:12 +01:00
Vittorio Giovara
e9a77f4bc0 h264: move macroblock decoding into its own file 2014-03-23 22:41:12 +01:00
Yogender Kumar Gupta
6f7ca1f55b avcodec/h264: fix Lossless Decoding (Profile 244) for 8x8 Intra Prediction
This is limited to the case where x264_build = -1, to not break x264 decoding

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-18 20:37:26 +01:00
Michael Niedermayer
ad9a6e19e1 Merge commit 'd24e9a99a40166bf881ccd2e3ae5688af4726658'
* commit 'd24e9a99a40166bf881ccd2e3ae5688af4726658':
  h264 does not depend on mpegvideo any more

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-17 13:46:23 +01:00
Michael Niedermayer
9517900bef Merge commit 'e0c16e4e3259cf50b5bac4c23bb6e517f397c74b'
* commit 'e0c16e4e3259cf50b5bac4c23bb6e517f397c74b':
  mpegvideo: move mpegvideo formats-related defines to mpegutils.h

Conflicts:
	libavcodec/h264_cabac.c
	libavcodec/h264_cavlc.c
	libavcodec/h264_mvpred.h
	libavcodec/svq1enc.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-17 12:36:48 +01:00
Michael Niedermayer
3e5833802e Merge commit '136034d86b5cb1819a2c3e6ecdfeb05dcba7140d'
* commit '136034d86b5cb1819a2c3e6ecdfeb05dcba7140d':
  h264: Remove MotionEstContext and move the relevant fields to H264Context

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-17 12:26:09 +01:00
Michael Niedermayer
bb3c0571d3 Merge commit 'e3c2d0f3d41f79f7be7ba944aaca2e287c7d5c7c'
* commit 'e3c2d0f3d41f79f7be7ba944aaca2e287c7d5c7c':
  h264: Replace mpegvideo-specific MAX_THREADS by private define

Conflicts:
	libavcodec/h264.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-17 06:17:29 +01:00
Michael Niedermayer
a81a2b514e Merge commit '5d1c2e53ab3ce27b48c138d22bb01ff8e8304f27'
* commit '5d1c2e53ab3ce27b48c138d22bb01ff8e8304f27':
  h264: Replace mpegvideo-specific MAX_PICTURE_COUNT by private define

Conflicts:
	libavcodec/h264.c
	libavcodec/h264_ps.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-17 06:05:19 +01:00
Michael Niedermayer
6102dda1d7 avcodec/h264: h264_set_erpic() clear destination
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-17 05:52:05 +01:00
Michael Niedermayer
c237e88d51 Merge commit '9b749c8274f6b6f35dde2cf29b99fa4f719abf87'
* commit '9b749c8274f6b6f35dde2cf29b99fa4f719abf87':
  h264: move relevant fields from Picture to H264Picture

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-17 05:51:27 +01:00
Michael Niedermayer
533bc4c0a3 Merge commit 'd66e305bd1b4f3e91ae4e7e549148509d0811672'
* commit 'd66e305bd1b4f3e91ae4e7e549148509d0811672':
  er: move relevant fields from Picture to ERPicture

Conflicts:
	libavcodec/error_resilience.c
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-17 05:08:46 +01:00
Michael Niedermayer
3dab9e804a Merge commit '1c79b1625d4d257bfd01eccb84cc0ab355fb9a9e'
* commit '1c79b1625d4d257bfd01eccb84cc0ab355fb9a9e':
  h264: directly use frames in ff_h264_draw_horiz_band()

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-17 04:41:36 +01:00
Michael Niedermayer
8e92ff2546 avcodec/h264: be more tolerant on what pixel format changes trigger reinits
Fixes Ticket3260

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-17 02:37:29 +01:00
Vittorio Giovara
d24e9a99a4 h264 does not depend on mpegvideo any more 2014-03-16 23:05:47 +01:00
Vittorio Giovara
e0c16e4e32 mpegvideo: move mpegvideo formats-related defines to mpegutils.h 2014-03-16 23:04:41 +01:00
Vittorio Giovara
136034d86b h264: Remove MotionEstContext and move the relevant fields to H264Context
Unused buffers scratchpad and temp have been dropped too.
2014-03-16 23:01:01 +01:00
Vittorio Giovara
e3c2d0f3d4 h264: Replace mpegvideo-specific MAX_THREADS by private define 2014-03-16 23:01:01 +01:00
Vittorio Giovara
5d1c2e53ab h264: Replace mpegvideo-specific MAX_PICTURE_COUNT by private define 2014-03-16 23:01:01 +01:00
Vittorio Giovara
9b749c8274 h264: move relevant fields from Picture to H264Picture 2014-03-16 23:01:00 +01:00
Vittorio Giovara
d66e305bd1 er: move relevant fields from Picture to ERPicture
This is done to disentangle ER from mpegvideo. In order to use a
classic Picture, callers can use ff_mpeg_set_erpic() or use a custom function
to set the fields. Please note that buffers need to be allocated before
calling ff_er_frame_end().
2014-03-16 23:01:00 +01:00
Vittorio Giovara
1c79b1625d h264: directly use frames in ff_h264_draw_horiz_band() 2014-03-16 23:01:00 +01:00
Michael Niedermayer
5c634cbeb7 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  Give IDCT matrix transpose macro a more descriptive name

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-28 02:08:11 +01:00
Diego Biurrun
f2408ec9d7 Give IDCT matrix transpose macro a more descriptive name
This also avoids a macro name clash and related warning on ARM.
2014-02-27 13:38:00 -08:00
Michael Niedermayer
4f4cc43fd8 avcodec/h264: allow mixing idr and non idr slices with frame threading again
This combination exists in the wild
Fixes Ticket3131

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-27 15:49:25 +01:00
Michael Niedermayer
64bb64f704 avcodec/h264: fix droped frame handling also for threads > 1
Seems i mistakely tested just with threads=1
Fixes part of Ticket3386

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-27 08:07:46 +01:00
Michael Niedermayer
b5005def8a avcodec/h264: avoid using lost frames as references
Fixes Ticket3386

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-27 03:26:03 +01:00
Michael Niedermayer
72e6913140 avcodec/h264: clear chroma planes when flags gray is used
Fixes Ticket3397
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-23 18:22:13 +01:00
Luca Barbato
d922c5a5fb h264: Fix a typo from the previous commit
f777504f64 changed a - in +

CC: libav-stable@libav.org
2014-02-22 12:26:32 +01:00
Michael Niedermayer
8c55ff3933 avcodec/h264: use subsample factors of the used pixel format
Fixes out of array read
Fixes: 1cb91c36c4e55463f14aacb9bdf55b38-asan_heap-oob_106cbce_5617_cov_11212800_h264_mmx_chroma_intra_lf.mp4
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-21 23:53:49 +01:00
Michael Niedermayer
76dd01ecd4 avcodec/h264: fix sign error
regression since f777504f64

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-21 00:33:57 +01:00
Michael Niedermayer
de7b50e9cd Merge remote-tracking branch 'qatar/master'
* qatar/master:
  h264: Lower bound check for slice offsets

Conflicts:
	libavcodec/h264.c

See: 91253839e1
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-21 00:20:55 +01:00
Vittorio Giovara
f777504f64 h264: Lower bound check for slice offsets
And use the value from the specification.

Sample-Id: 00000451-google
Found-by: Mateusz j00ru Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-02-20 18:58:38 +01:00
Michael Niedermayer
d0e236292d Merge remote-tracking branch 'qatar/master'
* qatar/master:
  h264: informative error reporting in decode_slice_header()

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-19 02:25:02 +01:00
Luca Barbato
fea6db064b h264: informative error reporting in decode_slice_header()
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
2014-02-18 23:47:55 +01:00
Luca Barbato
96f9fbe109 h264: fix slice_type value reported in decode_slice_header()
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
2014-02-18 23:47:32 +01:00
Michael Niedermayer
91253839e1 avcodec/h264: more completely check the loop filter parameters
Fixes out of array read
Fixes: caa65cc01655505705129b677189f036-signal_sigsegv_fdcc43_2681_cov_3043376737_PPH422I5_Panasonic_A.264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-17 01:29:34 +01:00
Michael Niedermayer
a7eb93b367 Merge commit '15210354cf27cf4e24d91f84d66cf471511ce718'
* commit '15210354cf27cf4e24d91f84d66cf471511ce718':
  h264: drop outdated comments

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-14 14:07:19 +01:00
Michael Niedermayer
99b12357f4 Merge commit '3a0576702825423abecb32627c530dbc4c0f73bc'
* commit '3a0576702825423abecb32627c530dbc4c0f73bc':
  h264: store current_sps_id inside the current sps

Conflicts:
	libavcodec/h264.c
	libavcodec/h264_ps.c

The current_sps_id is not removed as it used in security related code.

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-14 13:58:02 +01:00
Michael Niedermayer
60b46a00c6 Merge commit '73e8fab31dc19c4371499e612856accbc00b2820'
* commit '73e8fab31dc19c4371499e612856accbc00b2820':
  h264: print values in case of error

Conflicts:
	libavcodec/h264.c
	libavcodec/h264_ps.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-14 13:35:45 +01:00
Vittorio Giovara
15210354cf h264: drop outdated comments 2014-02-14 05:08:37 +01:00
Vittorio Giovara
3a05767028 h264: store current_sps_id inside the current sps
In preparation for MVC support.
2014-02-14 05:05:46 +01:00
Vittorio Giovara
73e8fab31d h264: print values in case of error
Also make error style consistent and drop redundant information.
2014-02-14 05:05:35 +01:00
Michael Niedermayer
84873794ad Merge commit 'f795a8a8bf5e312dad2c2829c543b9d309376ca1'
* commit 'f795a8a8bf5e312dad2c2829c543b9d309376ca1':
  h264: make context_count unsigned

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-14 01:24:26 +01:00
Janne Grunau
f795a8a8bf h264: make context_count unsigned
Removes the bogus but scary looking warning 'libavcodec/h264.c:4529:49:
warning: array subscript is below array bounds [-Warray-bounds]'.
2014-02-12 15:48:03 +01:00
Michael Niedermayer
c93e691369 Merge commit '1f097d168d9cad473dd44010a337c1413a9cd198'
* commit '1f097d168d9cad473dd44010a337c1413a9cd198':
  h264: reset data partitioning at the beginning of each decode call

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-04 16:44:32 +01:00
Anton Khirnov
1f097d168d h264: reset data partitioning at the beginning of each decode call
Prevents using GetBitContexts with data from previous calls.

Fixes access to freed memory.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-02-04 11:26:17 +01:00
Michael Niedermayer
8a3b85f3a7 avcodec/h264: update current_sps & sps->new only after the whole slice header decoder and init code finished
This avoids them being cleared before the full initialization finished

Fixes out of array read
Fixes: asan_heap-oob_f0c5e6_7071_cov_1605985132_mov_h264_aac__Demo_FlagOfOurFathers.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-04 04:49:25 +01:00
Michael Niedermayer
e708424b70 avcodec/h264: Disallow pps_id changing between slices
Such changes are forbidden in H.264 and lead to race conditions

Fixes out of array read
Fixes: signal_sigsegv_f9796a_1613_cov_3114610371_FM1_BT_B.h264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-04 00:07:27 +01:00
Michael Niedermayer
1a96b27ebf avcodec/h264: clear dequant8_coeff pointers if 8x8 mode is not enabled
This prevents stale pointers from being left

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-03 22:15:32 +01:00
Michael Niedermayer
965fa6b0d9 Merge commit 'fb0c9d41d685abb58575c5482ca33b8cd457c5ec'
* commit 'fb0c9d41d685abb58575c5482ca33b8cd457c5ec':
  avutil: remove timer.h include from internal.h

Conflicts:
	libavcodec/ffv1dec.c
	libavutil/internal.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-26 01:54:55 +01:00
Janne Grunau
fb0c9d41d6 avutil: remove timer.h include from internal.h
Added libavutil/timer.h include to all files with {START,STOP}_TIMER.
2014-01-25 21:50:20 +01:00
Janne Grunau
ea49f60523 h264: skip chroma edges at the picture boundary while deblocking 4:4:4
This handles macroblock edges for the chroma components in the same way
as for the luma compoment for 4:4:4 streams. The Spec explicitly states
that the deblocking filter is not applied to edges at the boundary of
the picture.

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
2014-01-22 13:44:28 +01:00
Michael Niedermayer
a52fbe5119 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  h264: check that an IDR NAL only contains I slices

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-18 23:13:33 +01:00
Michael Niedermayer
87e46dd5fb Merge commit '0652e024c680420d298cdf3719d0a0c030173fe3'
* commit '0652e024c680420d298cdf3719d0a0c030173fe3':
  h264: reset ref count if decoding the slice header fails

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-18 22:51:59 +01:00
Michael Niedermayer
329610303a Merge commit '00dbff4c3e048b4abd01bf805725aabff0fa5ee1'
* commit '00dbff4c3e048b4abd01bf805725aabff0fa5ee1':
  h264: do not call field_end if we do not have a current picture

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-18 21:51:40 +01:00
Michael Niedermayer
f2cb3b36ac Merge commit '7f0e81db3c4ee6f8ce15058bafa72ce928a89f3f'
* commit '7f0e81db3c4ee6f8ce15058bafa72ce928a89f3f':
  h264: limit allowed pred modes in ff_h264_check_intra_pred_mode() to 3

Conflicts:
	libavcodec/h264.c

See: d6a33f5d20
See: 2005fddcbb
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-18 21:44:18 +01:00
Michael Niedermayer
357a733f91 Merge commit 'd1b3fabe6945e511bb20fc9ca52b47eb952526ee'
* commit 'd1b3fabe6945e511bb20fc9ca52b47eb952526ee':
  h264: reset first_field if frame_start() fails for missing refs

See: d7599bd8e2
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-18 21:29:37 +01:00
Anton Khirnov
8b2e5e42bb h264: check that an IDR NAL only contains I slices
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-18 20:53:31 +01:00
Anton Khirnov
0652e024c6 h264: reset ref count if decoding the slice header fails
Otherwise the ER code might try to use some already freed references.

Fixes possible access to freed memory.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-18 20:51:04 +01:00
Anton Khirnov
00dbff4c3e h264: do not call field_end if we do not have a current picture
Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-18 20:42:21 +01:00
Anton Khirnov
7f0e81db3c h264: limit allowed pred modes in ff_h264_check_intra_pred_mode() to 3
Higher modes are not allowed for 16x16/chroma, which is what this
function is used for. Otherwise this function would return 0 (vertical
prediction) for invalid higher modes, which could result in invalid
reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-18 20:41:59 +01:00
Anton Khirnov
d1b3fabe69 h264: reset first_field if frame_start() fails for missing refs
In this case we may not have a current frame, while first_field being
set implies we do.

Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-18 20:41:24 +01:00
Michael Niedermayer
64591f8f86 Merge commit '9eef9eb3014b2ed9c3ff4aac510a9f04edb555cf'
* commit '9eef9eb3014b2ed9c3ff4aac510a9f04edb555cf':
  h264: check that execute_decode_slices() is not called too many times

Conflicts:
	libavcodec/h264.c

The check is replaced by an assert() as the mb index should not ever go out
of bounds.

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-06 16:39:38 +01:00
Michael Niedermayer
a60abb1ee0 Merge commit 'bfd26b7ce6efea594f2b99441d900419df3af638'
* commit 'bfd26b7ce6efea594f2b99441d900419df3af638':
  h264: reject mismatching luma/chroma bit depths during sps parsing

Conflicts:
	libavcodec/h264_ps.c

See: bdeb61ccc6
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-06 16:28:55 +01:00
Michael Niedermayer
98dcbb47fa avcodec/h264: reset list_count too in case of error in ff_set_ref_count()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-06 16:17:45 +01:00
Michael Niedermayer
7ee8a1c562 Merge commit '9a026c72982faf20e1c8dfbe48f0b312cdea69c8'
* commit '9a026c72982faf20e1c8dfbe48f0b312cdea69c8':
  h264: rebuild the default ref list if the reference count changes

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-06 16:17:38 +01:00
Anton Khirnov
9eef9eb301 h264: check that execute_decode_slices() is not called too many times
Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-06 08:25:25 +01:00
Anton Khirnov
bfd26b7ce6 h264: reject mismatching luma/chroma bit depths during sps parsing
There is no point in delaying the check and it avoids bugs with a
half-initialized context.

Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-06 08:23:45 +01:00
Anton Khirnov
9a026c7298 h264: rebuild the default ref list if the reference count changes
Fixes possible access to freed memory.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-06 08:23:17 +01:00
Michael Niedermayer
8e6af036b9 Merge commit '58312b2472d3a44d7458865c459d59ef2e02bf1a'
* commit '58312b2472d3a44d7458865c459d59ef2e02bf1a':
  h264: reset data_partitioning if decoding the slice header for NAL_DPA fails

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-04 02:06:21 +01:00
Anton Khirnov
58312b2472 h264: reset data_partitioning if decoding the slice header for NAL_DPA fails
If it was set before then we can end up trying to decode a slice without
a valid slice header, which can lead to invalid memory access.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-03 16:42:02 +01:00
Dale Curtis
4feca2214a h264: Clear ERContext.cur_pic when unref'ing current picture.
Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
2014-01-02 23:49:06 +01:00
Michael Niedermayer
74a9c92840 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  h264: namespace the decode function

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-29 11:37:42 +01:00
Michael Niedermayer
6ea05ef278 avcodec/h264: remove unused variable
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-28 18:19:46 +01:00
Luca Barbato
4d2bb28931 h264: namespace the decode function
Make much easier debugging.
2013-12-27 07:48:13 +01:00
Michael Niedermayer
e3578fd525 Merge commit '598ce4ab4f1893e0661fc038101487e511937877'
* commit '598ce4ab4f1893e0661fc038101487e511937877':
  h264: call av_frame_unref() instead of avcodec_get_frame_defaults().

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-12 00:25:31 +01:00
Anton Khirnov
598ce4ab4f h264: call av_frame_unref() instead of avcodec_get_frame_defaults().
This is a temporary workaround to allow deprecating
avcodec_get_frame_defaults(). The proper solution will be using a
properly allocated AVFrame in Picture.
2013-12-11 20:39:55 +01:00
Michael Niedermayer
d9339ab553 avcodec/h264: fix code that blindly dereferences NULL DPB
Fixes mixed flushing and decoding NULL packets
Found-by: wm4

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-10 23:50:08 +01:00
Michael Niedermayer
be6e81463c Merge commit '5b10ef729f610fcbc9c485e7b643ce53268144cb'
* commit '5b10ef729f610fcbc9c485e7b643ce53268144cb':
  h264: parse frame packing arrangement SEI messages and save relevant stereo3d information

Conflicts:
	libavcodec/h264.c
	libavcodec/h264_sei.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-09 21:53:23 +01:00
Vittorio Giovara
5b10ef729f h264: parse frame packing arrangement SEI messages and save relevant stereo3d information 2013-12-09 16:02:43 +01:00
Michael Niedermayer
fe540ae6b7 Merge commit 'f0259a587ee3419dd894873ea617b4c98eeaca1c'
* commit 'f0259a587ee3419dd894873ea617b4c98eeaca1c':
  h264: check buffer size before accessing it

Conflicts:
	libavcodec/h264.c

See: ea0ac11e52
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-22 11:56:46 +01:00
Anton Khirnov
f0259a587e h264: check buffer size before accessing it
Fixes invalid reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2013-11-21 20:54:20 +01:00
Michael Niedermayer
56e122787e Merge commit 'a553c6a347d3d28d7ee44c3df3d5c4ee780dba23'
* commit 'a553c6a347d3d28d7ee44c3df3d5c4ee780dba23':
  lavc: use buf[0] instead of data[0] in checks whether a frame is allocated

Conflicts:
	libavcodec/h264_refs.c
	libavcodec/mpegvideo.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-19 03:56:53 +01:00
Anton Khirnov
a553c6a347 lavc: use buf[0] instead of data[0] in checks whether a frame is allocated
data[0] may be NULL for valid frames with hwaccel pixel formats.
2013-11-18 18:09:48 +01:00
Michael Niedermayer
91e00c4a78 Merge commit '458446acfa1441d283dacf9e6e545beb083b8bb0'
* commit '458446acfa1441d283dacf9e6e545beb083b8bb0':
  lavc: Edge emulation with dst/src linesize

Conflicts:
	libavcodec/cavs.c
	libavcodec/h264.c
	libavcodec/hevc.c
	libavcodec/mpegvideo_enc.c
	libavcodec/mpegvideo_motion.c
	libavcodec/rv34.c
	libavcodec/svq3.c
	libavcodec/vc1dec.c
	libavcodec/videodsp.h
	libavcodec/videodsp_template.c
	libavcodec/vp3.c
	libavcodec/vp8.c
	libavcodec/wmv2.c
	libavcodec/x86/videodsp.asm
	libavcodec/x86/videodsp_init.c

Changes to the asm are not merged, they are left for volunteers or
in their absence for later.
The changes this merge introduces are reordering of the function
arguments

See: face578d56
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-15 15:07:10 +01:00
Ronald S. Bultje
458446acfa lavc: Edge emulation with dst/src linesize
Allow supporting files for which the image stride is smaller than
the maximum block size + number of subpel mc taps, e.g. a 64x64 VP9
file or a 16x16 VP8 file with -fflags +emu_edge.
2013-11-15 10:16:27 +01:00
Michael Niedermayer
9244a68092 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  h264: free the tables and uninitialize the context on flush

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-14 21:11:42 +01:00
Anton Khirnov
9eda9d3322 h264: free the tables and uninitialize the context on flush
Prevents referencing empty frames when the first packet after the flush
does not contain a frame.
2013-11-14 19:00:20 +01:00
Michael Niedermayer
9e5ef1c5c3 h264: Do not treat the initial frame special in handling of frame gaps
The not handling of frame gaps has lead to the lack of a dummy reference
frame, which has lead to the failure of decode_slice_header() which has
lead to one SEI recovery message being skiped which had introduced a
slightly suboptimal recovery point for at least 1 h264 file compared to
JM.

Found-by: Carl & BugMaster
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-14 12:16:01 +01:00
Michael Niedermayer
78e150c5e9 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  hwaccel: Simplify ff_find_hwaccel

Conflicts:
	libavcodec/mpeg12dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-10 17:37:51 +01:00
Luca Barbato
08303d7741 hwaccel: Simplify ff_find_hwaccel
It is always called by passing fields from an AVCodecContext.
2013-11-10 13:59:48 +01:00
Michael Niedermayer
4fb1221e66 h264: reduce whitespace differences to libav
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-03 01:16:31 +01:00
Michael Niedermayer
647adc4290 h264: factor "if(h->sei_recovery_frame_cnt >= 0)" out
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-02 01:02:21 +01:00
Michael Niedermayer
f62dfed3de h264: simplify frame_recovered code
This is a separate commit to ease future bisecting in case it breaks something.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-01 12:59:40 +01:00
Michael Niedermayer
78265fcfee Merge commit '28096e0a806e57376541e6222d315619906e3c55'
* commit '28096e0a806e57376541e6222d315619906e3c55':
  h264: wait for initial complete frame before outputing frames

Conflicts:
	doc/APIchanges
	libavcodec/h264.c
	libavcodec/mpegvideo.h
	libavutil/frame.h
	libavutil/version.h

See: a64b028aeb (as well as various later commits)
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-01 12:59:08 +01:00
John Stebbins
28096e0a80 h264: wait for initial complete frame before outputing frames
This can be optionally disabled whith the "output_corrupt" flags
option.  When in "output_corrupt" mode, incomplete frames are
signalled through AVFrame.flags FRAME_FLAG_INCOMPLETE_FRAME.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-10-31 20:14:14 +01:00
Derek Buitenhuis
58d13cea30 h264: Check all allocations
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2013-10-29 14:00:29 +00:00
Derek Buitenhuis
a483aae7d8 h264: Check all allocations
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2013-10-29 12:00:27 +00:00
Michael Niedermayer
3c9dd93faa h264: make flush_change() set mmco_reset
This ensures that frames do not get mixed on context reinits

Fixes Ticket2836

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-26 01:22:38 +02:00
Michael Niedermayer
29ffeef5e7 avcodec/h264: do not trust last_pic_droppable when marking pictures as done
This simplifies the code and fixes a deadlock

Fixes Ticket2927
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-24 23:15:30 +02:00
Michael Niedermayer
f7f74a37b8 Merge commit '4baba6c813b7a1f27370e20fb1a87b05fcb39208'
* commit '4baba6c813b7a1f27370e20fb1a87b05fcb39208':
  h264_parser: Fix POC parsing for the case where MMCO_RESET is present.

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-16 12:51:26 +02:00
Ronald S. Bultje
93f305473f lavc: Convert some remaining strides to ptrdiff_t 2013-10-15 23:58:20 +02:00
Yusuke Nakamura
4baba6c813 h264_parser: Fix POC parsing for the case where MMCO_RESET is present.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-10-15 20:02:55 +02:00
Michael Niedermayer
65c2fe7159 avcodec/h264: call flush_change() on init
This ensures that all the non zero fields get setup correctly.
Alternatively their initialization could be duplicated.

Fixes Ticket2738

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-10 02:25:51 +02:00
Clément Bœsch
b46f19100b cosmetics: group remaining .name and .long_name.
See b2bed9325.
2013-10-04 13:33:20 +00:00
Michael Niedermayer
ee77140afa Merge commit 'b2bed9325dbd6be0da1d91ffed3f513c40274fd2'
* commit 'b2bed9325dbd6be0da1d91ffed3f513c40274fd2':
  cosmetics: Group .name and .long_name together in codec/format declarations

Conflicts:
	libavcodec/8svx.c
	libavcodec/alac.c
	libavcodec/cljr.c
	libavcodec/dnxhddec.c
	libavcodec/dnxhdenc.c
	libavcodec/dpxenc.c
	libavcodec/dvdec.c
	libavcodec/dvdsubdec.c
	libavcodec/dvdsubenc.c
	libavcodec/ffv1dec.c
	libavcodec/flacdec.c
	libavcodec/flvdec.c
	libavcodec/fraps.c
	libavcodec/frwu.c
	libavcodec/g726.c
	libavcodec/gif.c
	libavcodec/gifdec.c
	libavcodec/h261dec.c
	libavcodec/h263dec.c
	libavcodec/iff.c
	libavcodec/imc.c
	libavcodec/libopencore-amr.c
	libavcodec/libopenjpegdec.c
	libavcodec/libopenjpegenc.c
	libavcodec/libspeexenc.c
	libavcodec/libvo-amrwbenc.c
	libavcodec/libvorbisenc.c
	libavcodec/libvpxenc.c
	libavcodec/libx264.c
	libavcodec/libxavs.c
	libavcodec/libxvid.c
	libavcodec/ljpegenc.c
	libavcodec/mjpegbdec.c
	libavcodec/mjpegdec.c
	libavcodec/mpeg12dec.c
	libavcodec/mpeg4videodec.c
	libavcodec/msmpeg4dec.c
	libavcodec/pgssubdec.c
	libavcodec/pngdec.c
	libavcodec/pngenc.c
	libavcodec/proresdec_lgpl.c
	libavcodec/proresenc_kostya.c
	libavcodec/ra144enc.c
	libavcodec/rawdec.c
	libavcodec/rv10.c
	libavcodec/sp5xdec.c
	libavcodec/takdec.c
	libavcodec/tta.c
	libavcodec/v210dec.c
	libavcodec/vp6.c
	libavcodec/wavpack.c
	libavcodec/xbmenc.c
	libavcodec/yop.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-04 12:34:23 +02:00
Diego Biurrun
b2bed9325d cosmetics: Group .name and .long_name together in codec/format declarations 2013-10-03 23:32:01 +02:00
Carl Eugen Hoyos
b5f219c904 Print pix_fmt name on h264 context reinitialisation. 2013-10-03 01:42:23 +02:00
Ronald S. Bultje
face578d56 Rewrite emu_edge functions to have separate src/dst_stride arguments.
This allows supporting files for which the image stride is smaller than
the max. block size + number of subpel mc taps, e.g. a 64x64 VP9 file
or a 16x16 VP8 file with -fflags +emu_edge.
2013-09-28 20:28:08 -04:00
Ronald S. Bultje
c341f734e5 Convert multiplier for MV from int to ptrdiff_t.
This prevents emulated_edge_mc from not undoing mvy*stride-related
integer overflows.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-28 11:28:09 +02:00
Michael Niedermayer
4a89868c5f Merge commit 'cab8c5f8e140c96ba3725ab709d823abfd1e31a5'
* commit 'cab8c5f8e140c96ba3725ab709d823abfd1e31a5':
  h264: do not reinitialize the global cabac tables at each slice header

See: 1e2e2c8095
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-25 10:25:17 +02:00
Anton Khirnov
cab8c5f8e1 h264: do not reinitialize the global cabac tables at each slice header 2013-09-24 17:13:52 +02:00
Michael Niedermayer
d195e6e410 Merge commit '71cabb521ac397db3903011d2de7afd3e0fc7ab6'
* commit '71cabb521ac397db3903011d2de7afd3e0fc7ab6':
  h264: do not discard NAL_SEI when skipping frames

Conflicts:
	libavcodec/h264.c

See: 7d75fb381b
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-24 15:38:34 +02:00
Michael Niedermayer
af2bb83dff Merge commit 'edaba18021e4793116f0f266e65077578a5e742e'
* commit 'edaba18021e4793116f0f266e65077578a5e742e':
  h264: log extradata skip only for non-ignored NALs

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-24 15:30:21 +02:00
Rainer Hochecker
71cabb521a h264: do not discard NAL_SEI when skipping frames
Setting AVDISCARD_NONREF (e.g. after flushing) resulted in 100% dropped frames.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-09-24 13:24:29 +02:00
Vittorio Giovara
edaba18021 h264: log extradata skip only for non-ignored NALs
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-09-24 13:24:28 +02:00
Vittorio Giovara
1cad7171dd h264: remove an unused static constant
Signed-off-by: Martin Storsjö <martin@martin.st>
2013-09-20 14:40:10 +03:00
Rainer Hochecker
7d75fb381b h264: do not discard NAL_SEI when skipping frames
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-09 11:42:26 +02:00
Michael Niedermayer
93cf7b0195 avcodec/h264: set er.ref_count earlier
Fixes Ticket2910

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-01 04:02:03 +02:00
Michael Niedermayer
626739ebbb avcodec/h264: Free rbsp_buffer before copying context over it
Fixes memleak
Fixes Ticket1900

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-25 03:30:31 +02:00
Michael Niedermayer
be30e44dd9 avcodec/h264: Zero rbsp_buffer earler to ensure no duplicated pointers can leak
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-25 03:29:52 +02:00
Michael Niedermayer
ecbf838c7d h264: prevent rbsp_buffer values from becoming duplicated
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-25 03:01:19 +02:00
Michael Niedermayer
16466d92b9 Merge commit 'c4e43560fe6677e9d60bfb3cffc41c7324e92a0b'
* commit 'c4e43560fe6677e9d60bfb3cffc41c7324e92a0b':
  h264data: Move some tables to the only place they are used

Conflicts:
	libavcodec/h264data.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-21 12:21:48 +02:00
Michael Niedermayer
58e12732db Merge commit '2a61592573d725956a4377641344afe263382648'
* commit '2a61592573d725956a4377641344afe263382648':
  avcodec: Remove some commented-out debug cruft

Conflicts:
	libavcodec/h264_cabac.c
	libavcodec/h264_cavlc.c
	libavcodec/h264_loopfilter.c
	libavcodec/h264_mvpred.h
	libavcodec/mjpegdec.c
	libavcodec/mjpegenc.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-21 11:18:43 +02:00
Diego Biurrun
c4e43560fe h264data: Move some tables to the only place they are used 2013-08-20 20:49:37 +02:00
Diego Biurrun
2a61592573 avcodec: Remove some commented-out debug cruft 2013-08-20 19:59:50 +02:00
Michael Niedermayer
a0c6c8e53e Revert "Merge commit of 'vdpau: remove old-style decoders'"
This reverts commit bf36dc50ea, reversing
changes made to b7fc2693c7.

Conflicts:

	libavcodec/h264.c

Keeping support for the old VDPAU API has been requested by our VDPAU maintainer

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-17 22:17:49 +02:00
Michael Niedermayer
921c1d4c95 Merge commit 'c1076d8479a6c0ee2e0c4b0e2151df5b0228438e'
* commit 'c1076d8479a6c0ee2e0c4b0e2151df5b0228438e':
  h264: check one context_init() allocation

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-11 10:35:44 +02:00
Michael Niedermayer
d2d8e259fd Merge commit '5eb488bfa835f2902a31ba99d57c16ae36c4f598'
* commit '5eb488bfa835f2902a31ba99d57c16ae36c4f598':
  h264: use explicit variable names for *_field_flag

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-11 10:28:49 +02:00
Michael Niedermayer
e0b45ca730 Merge commit 'b3dc260e7fa6f3f852dd5cb7d86763c4b5736714'
* commit 'b3dc260e7fa6f3f852dd5cb7d86763c4b5736714':
  h264: return meaningful values

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-11 10:08:48 +02:00
Vittorio Giovara
c1076d8479 h264: check one context_init() allocation
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-08-10 13:36:38 +02:00
Vittorio Giovara
5eb488bfa8 h264: use explicit variable names for *_field_flag
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-08-10 13:36:25 +02:00
Vittorio Giovara
b3dc260e7f h264: return meaningful values
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-08-10 13:28:51 +02:00
Michael Niedermayer
bf36dc50ea Merge commit '578ea75a9e4ac56e0bbbbe668700be756aa699f8'
* commit '578ea75a9e4ac56e0bbbbe668700be756aa699f8':
  vdpau: remove old-style decoders

Conflicts:
	libavcodec/allcodecs.c
	libavcodec/h263dec.c
	libavcodec/h264.c
	libavcodec/mpeg12dec.c
	libavcodec/mpeg4videodec.c
	libavcodec/vc1dec.c
	libavcodec/vdpau.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-06 13:24:22 +02:00
Rémi Denis-Courmont
578ea75a9e vdpau: remove old-style decoders
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-08-05 11:15:49 +02:00
Michael Niedermayer
82fdfe8e51 Merge commit 'a8b19271c3b40ac3c3dc769fe248887acf14ba5a'
* commit 'a8b19271c3b40ac3c3dc769fe248887acf14ba5a':
  avcodec: Add output_picture_number to AVCodecParserContext

Conflicts:
	doc/APIchanges
	libavcodec/version.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-03 10:48:11 +02:00
Yusuke Nakamura
a8b19271c3 avcodec: Add output_picture_number to AVCodecParserContext
Set output_picture_number in H.264 parser.

Signed-off-by: Diego Biurrun <diego@biurrun.de>
2013-08-02 20:59:45 +02:00
Michael Niedermayer
102397d2c1 Merge commit '16c22122c788b5e54a2f2e224bd0106429f0714c'
* commit '16c22122c788b5e54a2f2e224bd0106429f0714c':
  h264: K&R formatting cosmetics

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-28 11:40:49 +02:00
Diego Biurrun
16c22122c7 h264: K&R formatting cosmetics 2013-07-27 10:47:38 +02:00
Joakim Plate
8710a634a5 h264: add frame packing as stereo_mode frame metadata
This matches the matroska defintion of stereo_mode, with
no metadata written if no info exist in sei

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-21 16:04:02 +02:00
Michael Niedermayer
a35494c566 h264: fix indention of xchg_mb_border()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-02 03:27:14 +02:00
Michael Niedermayer
f27b22b497 h264: move 444 border xchg under if (deblock_top)
Fixes out of array access
Fixes Ticket2668

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-02 03:27:04 +02:00
Michael Niedermayer
bbe26eff22 h264: Fix null pointer dereference with disabled error concealment
Fixes Ticket2551

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-24 16:05:19 +02:00
Michael Niedermayer
d6a33f5d20 h264: fix size of arrays in ff_h264_check_intra_pred_mode()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-21 20:48:07 +02:00
Michael Niedermayer
2005fddcbb h264/ff_h264_check_intra_pred_mode: fix input value check
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-21 20:48:07 +02:00
Michael Niedermayer
443b29e475 h264: Try parsing SPS as complete NAL in more cases
Fixes Ticket2580

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-13 01:43:09 +02:00
Michael Niedermayer
029353e427 Merge commit 'e9e5a1bdc769a7225ab0d4f8b33bcacc6496bd68'
* commit 'e9e5a1bdc769a7225ab0d4f8b33bcacc6496bd68':
  Monkey's Audio old versions FATE tests
  h264_parser: Set field_order and picture_structure.

Conflicts:
	libavcodec/h264_parser.c
	tests/fate/lossless-audio.mak

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-24 12:15:56 +02:00
Yusuke Nakamura
3f1a7ceb2c h264_parser: Set field_order and picture_structure.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-05-24 08:45:05 +02:00
Michael Niedermayer
5a9e376049 h264: check mb_width/height
Fixes inconsistency that leads to out of array accesses with threads

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-06 01:58:10 +02:00
Michael Niedermayer
edabbfbaf2 ff_h264_decode_seq_parameter_set: dont set h->sps
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-06 00:37:47 +02:00
Michael Niedermayer
fbaf75b166 h264: Use the correct croping values.
This fixes returning pictures with corrupted data pointers.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-01 00:06:26 +02:00
Michael Niedermayer
5299ee54a7 h264: assert that croping values at slice level are consistent
These values where checked before already.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-04-30 23:49:56 +02:00
Diego Biurrun
fce99322b0 h264: Drop unused variable 2013-04-22 12:05:09 +02:00
Michael Niedermayer
8ebfd7c49e h264: remove unused variable
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-04-20 02:22:32 +02:00
Michael Niedermayer
f4b05cd841 Merge commit '5e83d9aced2fc2b2e1360452794c58aba55d497c'
* commit '5e83d9aced2fc2b2e1360452794c58aba55d497c':
  h264: fully support cropping.

Conflicts:
	doc/APIchanges
	libavcodec/h264.c
	libavcodec/h264_ps.c
	libavcodec/options_table.h
	libavcodec/version.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-04-19 22:46:51 +02:00
Ronald S. Bultje
c443117f25 dsputil: Remove dct_bits
dct_bits is never set except in h264, where it is never used,
thus remove it.

Signed-off-by: Martin Storsjö <martin@martin.st>
2013-04-19 23:29:22 +03:00
Anton Khirnov
5e83d9aced h264: fully support cropping.
Based on a patch by Vittorio Giovara <vittorio.giovara@gmail.com>

Fixes Bug 378.
2013-04-19 09:28:08 +02:00
Reimar Döffinger
c10d498bfd Add thread-safe wrapper for get_format().
Just like get_buffer, get_format should not be called from a different
thread if thread_safe_callbacks is not set.

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
2013-04-15 09:04:07 +02:00
Michael Niedermayer
65120f628c Merge commit '85deb51a01f1ecc5ac5faa52ad8ea141c384e23a'
* commit '85deb51a01f1ecc5ac5faa52ad8ea141c384e23a':
  h264: Only initialize dsputil if error resilience is enabled

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-04-11 12:00:05 +02:00
Michael Niedermayer
944ad46182 Merge commit 'e8cafd2773bc56455c8816593cbd9368f2d69a80'
* commit 'e8cafd2773bc56455c8816593cbd9368f2d69a80':
  h264: Clear the mb members via memset instead of using dsputil

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-04-11 11:40:05 +02:00
Ronald S. Bultje
85deb51a01 h264: Only initialize dsputil if error resilience is enabled
It is only used for error resilience. This allows building the
h264 decoder without dsputil, if error resilience is disabled.

Signed-off-by: Martin Storsjö <martin@martin.st>
2013-04-10 11:03:53 +03:00
Ronald S. Bultje
62844c3fd6 h264: Integrate clear_blocks calls with IDCT
The non-intra-pcm branch in hl_decode_mb (simple, 8bpp) goes from 700
to 672 cycles, and the complete loop of decode_mb_cabac and hl_decode_mb
(in the decode_slice loop) goes from 1759 to 1733 cycles on the clip
tested (cathedral), i.e. almost 30 cycles per mb faster.

Signed-off-by: Martin Storsjö <martin@martin.st>
2013-04-10 11:03:06 +03:00
Martin Storsjö
e8cafd2773 h264: Clear the mb members via memset instead of using dsputil
Signed-off-by: Martin Storsjö <martin@martin.st>
2013-04-10 11:01:21 +03:00
Michael Niedermayer
62a1181015 h264: wait for missing slices only on frames
The EC code does not support fields currently thus it makes no
sense to wait for these cases (which also the check doesnt handle
correctly)

Fixes Ticket 2454

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-04-09 23:12:24 +02:00
Michael Niedermayer
285c5f6838 h264: move last_slice_type reset into decode_slice_header
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-03-30 20:57:31 +01:00