generic-library/ffmpeg
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

h264: vdpau: fix crash with unsupported colorspace

Browse Source 
The h264_vdpau decoder crashed if output colorspace was not 8-bit 420.
Add a check to error out instead (current hardware does not support
other colorspaces, so successful decoding is not possible).

Signed-off-by: Martin Storsjö <martin@martin.st>
This commit is contained in:
Uoti Urpala 2012-08-07 14:48:06 +03:00 committed by Martin Storsjö
parent ca00a7e809
commit 2e8f3cbcda
1 changed files with 9 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
libavcodec/h264.c
View File

@ -3876,6 +3876,15 @@ again:
if (avctx->bits_per_raw_sample != h->sps.bit_depth_luma ||
h->cur_chroma_format_idc != h->sps.chroma_format_idc) {
if (s->avctx->codec->capabilities & CODEC_CAP_HWACCEL_VDPAU
&& (h->sps.bit_depth_luma != 8 ||
h->sps.chroma_format_idc > 1)) {
av_log(avctx, AV_LOG_ERROR,
"VDPAU decoding does not support video "
"colorspace\n");
buf_index = -1;
goto end;
}
if (h->sps.bit_depth_luma >= 8 && h->sps.bit_depth_luma <= 10) {
avctx->bits_per_raw_sample = h->sps.bit_depth_luma;
h->cur_chroma_format_idc = h->sps.chroma_format_idc;