Johann fea3556e20 Fix variance overflow ... In the variance calculations the difference is summed and later squared. When the sum exceeds sqrt(2^31) the value is treated as a negative when it is shifted which gives incorrect results. To fix this we cast the result of the multiplication as unsigned. The alternative fix is to shift sum down by 4 before multiplying. However that will reduce precision. For 16x16 blocks the maximum sum is 65280 and sqrt(2^31) is 46340 (and change). PPC change is untested. Change-Id: I1bad27ea0720067def6d71a6da5f789508cec265		2012-02-09 12:38:31 -08:00
..
csystemdependent.c	remove one set of 16x16 variance funcations	2011-06-09 11:23:05 -07:00
encodemb_altivec.asm	Use WebM in copyright notice for consistency	2010-09-09 10:01:21 -04:00
fdct_altivec.asm	Use WebM in copyright notice for consistency	2010-09-09 10:01:21 -04:00
rdopt_altivec.asm	Use WebM in copyright notice for consistency	2010-09-09 10:01:21 -04:00
sad_altivec.asm	Use WebM in copyright notice for consistency	2010-09-09 10:01:21 -04:00
variance_altivec.asm	Fix variance overflow	2012-02-09 12:38:31 -08:00
variance_subpixel_altivec.asm	Fix variance overflow	2012-02-09 12:38:31 -08:00