a2fcab9978
Td4 and Te4 are arrays of u8. A u8 << int promotes the u8 to an int first then shifts.
If the mathematical result of a shift (as modelled by lhs * 2^{rhs}) is not representable
in an integer, behaviour is undefined. In other words, you can't shift into the sign bit
of a signed integer. Fix this by casting to u32 whenever we're shifting left by 24.
(For consistency, cast other shifts, too.)
Caught by -fsanitize=shift
Submitted by Nick Lewycky (Google)
Reviewed-by: Andy Polyakov <appro@openssl.org>
(cherry picked from commit 8b37e5c14f0eddb10c7f91ef91004622d90ef361)
This is an OpenSSL-compatible version of AES (also called Rijndael). aes_core.c is basically the same as rijndael-alg-fst.c but with an API that looks like the rest of the OpenSSL symmetric cipher suite.