This commit reworks the TNS implementation to a hybrid between what
the specifications say, what the decoder does and what's the best
thing to do.
The filter application function was copied from the decoder and
modified such that it applies the inverse AR filter to the
coefficients. The LPC coefficients themselves are fed into the
same quantization expression that the specifications say should
be used however further processing is not done, instead they're
converted to the form that the decoder expects them to be in
and are sent off to the compute_lpc_coeffs function exactly the
way the decoder does. This function does all conversions and will
return the exact coefficients that the decoder will generate, which
are then applied to the coefficients.
Having the exact same coefficients on both the encoder and decoder
is a must since otherwise the entire sfb's over which the filter
is applied will be attenuated.
Despite this major rework, TNS might not work fine on some audio
types at very low bitrates (e.g. sub 90kbps) as it can attenuate
some coefficients too much. Users are advised to experiment with
TNS at higher bitrates if they wish to use this tool or simply
wait for the implementation to be improved.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Turns out autocorrelating more than 750 coefficients at once
will cause a segfault, despite there being enough space to
hold an entire frame of samples into the buffer.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
This commit adds a function to get the reflection coefficients on
floating point samples. It's functionally identical to
ff_lpc_calc_ref_coefs() except it works on float samples and will
return the global prediction gain. The Welch window implementation
which is more optimized works only on int32_t samples so a slower
generic expression was used.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Not needed anymore, it was only used by the AAC TNS
encoder and was replaced with a more suitable function
in the following commit.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Needed for following commits. Contains the starting sfb for
every samplerate and window type.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Fixes out of array access
Fixes: 87196d8bbc633629fc9dd851fce73e70/asan_heap-oob_26f6853_862_cov_585961513_sonic3dblast_intro-partial.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This was copied from the decoder, but is unneeded for the encoder.
tns_max_bands is unused and set to zero which zeroed out start, end
and size and thus no filter was actually applied.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Since the coefficients are stepped up to order + 1 it was possible
that it went over TNS_MAX_ORDER. Also just return in case the only
coefficient is less than the threshold.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
The encoder-side filter isn't that important. The PSNR
shouldn't change so the FATE test should still be fine.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
The order should never go above TNS_MAX_ORDER (and thus cause
the context to be reinitialized) but this is just in case.
Also fix a comparison, since the coefficients are zero-indexed.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
It also made no sense to actually make the filter span the entire
window including the first band of the next window.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Pulses are already on the way so expect to see the list
gone in the close future.
TNS is already of sufficiently high quality to be enabled
by default (but isn't yet, so you too can help by testing!).
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
This commit abandons the way the specifications state to
quantize the coefficients, makes use of the new LPC float
functions and is much better.
The original way of converting non-normalized float samples
to int32_t which out LPC system expects was wrong and it was
wrong to assume the coefficients that are generated are also
valid. It was essentially a full garbage-in, garbage-out
system and it definitely shows when looking at spectrals
and listening. The high frequencies were very overattenuated.
The new LPC function performs the analysis directly.
The specifications state to quantize the coefficients into
four bit index values using an asin() function which of course
had to have ugly ternary operators because the function turns
negative if the coefficients are negative which when encoding
causes invalid bitstream to get generated.
This deviates from this by using the direct TNS tables, which
are fairly small since you only have 4 bits at most for index
values. The LPC values are directly quantized against the tables
and are then used to perform filtering after the requantization,
which simply fetches the array values.
The end result is that TNS works much better now and doesn't
attenuate anything but the actual signal, e.g. TNS removes
quantization errors and does it's job correctly now.
It might be enabled by default soon since it doesn't hurt and
helps reduce nastyness at low bitrates.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
This commit removes the array which was made redundant with
the last commit. The current prediction system gets the
quantization error directly (and without the single-frame delay)
in the search_for_pred function.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
This commit completely alters the algorithm of prediction.
The original commit which introduced prediction was completely
incorrect to even remotely care about what the actual coefficients
contain or whether any options were enabled. Not my actual fault.
This commit treats prediction the way the decoder does and expects
to do: like lossy encryption. Everything related to prediction now
happens at the very end but just before quantization and encoding
of coefficients. On the decoder side, prediction happens before
anything has had a chance to even access the coefficients.
Also the original implementation had problems because it actually
touched the band_type of special bands which already had their
scalefactor indices marked and it's a wonder the asserion wasn't
triggered when transmitting those.
Overall, this now drastically increases audio quality and you should
think about enabling it if you don't plan on playing anything encoded
on really old low power ultra-embedded devices since they might not
support decoding of prediction or AAC-Main. Though the specifications
were written ages ago and as times change so do the FLOPS.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
This was missed when the original commits were done. FF_PROFILE_UNKNOWN
is what's in avctx->profile when no audio profile is specified.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
This commit simply duplicates the functionality of ff_lpc_calc_coefs()
for the case of a Levinson-Durbin LPC with the only difference being
that floating point samples are accepted and the resulting coefficients
are raw and unquantized.
The motivation behind doing this is the fact that the AAC encoder
requires LPC in TNS and LTP and converting non-normalized floating
point coefficients to int32_t using SWR and again back for the LPC
coefficients was very impractical.
The current LPC interfaces were designed for int32_t in mind possibly
because FLAC and ALAC use this type for most internal operations.
The mathematics in case of floats remains of course identical.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
This commit simply moves the TNS tables to a more appropriate
aactab.h since then they can be accessed by both the decoder
and encoder.
The encoder _shouldn't_ normally need the tables since the
specs describe a specific quantization process, but the exact
reason for this can be seen in the TNS commit following.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
At least for vdpau, the hwaccel init code tries to check the video
profile and ensure that there is a matching vdpau profile available.
If it can't find a match, it will fail to initialise.
In the case of wmv3/vc1, I observed initialisation to fail all the
time. It turns out that this is due to the hwaccel being initialised
very early in the codec init, before the profile has been extracted
and set.
Conceptually, it's a simple fix to reorder the init code, but it gets
messy really fast because ff_get_format(), which is what implicitly
trigger hwaccel init, is called multiple times through various shared
init calls from h263, etc. It's incredibly hard to prove to my own
satisfaction that it's safe to move the vc1 specific init code
ahead of this generic code, but all the vc1 fate tests pass, and I've
visually inspected a couple of samples and things seem correct.
Signed-off-by: Philip Langdale <philipl@overt.org>
The amv one probably looks suspicious, but since it's an intra-only
codec, I couldn't possibly imagine what it would use the edge for,
and the vsynth fate result doesn't change, so it's probably OK.
The current algorithm is just "try all the combinations, and pick the best".
It's not very fast either, probably due to a lot of copying, but will do for
an initial implementation.
Signed-off-by: Donny Yang <work@kota.moe>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Assert on `avctx->codec->encode2` to avoid a SEGFAULT on the subsequent
function call.
avcodec_encode_video2() uses a similar assertion.
Calling the wrong function on a stream is a serious inconsistency
which could at other places be potentially dangerous and exploitable,
it is thus safer to stop execution and not continue with such
inconsistency after returning an error.
Commit-message-extended-by commiter
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
