The current one, while correct, does not yield the best possible
results. The specificiations suggest another formula, which results
in quality gains in the decoded output from fate tests. This
justifies changing said formula.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Convert them to zigzag order, as the rest of them are.
When I was adding support for 10-bit DNxHD, I just copy-pasted the
missing quant matrices from the spec. Now it turns out the existing
matrices in dnxhddata.c were in zigzag order. This resulted in wrong
quantization for 10-bit DNxHD. The attached patch fixes the problem by
converting 10-bit quant matrices to zigzag order.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The System V ABI on x86-64 specifies that the al register contains an upper
bound of the number of arguments passed in vector registers when calling
variadic functions, so we aren't allowed to clobber it.
checkasm_fail_func() is a variadic function so also zero al before calling it.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Tested functions are internally kept in a binary search tree for efficient
lookups. The downside of the current implementation is that the tree quickly
becomes unbalanced which causes an unneccessary amount of comparisons between
nodes. Improve this by changing the tree into a self-balancing left-leaning
red-black tree with a worst case lookup/insertion time complexity of O(log n).
Significantly reduces the recursion depth and makes the tests run around 10%
faster overall. The relative performance improvement compared to the existing
non-balanced tree will also most likely increase as more tests are added.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
The System V ABI on x86-64 specifies that the al register contains an upper
bound of the number of arguments passed in vector registers when calling
variadic functions, so we aren't allowed to clobber it.
checkasm_fail_func() is a variadic function so also zero al before calling it.
Tested functions are internally kept in a binary search tree for efficient
lookups. The downside of the current implementation is that the tree quickly
becomes unbalanced which causes an unneccessary amount of comparisons between
nodes. Improve this by changing the tree into a self-balancing left-leaning
red-black tree with a worst case lookup/insertion time complexity of O(log n).
Significantly reduces the recursion depth and makes the tests run around 10%
faster overall. The relative performance improvement compared to the existing
non-balanced tree will also most likely increase as more tests are added.
This patch tweaks search_for_pns to be both more
aggressive and more careful when applying PNS. On
the one side, it will again try to use PNS on zero
(or effectively zero) bands. For this, both zeroes
and band_type have to be checked (some ZERO bands
aren't marked in zeroes). On the other side, a more
accurate rate-distortion measure avoids using PNS
where it would cause audible distortion.
Also fixed a small bug in the computation of freq
that caused PNS usage on low-frequency bands during
8-short windows. This allows re-enabling PNS during
8-short.
The sample position is made weird and non-nominal to force catching
such issues as default values or specialized operations hiding
issues in corner cases.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This patch modifies the encode frame function to
retry encoding the frame when the resulting bit count
is too far off target, but only adjusting lambda
in small, incremental step. It also makes the logic
more conservative - otherwise it will contend with
bit reservoir-related variations in bit allocation,
and result in artifacts when frame have to be truncated
(usually at high bit rates transitioning from low
complexity to high complexity).
The randomize_buffer() implementation assures that "most of the time",
we'll do a good mix of wide16/wide8/hev/regular/no filters for complete
code coverage. However, this is not mathematically assured because that
would make the code either much more complex, or much less random.
This patch refactors the AAC coders to reuse code
between the MIPS port and the regular, portable C code.
There were two main functions that had to use
hand-optimized versions of quantization code:
- search_for_quantizers_twoloop
- codebook_trellis_rate
Those two were split into their own template header
files so they can be inlined inside both the MIPS port
and the generic code. In each context, they'll link
to their specialized implementations, and thus be
optimized by the compiler.
This approach I believe is better than maintaining
several copies of each function. As past experience has
proven, having to keep those in sync was error prone.
In this way, they will remain in sync by default.
Also, an implementation of the dequantized output
argument for the optimized quantize_and_encode
functions is included in the patch. While the current
implementation of search_for_pred still isn't using
it, future iterations of main prediction probably will.
It should not imply any measurable performance hit while
not being used.
This introduces a slight timebase computation difference in zmbv-8bit
fate test. This is expected since the new options are double instead
of ints, and the additional precision skews the results in a non
meaningful way.
The recent commits change the value slightly. Even though it's
within the threshold it's better to risk as little as possible
especially when different systems, processors, FPUs and compilers
are involved.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
This commit changes a few things about the noise substitution
logic:
- Brings back the quantization factor (reduced to 3) during
scalefactor index calculations.
- Rejects any zeroed bands. They should be inaudiable and it's
a waste transmitting the scalefactor indices for these.
- Uses swb_offsets instead of incrementing a 'start' with every
window group size.
- Rejects all PNS during short windows.
Overall improves quality. There was a plan to use the lfg system
to create the random numbers instead of using whatever the decoder
uses but for now this works fine. Entropy is far from important here.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
This commit once again improves the PNS implementation by scaling the
thresholds with frequency. The thresholds get looser as the frequency
increases since higher frequencies are basically noise to human ears.
Also, this introduces quantization error correction for PNS. Should
the error be too much, no PNS will be used. The energy_ratio is used
to regulate the actual encoded PNS energy: if the generated PNS
energy is higher than the energy from the psy system, energy_ratio
is used to correct it so that hopefully once requantized and
transmitted the value in the decoder will be closer to what the
encoder has.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
This was an oversight when the IS system was being first implemented.
The ener01 part was largely a result of trial and error and the fact
that the sum of coef0 and coef1 could result in a zero was
overlooked. Once ener01 turns to zero it's used to divide the left
channel energy which doesn't turn out so well as it fills IS[]
with -nan's and inf's which in turn confused the quantize_band_cost.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
FATE refs changed to accomodate for the new default behavior of the function.
Numbers are now interpreted as a channel layout, instead of a number of channels.
TNS had both IS and PNS switched on when it makes more sense
to have them both off.
Prediction had a redundant argument.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
IS and PNS increase quality a ton so as a result the PSNR changed.
Disable the extensions and keep the tests separate such that there
will be no red herrings if one test fails.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Without this fate-filter-join failes with
FF_API_GET_CHANNEL_LAYOUT_COMPAT disabled.
Reviewed-by: Ronald S. Bultje <rsbultje@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes fate with FF_API_LAVF_BITEXACT disabled.
Reviewed-by: Ronald S. Bultje <rsbultje@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Tests fails on some ARM builds but it's close enough so it's okay.
NEON, half-precision floats, rounding errors, who knows.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
This commit introduces a test for AAC-Main prediction
which was just reworked in this series of commits.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Works only for flv, h263 and huffyuv decoders.
Makes only one pass through the file (this should be changed to two passes)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This fixes fate with FF_API_REQUEST_CHANNELS disabled.
Reviewed-by: wm4 <nfxjfg@googlemail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Works only with video stream.
First pass without seeking -- counts crcs of a frames and store it in an array.
After that it seeks a lot in different places and checks if crcs of these frames and crcs of frames in array are the same.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Now we no longer have to rely on function pointers intentionally
declared without specified argument types.
This makes it easier to support functions with floating point parameters
or return values as well as functions returning 64-bit values on 32-bit
architectures. It also avoids having to explicitly cast strides to
ptrdiff_t for example.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
If the return value doesn't fit in a single register rdx/edx can in some
cases be used in addition to rax/eax.
Doesn't affect any of the existing checkasm tests but might be useful later.
Also comment the relevant code a bit better.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
If the return value doesn't fit in a single register rdx/edx can in some
cases be used in addition to rax/eax.
Doesn't affect any of the existing checkasm tests but might be useful later.
Also comment the relevant code a bit better.
Now we no longer have to rely on function pointers intentionally
declared without specified argument types.
This makes it easier to support functions with floating point parameters
or return values as well as functions returning 64-bit values on 32-bit
architectures. It also avoids having to explicitly cast strides to
ptrdiff_t for example.
* commit '58c3720a3cc71142b5d48d8ccdc9213f9a66cd33':
fate: Make sure a corner-case for ASF is covered
Adjusted fate ref to match the different timebase of the ffasf demuxer
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
