Previously, the pixel format of the input file was checked every time
a chunk was started. Now, we just get the pixel format once at the
beginning and reuse it for all chunks.
The SSE4.1/SSSE3 implementation of the parsing routines for
aomenc/vpxenc outperform the old regex by a factor of about 5000x
(3.5 μs vs. 0.7 ns to parse a single line on a Zen 1 1950x).
The idea is to eventually remove regex as a dependency entirely,
especially since there is an open PR to indicatif that speeds up the
template expansion (which replaces regexes with hand-written parsing).
* Rename `--logging` to `--log-file` (`-l` shorthand still works)
* Introduce new `--log-level` option to set the log level filter
This change makes `--verbose` not log chunk starts and finishes, and
that information is not logged in the log file either unless
`--log-level=debug` is set. By default, `--log-level` is set to `info`.
The `debug` log level also logs the rav1e scenechange decision info
for each frame.
* Adjust verbose progress style for workers
* Also add `-y` flag to overwrite output file without confirmation
* Update dependencies
* Fix compilation on older versions of rustc
* Improve logging (especially with `--verbose`)
Much more information is now logged when using `--verbose` (all INFO
messages), and the output is also much cleaner.
The issue of logs being hidden while the progress bar is active
because of conflicting ANSI escape codes has been completely fixed, and
no more undesirable hacks are required to correctly display the
set_thread_affinity warning message correctly, for example. This has
been achieved by using indicatif's `println` function instead of just
printing to stderr directly.
flexi_logger has been moved to av1an-cli as it should be, since a
logging framework that actually prints things and handles log level
filtering is not the responsibility of the core library. However, the
actual progress bar code with indicatif still lives in av1an-core for
now.
* Make `--quiet` conflict with `--verbose`
* Fix resuming and improve error handling slightly
Introduces a new command line option called `--set-thread-affinity=n`,
where `n` specifies the number of threads to assign to each worker.
We use the `affinity` crate instead of the `nix` crate directly to
get abstractions for Windows and Linux.
* Always copy subtitles and all audio tracks
Require mkvmerge for x265
* cargo update
* Fix concat if audio file does not exist
* Use stable rustc in Docker, do not compile VVC
* Use old Docker image for now which works with vapoursynth
* Fix tests.yml
* Use mkvmerge for tests
* Add mkvtoolnix as a dependency in tests.yml
* Refactor: Better ffmpeg pipe handling + more info in --version
* Update dependencies
* Better documentation in `--help`
* Include git commit hash and more info in `--version`
* Add `-k` as shorthand for `--keep`
* Do not pass `ffmpeg_pipe` around everywhere
* Use macro to generate encoder bit depth functions
* Fix documentation in README
* Fix some clippy warnings
Previously, we always piped the chunk method into FFmpeg
to apply filters and convert the pixel format, but this
just adds unnecessary overhead if there is no FFmpeg filter
specified and if the pixel format of the source and encode
are the same. So, in that situation, we now pipe the chunk
method directly into the encoder instead. This should
nicely speed things up a bit (and possibly reduce memory
consumption).
Additionally, `--pix-format` is now verified using the FFmpeg
API when the CLI args are parsed, making it impossible to
accidentally specify an incorrect format. Previously, there
were no checks at all, so if you specified the wrong format it
would just fail during the encoding process with no obvious
indication of the root cause.
This does not apply to vapoursynth input for now, as that
would be slightly more complicated since it has a different
API for getting the pixel format.
* Refactor into `Input` type for handling vapoursynth input
Fixes several issues, namely `--vmaf` not working on vapoursynth input
as well as unifying the VMAF interface, which also results in
`--vmaf-threads` being passed correctly to `--vmaf`, which is massively
faster now as a result. If `--vmaf-threads` is not specified, then
the number of detected CPU threads is used for `--vmaf`.
Additionally, VMAF plots are rendered as SVGs now, which also removes
some dependencies since we don't use the bitmap backend at all anymore.
* Small cleanup: more consistent usage of `as_path`
This leaves the default at medium, which is an improved version
of the current scenecut detection method.
The fast method has also been improved from its previous version.
The slow method is a completely new implementation, which has
been shown to be more accurate, but is also about 15% slower.
This also removes the av-scenechange-fast split method,
in favor of making a separate CLI option for --sc-method,
which can be set to "fast", "medium", or "slow".
This also adds a CLI option for scenecut downscaling,
which can be specified by providing a maximum height e.g.
--sc-downscale-height 720 to downscale any input which is
above 720p to 720p for scenecut detection.
This will avoid upscaling anything below 720p, because that
would just slow down scenecut detection for no benefit.
- This removes the `parking_lot::Mutex` wrapping the multi progress bar, as it was not actually necessary. This also removes `parking_lot` as a dependency, as it is unused now.
- Rework the implementation of `mkvmerge` concatenation to not perform any unnecessary UTF-8 validation or allocations. This requires an algorithm to specify chunks with a '+' in between, but without a trailing '+', without later removing arguments (as the API of `std::process:Command` does not support this). This does not change the behavior of `-c mkvmerge` in any way, however.
- Also fix some miscellaneous clippy warnings.
Switch back to an old version of sysinfo, as 0.20 requires 1.54 to compile it, which is too new for MSYS2. Many minor simplifications.
We also switch to structopt for now instead of the beta release of clap, as clap officially recommends to not use the beta yet, and it caused some problems when compiling on Windows or with an older compiler.
Some minor changes:
- The `decow_strings` function has been removed, and we deal with `Cow<str>` properly now.
- The `remove_patterns` function now takes a mutable reference to a `Vec<String>` instead of copying one and modifing it to reduce complexity and overhead.
- The structs used to serialize the VMAF json result have been renamed to `VmafResult`, `Metrics`, and `VmafScore` (they were previously `Foo`, `Bar`, and `Baz`)
- Using an enum instead of a string as the argument to `log_probes` for better type safety and less overhead
- `frame_check_output` now warns if there is a mismatch, and the message has been updated to be more clear that a frame mismatch has occurred
- Several other small changes
Selectable with `--split-method av-scenechange-fast`.
The fast method enables downscaling the video before scene detection,
and uses av-scenechange's fast mode, which is similar to pyscenedetect.
This also removes the downscaling from the normal scenecut mode,
since downscaling does affect accuracy of the scene detection.
Implements more robust logging via flexi_logger. This replaces the old
logging implementation, and is also more ergonomic, since macros from the
`log` crate are now used instead of `format!` directly. Now, warnings
and errors are also logged to stderr, in addition to the log file.
This also removes av1an-pyo3, and integrates the code into av1an-core.
Concatenation with FFmpeg is now handled on Windows differently than
other platforms through conditional compilation. FFmpeg seems to need
double backslashes in the concat file, so this is explicitly handled
in the Windows build now. Also, the spinner is now disabled on Windows
builds since the default command prompt cannot display the characters
correctly.
Previously, we represented quantizer values as an owned `String` for
constructing various encoder commands, which is much slower and less
robust than representing the quantizer as an integral value. This has
been updated to represent the `q` and `n_threads` both as a `usize`, and
only convert to a `String` for formatting as needed.
Furthermore, several struct fields containing paths have been updated to
use a `PathBuf` or `&Path` instead of a `String`. These are the first
steps required to supporting non-UTF8 filenames on Windows and
Unix-based operating systems.
`run_vmaf_on_chunk`'s function signature has been update to take generic
arguments that satisfy `AsRef<Path>`, which allows the caller to save
themselves from creating a new `Path` if the original argument was a
`String`, for example.
* Do not represent the concatenation method as a string
* Fix pedantic clippy warnings
* Remove Cargo.lock from .gitignore
* Fix startup_check for ivf concatenation
* Add doc comment to `Args` for better help generation in Clap
* Use Display impl instead of directly using str::From for `Encoder` and `ConcatMethod`
* Do not unwrap the result of killing child processes
* Assert that VMAF calculation was successful
* Use more idiomatic Rust
* Fix clippy warnings, run `cargo fmt`
* Add clippy::needless_pass_by_value lint to av1an-core
* Add av-scenechange as the new and default scene detector
