Commit graph

2040 commits

Author SHA1 Message Date
redzic
f268d19641
Refactor: Remove parking_lot::Mutex from multi progress bar, rework mkvmerge concatenation implementation (#354)
- This removes the `parking_lot::Mutex` wrapping the multi progress bar, as it was not actually necessary. This also removes `parking_lot` as a dependency, as it is unused now.

- Rework the implementation of `mkvmerge` concatenation to not perform any unnecessary UTF-8 validation or allocations. This requires an algorithm to specify chunks with a '+' in between, but without a trailing '+', without later removing arguments (as the API of `std::process:Command` does not support this). This does not change the behavior of `-c mkvmerge` in any way, however.

- Also fix some miscellaneous clippy warnings.
2021-09-04 23:45:47 +03:00
Zen
906beb65f3 declutter lib.rs + prettify imports 2021-09-03 22:01:11 +03:00
Zen
6439b289df Cargo update 2021-09-02 01:06:56 +03:00
mxsrm
0316a4c0ce
Prevent data loss if concatenation failed (#353)
In some cases concatenation silently fails. To prevent loss of the
encoded data we now check if the output file exists. If it does not
exist we do not delete the temp folder.
2021-09-04 00:43:16 +03:00
Zen
9a95a00557 av1an to 0.1.0 + cargo release 2021-09-01 00:59:57 +03:00
Zen
1c7e3c816f cargo release 2021-08-31 14:22:24 +03:00
Zen
72cdd15b51 change av1an-cli structure, bump to 0.1.1 2021-08-31 14:22:15 +03:00
redzic
d0b27f9b73
Add output file confirmation prompt, improve error handling slightly (#349) 2021-08-31 12:36:31 +03:00
Zen
f98431900f av1an-cli dependencies 2021-08-30 16:15:26 +03:00
Zen
2b842e8c5a License corrections 2021-08-30 16:04:27 +03:00
Zen
0a9b664fdd fix versions 2021-08-30 15:54:53 +03:00
Zen
d48d3c92b4 add descriptions to av1an-cli, av1an-core 2021-08-30 15:53:31 +03:00
redzic
14574f17d1
Remove other unused dependencies (#347) 2021-08-29 19:29:03 +03:00
Zen
665249c104 set versions for av1an-cli, av1an-core 2021-08-29 19:14:34 +03:00
Zen
9eb9578340 removed unused clap 2021-08-29 19:05:22 +03:00
Zen
a99508f47f update cargo lock 2021-08-29 19:04:02 +03:00
Zen
44a7fe1831 fix dependency 2021-08-29 18:59:28 +03:00
Zen
8cf8e7a62f format toml for release 2021-08-29 18:53:56 +03:00
redzic
cebbee90d4
Refactoring, allow compiling with Rust 1.53 (#346)
Switch back to an old version of sysinfo, as 0.20 requires 1.54 to compile it, which is too new for MSYS2. Many minor simplifications.

We also switch to structopt for now instead of the beta release of clap, as clap officially recommends to not use the beta yet, and it caused some problems when compiling on Windows or with an older compiler.

Some minor changes:

- The `decow_strings` function has been removed, and we deal with `Cow<str>` properly now.
- The `remove_patterns` function now takes a mutable reference to a `Vec<String>` instead of copying one and modifing it to reduce complexity and overhead.
- The structs used to serialize the VMAF json result have been renamed to `VmafResult`, `Metrics`, and `VmafScore` (they were previously `Foo`, `Bar`, and `Baz`)
- Using an enum instead of a string as the argument to `log_probes` for better type safety and less overhead
- `frame_check_output` now warns if there is a mismatch, and the message has been updated to be more clear that a frame mismatch has occurred
- Several other small changes
2021-08-29 18:25:57 +03:00
redzic
cef0df5e87
Lazily compile all regexes, move macros to util module (#345)
* Lazily compile all regexes, move macros to util module

* Clarify/fix documentation
2021-08-29 06:11:25 +03:00
Zen
9240f27c20 some wip 2021-08-18 21:11:41 +03:00
Tatsuyuki Ishi
43da6b8687
ffmpeg: fix frame count parsing (#339)
The old code used to search for the first match that looked like a frame
count. When ffmpeg emits a warning, it would also emit the frame count at
the point of warning, therefore the previous implementation gave wrong
frame count when it happened.

Fix this by modifying the code to take the last match. Also replace the
index of cap by an explicit one since this is the index of capture group
(which is constant given the regex), not matches.
2021-08-22 07:35:54 +03:00
Tatsuyuki Ishi
f08628df2f
Replace some panics with Err returns (#341)
* av1an-core: replace some panics with Err returns
* av1an-cli: use anyhow::Result as the return type of main
2021-08-22 07:35:24 +03:00
Zen
9d8290d33e Rename queue to broker and move to different module 2021-08-17 07:32:35 +03:00
redzic
5f9fa28f5d
Integrate av1an-scene-detection into av1an-core, fix some clippy warnings, update dependencies (#336) 2021-08-19 22:27:46 +03:00
Josh Holmer
a8a6642b22
Add a fast scenecut method option (#335)
Selectable with `--split-method av-scenechange-fast`.
The fast method enables downscaling the video before scene detection,
and uses av-scenechange's fast mode, which is similar to pyscenedetect.
This also removes the downscaling from the normal scenecut mode,
since downscaling does affect accuracy of the scene detection.
2021-08-19 18:44:24 +03:00
Zen
eb899b9fc5 move target quality 2021-08-13 01:51:30 +03:00
Zen
2b1fd10094 chunk new int 2021-08-12 00:51:33 +03:00
Zen
20a32fb8be move chunk to separate mod + tests 2021-08-11 23:10:17 +03:00
Zen
9d19fb9be2 Output usage clarification 2021-08-09 02:42:48 +03:00
Zen
2980b70537 some doc comments 2021-08-07 02:40:00 +03:00
mxsrm
213e7c8e79
Small improvements to ffmpeg.rs (#331)
* ffmpeg: Ensure to map video streams only

Using 0✌️0 could map image files embedded into an mkv if they are
the first entry. 0:V:0 ensures that the first actual video stream
is mapped.

* ffmpeg: Drop subtitles when encoding audio

Formerly, when encoding audio av1an would also convert subtitles
(e.g. SRT would be converted to ASS). We should leave this decision
up to the user and drop subtitles while encoding audio.

* ffmpeg: Do not drop metadata

Formerly, metadata was dropped when encoding audio. This lead to missing
language tags on the audio file and to missing chapter names.
2021-08-16 20:17:52 +03:00
Zen
ba3295885c removed --webm flag 2021-08-16 20:17:33 +03:00
Josh Holmer
30aac252a3
Bump av-scenechange to 0.6 (includes rav1e 0.5-beta) (#330) 2021-08-14 07:57:06 +03:00
Zen
6ea2299465 Fix scenes file not working 2021-08-14 00:24:50 +03:00
Christophe Le Roy
013fc4f139
Fix --vmaf-filter handling in run_vmaf_on_chunk() (#327)
Co-authored-by: Le Roy Christophe <c.leroy@alphalink.fr>
2021-08-13 12:58:32 +03:00
Zen
c47e19bead Check validity of output file path 2021-08-06 19:06:39 +03:00
Zen
253b2bc551 Update cargo lock 2021-08-05 17:51:11 +03:00
Zen
5cbdae095d Fix hybrid chunking 2021-08-08 07:20:11 +03:00
Zen
06fb8c9edd update cargo lock 2021-07-27 22:43:27 +03:00
redzic
39494f06e2
Fix target quality and VMAF plotting on Windows (#319)
FFmpeg requires a very specific syntax to escape the file path in
Windows when using a filter, so this has been implemented with
conditional compilation. The behavior is the same on Windows, except
that the path is converted to an absolute path. Also, the only usage of
`std::env::current_dir` has been removed, since it was being used as an
ad-hoc way to convert a relative path to an absolute one, which doesn't
work if the temporary directory is in a different directory than the
current one.

We no longer kill the child processes from `process_pipe`,
since calling `wait` or `wait_with_output` closes the stdin handle
anyway.

Also, all usages of `cfg!(windows)` have been changed to `cfg!(target_os
= "windows")` to keep a consistent style.
2021-07-29 09:30:24 +03:00
redzic
239008e796
Fix scenes.json, update done.json when audio is done (#318) 2021-07-28 22:10:04 +03:00
redzic
adf67a85f3
Refactor: fix chunk restarting, progress bar, and potential race condition (#316)
`DoneJson` is now a concurrent data structure, via `DashMap` and atomics
in the standard library. This should help avoid any potential race
conditions involved with one thread reading `done.json` while another
thread writes to it. Now, we only write to `done.json`, so any failure
from parsing the json while encoding is impossible.

The audio encoding is now done on a separate thread, and it is not
assumed that the audio encoding was finished if the `--resume` flag was
specified, which is now kept track of within `done.json`.

The initialization of the progress bar(s) now does not occur if the
corresponding initialization functions were not called, and the
functions to update the progress bar(s) are a NO-OP if it was not
initialized, rather than implicitly initializing it through the use of
the `Lazy` type from `once_cell`.

Additionally, all 3 concatenation functions now reside in the `concat` module.
2021-07-26 21:01:52 +03:00
Nathaniel Chin
33e8fd3296
fix aur link (#315) 2021-07-26 13:02:39 +03:00
Zen
5e7f8841a3 improve readme 2021-07-22 12:49:04 +03:00
redzic
bf1b6e98b8
Better logging, integrate av1an-pyo3 into av1an-core (#314)
Implements more robust logging via flexi_logger. This replaces the old
logging implementation, and is also more ergonomic, since macros from the
`log` crate are now used instead of `format!` directly. Now, warnings
and errors are also logged to stderr, in addition to the log file.

This also removes av1an-pyo3, and integrates the code into av1an-core.

Concatenation with FFmpeg is now handled on Windows differently than
other platforms through conditional compilation. FFmpeg seems to need
double backslashes in the concat file, so this is explicitly handled
in the Windows build now. Also, the spinner is now disabled on Windows
builds since the default command prompt cannot display the characters
correctly.
2021-07-25 10:31:54 +03:00
Zen
f4513286d7 update cargo.lock 2021-07-24 20:10:10 +03:00
redzic
4ef9bdaa8a
More idiomatic Rust, add Cargo.lock to the repo (#313)
Previously, we represented quantizer values as an owned `String` for
constructing various encoder commands, which is much slower and less
robust than representing the quantizer as an integral value. This has
been updated to represent the `q` and `n_threads` both as a `usize`, and
only convert to a `String` for formatting as needed.

Furthermore, several struct fields containing paths have been updated to
use a `PathBuf` or `&Path` instead of a `String`. These are the first
steps required to supporting non-UTF8 filenames on Windows and
Unix-based operating systems.

`run_vmaf_on_chunk`'s function signature has been update to take generic
arguments that satisfy `AsRef<Path>`, which allows the caller to save
themselves from creating a new `Path` if the original argument was a
`String`, for example.

* Do not represent the concatenation method as a string
* Fix pedantic clippy warnings
* Remove Cargo.lock from .gitignore
* Fix startup_check for ivf concatenation
* Add doc comment to `Args` for better help generation in Clap
* Use Display impl instead of directly using str::From for `Encoder` and `ConcatMethod`
* Do not unwrap the result of killing child processes
* Assert that VMAF calculation was successful
* Use more idiomatic Rust
2021-07-24 20:09:21 +03:00
Josh Holmer
772eddd103
Add --verbose flag to show worker progress bars (#311) 2021-07-20 05:05:18 +03:00
redzic
5e1425f22e
Rename libvpx to vpx (#310) 2021-07-19 05:15:03 +03:00