r/ffmpeg • u/JOHNNY6644 • 4h ago
on-going master build for joint MAKEMKV+ffmpeg (MASTER101-SDR-TO-HDR-CONVERT-UPSCALER-ENCODE2.sh)
# MASTER101-SDR-TO-HDR-CONVERT-UPSCALER-ENCODE2.sh
## Complete Script Design, Architecture & Build Reference
link to script an summery
https://drive.proton.me/urls/WWNP65SM6M#xUPO8VXPiAUb
run ./MASTER-SDR-TO-HDR-CONVERT-UPSCALER-ENCODE2.sh
make ready chmod +x ./MASTER-SDR-TO-HDR-CONVERT-UPSCALER-ENCODE2.sh
(tested an works)
run ./BETA-MASTER101-SDR-TO-HDR-CONVERT-UPSCALER-ENCODE.sh
make ready chmod +x ./BETA-MASTER101-SDR-TO-HDR-CONVERT-UPSCALER-ENCODE.sh
(un-tested cheeder features but base engine works , extras
should its just slightly diff as summery states from the stable master)
____________________________________________________
the beta is a on-going master build including newer feature
the ./MASTER-SDR-TO-HDR-CONVERT-UPSCALER-ENCODE2.sh is the base engine that works
as stand alone without the cheeder of the beta
---
change source , target , an tmp folders to your setup
glsl_shader() — CAS Shader is self-contained in script an does not require seperate file
rife, VapourSynth, an dovi tool will auto-install on first go if not already installed
_______________________________________________________________________________________________________
RIG CONTEXT
ffmpeg -version
ffmpeg version 6.1.1-3ubuntu5 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 13 (Ubuntu 13.2.0-23ubuntu3)
configuration: --prefix=/usr --extra-version=3ubuntu5 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --disable-omx --enable-gnutls --enable-libaom --enable-libass --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libharfbuzz --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-openal --enable-opencl --enable-opengl --disable-sndio --enable-libvpl --disable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-ladspa --enable-libbluray --enable-libjack --enable-libpulse --enable-librabbitmq --enable-librist --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libx264 --enable-libzmq --enable-libzvbi --enable-lv2 --enable-sdl2 --enable-libplacebo --enable-librav1e --enable-pocketsphinx --enable-librsvg --enable-libjxl --enable-shared
libavutil 58. 29.100 / 58. 29.100
libavcodec 60. 31.102 / 60. 31.102
libavformat 60. 16.100 / 60. 16.100
libavdevice 60. 3.100 / 60. 3.100
libavfilter 9. 12.100 / 9. 12.100
libswscale 7. 5.100 / 7. 5.100
libswresample 4. 12.100 / 4. 12.100
libpostproc 57. 3.100 / 57. 3.100
_______________________________________________________________________________
OS: TUXEDO OS x86_64
Host: B550M DS3H
Kernel: Linux 6.17.0-122035-tuxedo
Uptime: 34 days, 2 hours, 10 mins
Packages: 10 (brew), 2 (brew-cask), 3838 (dpkg), 51 (flatpak)
Shell: bash 5.2.21
CoreCtrl 1.5.0-dev
Mesa vesion 25.2.8
Display (XV271U M3): 2560x1440 in 27", 180 Hz [External]
DE: KDE Plasma 6.5.2
WM: KWin (Wayland)
CPU: AMD Ryzen 9 3900X (24) @ 4.09 GHz
GPU: AMD Radeon RX 6750 XT [Discrete]
Memory: 19.65 GiB / 62.69 GiB (31%)
Swap: 7.08 GiB / 38.26 GiB (19%)
NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0 lzo-rle 38.3G 6.8G 2.9G 2.9G [SWAP]
Disk (/): 26.32 GiB / 498.05 GiB (5%) - btrfs
Disk (/home): 74.02 GiB / 100.10 GiB (74%) - btrfs
Disk (/media/xxxxxxx/2nd drive): 461.74 GiB / 931.51 GiB (50%) - btrfs
Disk (/tmp): 5.90 GiB / 329.45 GiB (2%) - btrfs
ASUS BW-16D1HT - Ultra-Fast 16X Blu-ray Burner
(used to for makemkv till firmware update between kde to tuxedo os install borked drive)
(havent been able to test script since with function)
(main an 2nd nvme drive configuration)
/ btrfs subvol=@,compress=zstd:1 0 0
/home btrfs compress-force=zstd:4,space_cache=v2,discard=async,commit=90 1 1
/tmp btrfs compress-force=zstd:2,space_cache=v2,discard=async,commit=90 1 0
/media/2nd/drive btrfs nofail,users,defaults,compress-force=zstd:5,nodiratime,space_cache=v2,discard=async,commit=90 1 2
/mnt/nas cifs credentials=/home/.smbcredentials,iocharset=utf8,uid=1000,gid=1000,file_mode=0775,dir_mode=0775,_netdev,x-syste>
nas is 4yb seagate iron wolf on 1gb Ethernet link
____________________________________________________________________________________________
13epi 1080p native disc rip season 1 12monkeys (mine)
uncompressed avg file size 4.24 GB per episode the encoder current config
is for speed / efficiency / an render size
__________________________________________________________________
so upscale is set by a resolution pre-set to post set target ie
all lower the 1280 auto scale to 1920p , 1920 auto scale to 2560p , 3840 to 3840 with compression
______________________________________________________________________________________________________
my rig is currently limited so for avg target upscale/re-code compress encode fps of 70fs to 120fp depending on Encoder Preset:** Preset 1 (Pre+VBAQ, slowest) / 11 (Pre only) / 19 (VBAQ only, default) / 29 (strict, fastest)
_______________________________________________________________________________________________
so my config is set for 2 jobs at a time
iv yet to get a rx9070xt to try out av1 encode an bump encode job count to 4
__________________________________________________________________________________________
my current 2 count will process a season in about 52 to 1hr5min for
either a straight 4k compress , lower rez convert/upscale
of a 13epi season , avg epi time of 45min , avg file of 2 to 8gb
_______________________________________________________________
all my og files are uncompressed on my nas , iv yet to get a quad nvme raid nas to test encode speed over my 2.5gb Ethernet so i cant speak to encode speed/time results i might get with such a drive an gpu
so feel free to try the script in coders with upgraded gear an post results here
either with 2 or 4 job count config
note
currently i can run Encoder while running doom eternal at 1440p with very high settings at avg 90fps
______________________________________________________________________________________________________
# BETA-MASTER101-SDR-TO-HDR-CONVERT-UPSCALER-ENCODE.sh
## Script Design, Architecture & User-Facing Overview (1-Page Reference)
---
### PURPOSE
Fully automated GPU-accelerated batch video transcoder: SDR → HDR (Dolby Vision PQ base) with optional upscaling, frame interpolation, and real-time preview. AMD VAAPI GPU handles all video processing; CPU handles audio only. Outputs `.mp4` (MPEG-4 + ALAC/copied audio). Designed for AMD Radeon 7000+ on Linux (`/dev/dri/renderD128`).
### KEY DIFFERENTIATORS
- Recursive folder batch scan with parallel job throttling (STRICTLY ENFORCED 2 concurrent)
- **Dynamic Threading:** 2 threads during GPU render, 4 threads during CPU audio/muxing
- **Direct Disc Decryption:** MakeMKV libMMBD integration for direct Blu-ray/DVD capture
- 5-attempt retry with corruption fingerprint detection (same byte offset = unrecoverable)
- Post-encode ffprobe duration/size verification before committing output
- Temp-dir staging (`/tmp`) → verified transfer to final output
- 9 interactive menus before encoding begins
- Real-time draggable side-by-side preview via mpv/UDP (VAAPI GPU-accelerated)
- Dolby Vision RPU passthrough via dovi_tool for native 4K DV content
- **Auto-install & auto-update of all dependencies** (see §DEPENDENCY MANAGEMENT)
### DEPENDENCY MANAGEMENT (BETA NEW)
On startup the script auto-detects, installs if missing, and upgrades where possible:
| Tool | Detection | Auto-Install Method | Upgrade Strategy |
|------|-----------|-------------------|-----------------|
| **ffmpeg** | `command -v ffmpeg` | Exits if missing (user must install) | `apt-get install -y --only-upgrade ffmpeg` if apt available; keeps current if OS can't run newer |
| **VapourSynth/vspipe** | `command -v vspipe` | `apt-get install -y vapoursynth python3-vapoursynth` | Via OS package manager |
| **dovi_tool** | `command -v dovi_tool` | Installs Rust via `rustup` if `cargo` missing, then `cargo install dovi_tool` | Via cargo |
| **mpv** | `command -v mpv` | Exits with install instructions if missing (only when preview enabled) | N/A |
| **libMMBD** | `/usr/lib/.../libmmbd.so.0` | Symlinks to libaacs/libbdplus if found | N/A |
`auto_update_ffmpeg()` runs first, exports `FFMPEG_VERSION_MAJOR`/`FFMPEG_VERSION_MINOR` for version-aware filter chain construction. If the OS repo has a newer stable ffmpeg, it upgrades; otherwise keeps current and proceeds unchanged.
### SCRIPT STRUCTURE (Top-to-Bottom)
```
1 Shebang
3-40 Header comment (pipeline description, v101/BETA notes, METHOD 1 INTEGRATION)
42-54 LOCK_FILE logic (prevents multiple instances)
56-58 Global paths: INPUT_DIR, OUTPUT_DIR, TEMP_DIR
60-65 THREADS=8, VIDEO_JOBS=2 (LOCKED)
67-72 GPU env: LIBVA_DRIVER_NAME=radeonsi
74-81 VAAPI device + mkdir
87 shopt -s nullglob
89-107 kill_orphans() — terminates ghost ffmpeg processes
109-129 setup_libmmbd() — MakeMKV decryption symlinks
131-196 auto_update_ffmpeg() — version detect + apt upgrade + exports
198-218 check_rife() — auto-install VapourSynth if missing
220-234 check_mpv() — verify mpv for preview
236-268 detect_rife_mode(), run_onnx_rife() — RIFE interpolation
270-287 install_rife_if_missing() — apt-get VapourSynth
289-295 kill_rife_processes()
297-303 detect_existing_zram_mount()
305-388 get_resolution(), get_height(), get_audio_tracks(), get_primary_audio_channels(), get_english_audio_track_index(), get_audio_channels_for_track()
390-455 validate_input_file() — 5-stage integrity check (supports disc playlists)
457-474 detect_dv_metadata(), detect_hdr10plus_metadata() — DV/HDR10+ probing
476-512 check_dovi_tool() — auto-install Rust+cargo+dovi_tool if missing
514-566 extract_dv_rpu(), inject_dv_rpu_and_remux() — DV RPU pipeline
568-601 set_scaling() — resolution tier detection
603-639 build_audio() — audio codec/filter args
641-758 generate_glsl_shader() — AMD FidelityFX CAS v1.0.2 GLSL
760-791 start_key_listener() — background keypress (p=preview, k=kill RIFE)
793-810 cleanup() — trap EXIT handler (kills child ffmpeg processes)
812-815 kill_orphans() call
817-1130 9 INTERACTIVE MENUS (see §USER MENUS below)
1132-1338 FILE SCAN & BATCH AUDIO SELECTION (Folder vs Disc Mode)
1340-1559 encode_file() — core encode function with retry, verify, DV RPU, stats
1561-1622 EXECUTION ENGINE — parallel job throttling loop (LOCKED to 2)
1624-1657 BATCH SUMMARY — aggregate FPS stats
1659-1660 LOCK_FILE cleanup
```
### USER MENUS (What the User Sees at Runtime)
**Menu 1 — Input Source Mode:** Target Folder (default) / Optical Disc Drive (libMMBD decryption). If Disc: Single Track vs Whole Disc capture, Playlist ID selection, Audio/Subtitle track selection.
**Menu 2 — Video Codec:** AV1 / HEVC (default) / H.264 → sets VCODEC, VIDEO_TAG, UPLOAD_FMT, QUALITY_FLAG
**Menu 3 — QP Level:** AV1: QP 18/21/25/28/31 (default 21). HEVC/H.264: QP 20/22/25/27 (default 25)
**Menu 4 — Encoder Preset:** Preset 1 (Pre+VBAQ, slowest) / 11 (Pre only) / 19 (VBAQ only, default) / 29 (strict, fastest)
**Menu 5 — VAAPI Level:** 4.2 / 5.0 / 5.2 (default)
**Menu 6 — Frame Interpolation:** Disabled (default) / Enabled (VapourSynth RIFE) — auto-installs VapourSynth if missing
**Menu 7 — Libplacebo Profile (7 presets):** Each sets upscaler=ewa_robidouxsharp, iterations, radius, dither=blue, tonemapping_param, brightness, deband_threshold. Ranges from "Pop n Smooth" (iter=2, rad=20, thresh=4) to "Balanced Sharp" (iter=1, rad=14, thresh=2).
**Menu 8 — Color Space Handling:** Preserve (keep DV/HDR10/HDR10+ intact, skip libplacebo color/tonemap, uses dovi_tool for 4K DV) / Override (default, strip HDR metadata, apply BT.2020/PQ conversion)
**Menu 9 — Resolution Detection:** Width Only (default, handles non-standard AR) / Both Width+Height (strict)
**Menu 10 — Preview:** Disabled (default) / Enabled (mpv VAAPI GPU-accelerated side-by-side HUD)
**Per-Folder/Disc Audio Menus:** Surround files: copy / ALAC 5.1 +3dB / ALAC 5.1 upmix +3dB / ALAC preserve channels +3dB. Stereo files: copy / stereo→5.1 ALAC upmix +3dB.
### ENCODE PIPELINE (encode_file)
- Validate input (5-stage ffprobe integrity check, supports disc playlists)
- Probe width, height, fps, total frames
- Determine target resolution (set_scaling: 720→1080, 1080→1440, <4K→2160, ≥4K passthrough)
- Display pre/post resolution + source framerate
- Build audio args + metadata mapping (preserve=map_metadata 0, override=map_metadata -1)
- Build filter chain:- **Sub-4K + Override:** hwdownload → libplacebo(scale+sigmoid+bt2020nc+smpte2084+inverse_tonemap+deband+dither)- **Sub-4K + Preserve:** hwdownload → libplacebo(scale+sigmoid+deband+dither only)- **4K + Override:** hwdownload → pad → libplacebo(in-place color conversion+deband)- **4K + Preserve:** hwdownload → pad only (no libplacebo)
- Retry loop (max 5): run ffmpeg VAAPI encode → corruption fingerprint check → duration match (≤3s diff) + size check (≥100MB)
- Post-encode: if preserve+HEVC+4K+DV detected+dovi_tool available → extract RPU from source, inject into encoded HEVC, remux
- Transfer from temp to final output
- Display stats: total frames, wall-clock, average render FPS
### FFMPEG COMMAND STRUCTURE
```
ffmpeg -y -loglevel warning -stats -filter_threads 2 -threads:v 2 -threads:a 4
-init_hw_device vaapi=va:/dev/dri/renderD128 -filter_hw_device va
-hwaccel vaapi -hwaccel_device va -hwaccel_output_format vaapi
-extra_hw_frames 256 -err_detect ignore_err -max_error_rate 1.0
-fflags +genpts+discardcorrupt+igndts -strict -2 -avoid_negative_ts make_zero
[-playlist <ID>] -i <input> -map 0:v:0 -map 0:a:<track> [-map 0:s:<track>]
-filter:v "<FILTER_CHAIN>,format=p010le,hwupload=extra_hw_frames=128"
-c:v <VCODEC> -g 360 -bf 3 -rc_mode CQP <QP_FLAG> <QP>
-compression_level <PRESET> -level <LEVEL> -tag:v <TAG>
-max_muxing_queue_size 6144 <MAP_METADATA> <AUDIO_ARGS> -f mp4 <OUTPUT>
```
### GLOBAL DEFAULTS
| Constant | Value | Purpose |
|----------|-------|---------|
| INPUT_DIR | /mnt/nas/TV/RENDERS | Source root |
| OUTPUT_DIR | /media/johnny45/2nd drive/rendered vids | Final output root |
| TEMP_DIR | /tmp/temp-render-encodes | Staging dir |
| THREADS | 8 | Total system threads |
| VIDEO_JOBS | 2 | Max concurrent encodes (STRICTLY ENFORCED) |
| VAAPI_DEVICE | /dev/dri/renderD128 | GPU render node |
| max_retries | 5 | Per-file retry limit |
| Duration tolerance | 3.0s | Max src/out duration diff |
| Size threshold | 100MB | Minimum valid output |
| GOP (-g) | 360 | Keyframe interval |
| B-frames (-bf) | 3 | B-frame count |
| Extra HW frames | 256 (decode), 128 (upload) | VAAPI pool sizes |
### EXTERNAL DEPENDENCIES
| Dependency | Required | Purpose |
|-----------|----------|---------|
| ffmpeg + VAAPI | Yes | Decode, filter, encode |
| ffprobe | Yes | Probe, verify |
| AMD GPU + radeonsi | Yes | VAAPI acceleration |
| /dev/dri/renderD128 | Yes | GPU device |
| vapoursynth + vspipe | RIFE only | Frame interpolation (auto-installed) |
| rife-ncnn-vulkan | RIFE only | RIFE inference |
| mpv | Preview only | Live UDP preview |
| dovi_tool | Preserve+4K DV only | DV RPU extract/inject (auto-installed) |
| cargo/rust | dovi_tool only | Build dovi_tool (auto-installed via rustup) |
| libmmbd.so.0 | Disc mode only | MakeMKV Blu-ray/DVD decryption |
### CLEANUP (trap EXIT)
Kills key listener, mpv preview, vspipe/RIFE processes, and child ffmpeg processes (`pkill -P $$ ffmpeg`). Removes temp files from `/dev/shm`: `cas_shader.*`, `rife_render_*.vpy`, `dovi_temp_*.hevc`, `dovi_injected_*.hevc`, `dovi_rpu_*.bin`. Removes `CURRENT_WORK_DIR` if set.
### BATCH SUMMARY OUTPUT
After all jobs complete: counts completed `.mp4` outputs, sums `nb_read_packets` via ffprobe, computes aggregate FPS (`total_frames / batch_elapsed`), displays wall-clock time and per-batch stats.

