diff --git a/compose/base.yaml b/compose/base.yaml index e369c25..1f36fc5 100644 --- a/compose/base.yaml +++ b/compose/base.yaml @@ -215,11 +215,14 @@ services: - SEERR_DELETE_REQUEST_PURGE=${SEERR_DELETE_REQUEST_PURGE:-1} - SONARR_URL=http://sonarr:8989 - SONARR_CONFIG=/config/sonarr/config.xml + - RADARR_URL=http://radarr:7878 + - RADARR_CONFIG=/config/radarr/config.xml volumes: - /etc/localtime:/etc/localtime:ro - ${INSTALL_DIRECTORY}/scripts/seerr-gateway.py:/scripts/seerr-gateway.py:ro # Bind the dir, not config.xml — Docker creates a directory if the file is missing. - ${INSTALL_DIRECTORY}/config/sonarr:/config/sonarr:ro + - ${INSTALL_DIRECTORY}/config/radarr:/config/radarr:ro ports: - 5055:5055 depends_on: diff --git a/docs/seerr-delete-request.md b/docs/seerr-delete-request.md index efc6772..4664d9f 100644 --- a/docs/seerr-delete-request.md +++ b/docs/seerr-delete-request.md @@ -22,7 +22,11 @@ On **`DELETE /api/v1/request/:id`** only (when purge is enabled): 1. `GET /api/v1/request/:id` — capture linked `media.id`, `is4k`, type, and requested season numbers (same auth headers as the client). -2. Purge files: +2. **Drain the *arr download queue first** (while the title still exists): + `DELETE /api/v3/queue/{id}?removeFromClient=true`. That is how an in-progress + grab is cancelled — Radarr/Sonarr already know the torrent hash. Queue drain + is best-effort (missing API key or empty queue is not a hard failure). +3. Purge files: - **Movie**, or a TV request that covers every remaining on-disk/monitored season (specials count as remaining) → `DELETE /api/v1/media/:mediaId/file?is4k=…` (Seerr → *arr `deleteFiles` → assemblrr purge hooks / qB). @@ -30,7 +34,7 @@ On **`DELETE /api/v1/request/:id`** only (when purge is enabled): seasons only, then unmonitor them. Seerr title-level file-delete is **not** called, so `/data/media/tv/Show` is not wiped. `media-purge-watch` then season-scopes qB cleanup when that season folder empties. -3. `DELETE /api/v1/request/:id` — drop the request row (response returned to +4. `DELETE /api/v1/request/:id` — drop the request row (response returned to the client). If media was never linked, step 2 is skipped; the request is still deleted. @@ -63,15 +67,19 @@ qB row first, which used to leave the download directory behind). `media-purge.sh` prefers under-delete over collateral damage: -1. *arr history download hashes that still exist in qB **and** match the library folder key (`Title (YYYY)`) -2. Exact folder-key match on torrent name / content path (no bare short-title prefix) -3. Refuse ambiguous multi-matches +1. *arr **queue + history** download hashes that still exist in qB. The hash is + identity — a live grab named `The Avengers 2012 REPACK…` is removed even + though it does not look like `The Avengers (2012)`. +2. Title + year match on torrent name / content path. `(2012)` and `2012` are + the same year; indexer prefixes (`www.UIndex.org - …`) are allowed; the + token after the title must be that year so `Avengers Age of Ultron 2015` + and `It Comes at Night 2017` stay put. +3. Refuse ambiguous multi-matches on the name step. Short keys (`Oz`) skipped. 4. **Season purge:** torrent must name one of the requested seasons (`S01` / `Season 1`) and must **not** also name another season. Packs like `Show.S01.S02.COMPLETE` and ranges like `Show.S01-03` are left alone - when deleting S01 only. A torrent with no season token is left alone. - Folder-key matching uses the same title-boundary rules as title-level - (short keys such as `Oz` are skipped). + when deleting S01 only. A torrent with no season token is left alone + on the name step (queue/history hashes for that season are still removed). See `media-purge.sh --match-self-test`. @@ -82,13 +90,15 @@ Env: | `SEERR_DELETE_REQUEST_PURGE` | `1` | `1` = intercept delete-request; `0`/`false`/`off` = pure passthrough (stock Seerr) | | `MEDIA_PURGE_WATCH` | `1` | `1` = inotify watcher on library video deletes; `0` = disable watcher only | | `SEERR_UPSTREAM` | `http://seerr:5055` | Upstream Seerr (container) | -| `SONARR_URL` | `http://sonarr:8989` | Used by the gateway for TV season deletes | +| `SONARR_URL` | `http://sonarr:8989` | Used by the gateway for TV season deletes + queue drain | | `SONARR_CONFIG` | `/config/sonarr/config.xml` | API key source when `SONARR_API_KEY` is unset | +| `RADARR_URL` | `http://radarr:7878` | Used by the gateway to drain in-progress movie grabs | +| `RADARR_CONFIG` | `/config/radarr/config.xml` | API key source when `RADARR_API_KEY` is unset | Debug response headers on intercepted deletes: - `X-Assemblrr-Request-Delete-Purge: 1` -- `X-Assemblrr-Purge-Detail: …` (e.g. `media_file_deleted:7:204`, +- `X-Assemblrr-Purge-Detail: …` (e.g. `queue_movie:1:1;media_file_deleted:7:204`, `seasons_deleted:6:10:1:3`, `no_media`) Health (gateway only, not Seerr): diff --git a/scripts/media-purge.sh b/scripts/media-purge.sh index b460a02..8cb7a5b 100755 --- a/scripts/media-purge.sh +++ b/scripts/media-purge.sh @@ -11,10 +11,13 @@ # also name another season. # # Safety ladder for qB cleanup (prefer under-delete over collateral damage): -# 1) *arr history download hashes that still exist in qB and match the folder key -# 2) Exact library-folder key match on torrent name/content_path (Title (YYYY)) -# 3) Never bare short titles when a year-bearing folder exists -# 4) Refuse ambiguous multi-matches on the folder-key step +# 1) *arr queue + history download hashes that still exist in qB (hash is identity) +# 2) Title + year match on torrent name / content path (The Avengers (2012) == +# The Avengers 2012 REPACK…; indexer prefixes allowed; next token after the +# title must be that year). No-year keys still require a season/year/end +# boundary so Lost does not hit Lost in Space. +# 3) Never bare short titles (len < 3). Refuse ambiguous multi-matches on +# the name step. Season purge still refuses packs / other seasons. # After a successful non-qb-only purge, best-effort Jellyfin /Library/Refresh. set -euo pipefail @@ -207,95 +210,107 @@ qbit_delete_hashes() { PURGE_TOUCHED=1 } -# Folder-key matcher: exact / boundary-safe match only. -# stdin: qB torrents JSON; arg1: JSON string array of folder keys (prefer Title (YYYY)). -qb_match_jq() { - jq -r --argjson tokens "$1" ' +# Title + year matcher shared by name fallback and leftover-dir cleanup. +# A movie key "The Avengers (2012)" hits "The Avengers 2012 REPACK…" and +# "www.UIndex.org - The Avengers 2012 …" because the title tokens appear +# consecutively and the next token is that year. It does not hit Age of Ultron +# (next token is not 2012) or "It Comes at Night" for It (2017). +QB_TITLE_YEAR_JQ=$(cat <<'JQ' def norm: ascii_downcase | gsub("\\\\"; "/") - | gsub("[._]+"; " ") + | gsub("['’]"; "") + | gsub("[^a-z0-9]+"; " ") | gsub(" +"; " ") | gsub("^ +| +$"; ""); - def has_year: test("\\([0-9]{4}\\)"); - def basename_norm: - gsub("/+$"; "") | split("/") | map(select(length > 0)) | .[-1] // "" | norm; - # After token: end, year, tag, separator — or a season token (Loki.S02.COMPLETE). - # "Lost in Space" still fails: remainder is " in space", not a season. - def boundary_ok($t; $tok): - ($tok | length) as $m - | ($t | length) as $n - | if $n < $m then false - elif $n == $m then true - else ($t[$m:] | test("^($| \\([0-9]{4}\\)| \\[| - | /|\\.[0-9]{4}| s[0-9]| season |[-.]s[0-9])")) - end; - def folder_hit($text; $tok): - ($text | norm) as $t - | ($tok | norm) as $k - | if ($k | length) < 3 then false + def tokens($text): + [($text | norm) | split(" ")[] | select(length > 0)]; + def basename_of($text): + ($text | gsub("\\\\"; "/") | gsub("/+$"; "") | split("/") + | map(select(length > 0)) | .[-1] // ""); + def is_year: test("^[0-9]{4}$"); + def is_season_tok: + test("^s[0-9]{1,2}(e[0-9]{1,3})?$") or . == "season" or . == "specials"; + def parse_folder($folder): + ($folder // "") as $raw + | ([$raw | ascii_downcase | scan("\\(([0-9]{4})\\)") + | if type == "array" then .[0] else . end] + | .[-1] // "") as $paren_year + | (tokens($raw | gsub("\\([0-9]{4}\\)"; " "))) as $words + | if $paren_year != "" then + {title: $words, year: $paren_year} + elif (($words | length) > 0) and ($words[-1] | is_year) then + {title: $words[0:-1], year: $words[-1]} else - ($t | basename_norm) as $base - | ( - ($base == $k) - or ($t == $k) - or ($t | endswith("/" + $k)) - or ($t | contains("/" + $k + "/")) - or (($base | startswith($k)) and boundary_ok($base; $k)) - or (($t | startswith($k)) and boundary_ok($t; $k)) - ) + {title: $words, year: ""} end; - ($tokens | map(norm) | map(select(length >= 3)) | unique) as $all + def drop_articles($words): + if ($words | length) > 1 and ($words[0] == "the" or $words[0] == "a" or $words[0] == "an") + then $words[1:] + else $words + end; + def find_title_then($toks; $title; $need_year): + ($title | length) as $n + | if $n == 0 then false + else + any(range(0; ($toks | length) - $n + 1); + . as $i + | $toks[$i:$i+$n] == $title + and ( + ($toks[$i+$n] // "") as $next + | if $need_year != "" then + $next == $need_year + else + $next == "" or ($next | is_season_tok) or ($next | is_year) + end + ) + ) + end; + def title_year_hit($text; $folder): + if ($folder | length) == 0 or (($folder | norm | length) < 3) then false + else + parse_folder($folder) as $fk + | tokens($text) as $toks + | tokens(basename_of($text)) as $base + | ($fk.title | length) > 0 + and ( + ($text | norm) == ($folder | norm) + or (basename_of($text) | norm) == ($folder | norm) + or any([$fk.title, drop_articles($fk.title)][]; + . as $cand + | ($cand | length) > 0 + and (find_title_then($toks; $cand; $fk.year) + or find_title_then($base; $cand; $fk.year)) + ) + ) + end; + def has_year: test("\\([0-9]{4}\\)") or test("(^| )[0-9]{4}$"); +JQ +) + +# stdin: qB torrents JSON; arg1: JSON string array of folder keys (prefer Title (YYYY)). +qb_match_jq() { + jq -r --argjson tokens "$1" "${QB_TITLE_YEAR_JQ}"' + ($tokens | map(select(length >= 3)) | unique) as $all | ([$all[] | select(has_year)]) as $yeared | (if ($yeared | length) > 0 then $yeared else $all end) as $use | .[] | . as $row | select( any($use[]; - folder_hit($row.name // ""; .) - or folder_hit($row.content_path // ""; .) + title_year_hit($row.name // ""; .) + or title_year_hit($row.content_path // ""; .) ) ) | .hash ' } -# Season-scoped matcher: require at least one requested season token in the -# name/path, refuse if the torrent also names a season we are not deleting -# (including S01-03 / Season 1-3 ranges), and require the same folder-key -# rules as title-level (short keys skipped). Torrents with no parseable -# season are skipped (under-delete). +# Season-scoped matcher: title+year (or no-year title boundary) plus season +# tokens. Refuse packs that also name a season we are not deleting +# (S01.S02 / S01-03 / Season 1-3). No season token → skip (under-delete). qb_match_season_jq() { - jq -r --arg folder "$1" --argjson seasons "$2" ' - def norm: - ascii_downcase - | gsub("\\\\"; "/") - | gsub("[._]+"; " ") - | gsub(" +"; " ") - | gsub("^ +| +$"; ""); - def basename_norm: - gsub("/+$"; "") | split("/") | map(select(length > 0)) | .[-1] // "" | norm; - def boundary_ok($t; $tok): - ($tok | length) as $m - | ($t | length) as $n - | if $n < $m then false - elif $n == $m then true - else ($t[$m:] | test("^($| \\([0-9]{4}\\)| \\[| - | /|\\.[0-9]{4}| s[0-9]| season |[-.]s[0-9])")) - end; - def folder_hit($text; $tok): - ($text | norm) as $t - | ($tok | norm) as $k - | if ($k | length) < 3 then false - else - ($t | basename_norm) as $base - | ( - ($base == $k) - or ($t == $k) - or ($t | endswith("/" + $k)) - or ($t | contains("/" + $k + "/")) - or (($base | startswith($k)) and boundary_ok($base; $k)) - or (($t | startswith($k)) and boundary_ok($t; $k)) - ) - end; + jq -r --arg folder "$1" --argjson seasons "$2" "${QB_TITLE_YEAR_JQ}"' def expand_range($a; $b): if $a <= $b then [range($a; $b + 1)] else [range($b; $a + 1)] end; def seasons_in($text): @@ -320,65 +335,25 @@ qb_match_season_jq() { and (any($found[]; in_wanted(.))) and (all($found[]; in_wanted(.))) and ( - folder_hit($row.name // ""; $folder) - or folder_hit($row.content_path // ""; $folder) + title_year_hit($row.name // ""; $folder) + or title_year_hit($row.content_path // ""; $folder) ) ) | .hash ' } -# Keep history hashes that (a) still exist in qB and (b) match folder key when known. -# stdin: qB info JSON. args: folder_key, then candidate hashes. +# Keep *arr queue/history hashes that still exist in qB. The hash is identity — +# do not also require a folder-key name match (release names rarely look like +# "Title (YYYY)"). stdin: qB info JSON. args: candidate hashes. qbit_filter_hashes_jq() { - local folder_key="$1" - shift local hashes_json # Lowercase/trim only — do not strip non-hex (breaks tests and rare non-standard ids). hashes_json=$(printf '%s\n' "$@" | jq -R . | jq -s -c 'map(ascii_downcase | gsub("^ +| +$"; "")) | map(select(length > 0)) | unique') - jq -r --argjson want "$hashes_json" --arg folder "$folder_key" ' - def norm: - ascii_downcase - | gsub("\\\\"; "/") - | gsub("[._]+"; " ") - | gsub(" +"; " ") - | gsub("^ +| +$"; ""); - def basename_norm: - gsub("/+$"; "") | split("/") | map(select(length > 0)) | .[-1] // "" | norm; - def boundary_ok($t; $tok): - ($tok | length) as $m - | ($t | length) as $n - | if $n < $m then false - elif $n == $m then true - else ($t[$m:] | test("^($| \\([0-9]{4}\\)| \\[| - | /|\\.[0-9]{4}| s[0-9]| season |[-.]s[0-9])")) - end; - def folder_hit($text; $tok): - ($tok | length) == 0 or ( - ($text | norm) as $t - | ($tok | norm) as $k - | ($t | basename_norm) as $base - | ( - ($base == $k) - or ($t | endswith("/" + $k)) - or ($t | contains("/" + $k + "/")) - or (($base | startswith($k)) and boundary_ok($base; $k)) - ) - ); - # index==0 is falsy in jq — must compare to null explicitly - [ .[] | select((.hash | ascii_downcase) as $h | (($want | index($h)) != null)) ] as $live - | if ($folder | length) == 0 then - $live[] | .hash - else - ($live - | map(select( - folder_hit(.name // ""; $folder) - or folder_hit(.content_path // ""; $folder) - )) - ) as $matched - | if ($matched | length) > 0 then $matched[] | .hash - else empty - end - end + jq -r --argjson want "$hashes_json" ' + .[] + | select((.hash | ascii_downcase) as $h | (($want | index($h)) != null)) + | .hash ' } @@ -422,7 +397,8 @@ remove_orphan_torrent_dirs() { tokens_json=$(jq -nc --arg k "$folder_key" '[$k]') shopt -s nullglob - for d in "$root/tv"/* "$root/movies"/*; do + for d in "$root/tv"/* "$root/movies"/* \ + "$root/incomplete/tv"/* "$root/incomplete/movies"/*; do [ -d "$d" ] || continue base=$(basename_of "$d") case "$base" in @@ -523,6 +499,43 @@ arr_history_hashes() { fi } +# Active *arr queue download hashes (in-progress grabs). Hash is identity. +arr_queue_hashes() { + local base="$1" key="$2" kind="$3" id="$4" + local endpoint raw + if [ "$kind" = "movie" ]; then + endpoint="/api/v3/queue?pageSize=250&movieId=${id}" + else + endpoint="/api/v3/queue?pageSize=250&seriesId=${id}" + fi + raw=$(arr_get "$base" "$key" "$endpoint") + echo "$raw" | jq -r ' + if type == "array" then .[] + elif type == "object" then .records[]? + else empty end + | .downloadId // .data.torrentInfoHash // empty + ' 2>/dev/null | tr 'A-F' 'a-f' | sort -u +} + +arr_queue_hashes_seasons() { + local base="$1" key="$2" id="$3" seasons_json="$4" + local raw + raw=$(arr_get "$base" "$key" "/api/v3/queue?pageSize=250&seriesId=${id}") + echo "$raw" | jq -r --argjson seasons "$seasons_json" ' + if type == "array" then .[] + elif type == "object" then .records[]? + else empty end + | . as $r + | ( + $r.episode.seasonNumber + // $r.seasonNumber + // empty + ) as $sn + | select(($sn != null) and (($seasons | index($sn)) != null)) + | $r.downloadId // $r.data.torrentInfoHash // empty + ' 2>/dev/null | tr 'A-F' 'a-f' | sort -u +} + # Same as series history, but only grab/import rows whose episode season is in $5 (lines of ints). arr_history_hashes_seasons() { local base="$1" key="$2" id="$3" seasons_json="$4" @@ -693,51 +706,41 @@ purge_qb() { log "qB purge kind=${kind} id=${id} folder_key=${folder:-} seasons=${seasons_json:-}" if [ -n "$seasons_json" ] && [ "$seasons_json" != "[]" ]; then + while IFS= read -r h; do + [ -n "$h" ] && hlist+=("$h") + done < <(arr_queue_hashes_seasons "$base_url" "$key" "$id" "$seasons_json") while IFS= read -r h; do [ -n "$h" ] && hlist+=("$h") done < <(arr_history_hashes_seasons "$base_url" "$key" "$id" "$seasons_json") else + while IFS= read -r h; do + [ -n "$h" ] && hlist+=("$h") + done < <(arr_queue_hashes "$base_url" "$key" "$kind" "$id") while IFS= read -r h; do [ -n "$h" ] && hlist+=("$h") done < <(arr_history_hashes "$base_url" "$key" "$kind" "$id") fi + if [ "${#hlist[@]}" -gt 0 ]; then + # Unique, keep order + hlist=($(printf '%s\n' "${hlist[@]}" | awk 'NF && !seen[$0]++')) + fi if [ "${#hlist[@]}" -gt 0 ]; then - log "${kind} history candidates: ${hlist[*]}" + log "${kind} queue/history candidates: ${hlist[*]}" info=$(qbit_torrents_info) while IFS= read -r h; do [ -n "$h" ] && filtered+=("$h") - done < <(echo "$info" | qbit_filter_hashes_jq "$folder" "${hlist[@]}" 2>/dev/null || true) + done < <(echo "$info" | qbit_filter_hashes_jq "${hlist[@]}" 2>/dev/null || true) if [ "${#filtered[@]}" -gt 0 ]; then - # Season-scoped: drop hashes whose torrent names another season. - if [ -n "$seasons_json" ] && [ "$seasons_json" != "[]" ]; then - local kept=() - local season_hits - season_hits=$(echo "$info" | qb_match_season_jq "$folder" "$seasons_json" 2>/dev/null || true) - for h in "${filtered[@]}"; do - if echo "$season_hits" | grep -qx "$h"; then - kept+=("$h") - else - log "qB: drop history hash $h (other season or no season token)" - fi - done - if [ "${#kept[@]}" -gt 0 ]; then - log "qB: history hashes validated (${#kept[@]}): ${kept[*]}" - qbit_delete_hashes "${kept[@]}" - remove_orphan_torrent_dirs "$folder" "$seasons_json" - return 0 - fi - else - log "qB: history hashes validated (${#filtered[@]}): ${filtered[*]}" - qbit_delete_hashes "${filtered[@]}" - remove_orphan_torrent_dirs "$folder" "$seasons_json" - return 0 - fi + log "qB: queue/history hashes in qB (${#filtered[@]}): ${filtered[*]}" + qbit_delete_hashes "${filtered[@]}" + remove_orphan_torrent_dirs "$folder" "$seasons_json" + else + log "qB: queue/history hashes not in qB; trying title+year match" fi - log "qB: history hashes not in qB or failed folder check; trying folder key" else - log "no history hashes for ${kind} ${id}" + log "no queue/history hashes for ${kind} ${id}" fi if [ -n "$folder" ]; then @@ -747,7 +750,7 @@ purge_qb() { qbit_delete_by_folder_key "$folder" fi else - log "qB: skip token fallback (no safe folder key)" + log "qB: skip name fallback (no safe folder key)" fi } @@ -813,10 +816,10 @@ if [ "${1:-}" = "--match-self-test" ]; then fi } check_filter() { - local want="$1" name="$2" folder="$3" torrents="$4" - shift 4 + local want="$1" name="$2" torrents="$3" + shift 3 local got - got=$(echo "$torrents" | qbit_filter_hashes_jq "$folder" "$@" | tr '\n' ' ' | sed 's/[[:space:]]*$//') + got=$(echo "$torrents" | qbit_filter_hashes_jq "$@" | tr '\n' ' ' | sed 's/[[:space:]]*$//') if [ "$got" = "$want" ]; then echo " PASS: $name" else @@ -841,14 +844,33 @@ if [ "${1:-}" = "--match-self-test" ]; then check "hash_m1" "bare title does not hit same-prefix sequel" \ '["Beta Movie"]' \ '[{"hash":"hash_m1","name":"Beta Movie","content_path":"/data/torrents/movies/Beta Movie"},{"hash":"hash_m2","name":"Beta Movie Reloaded","content_path":"/data/torrents/movies/Beta Movie Reloaded"}]' - check_filter "hash_a" "history filter keeps folder-matched hash only" \ - "Alpha Film (2016)" \ - '[{"hash":"hash_a","name":"Alpha Film (2016)","content_path":"/data/torrents/movies/Alpha Film (2016)"},{"hash":"hash_b","name":"Alpha Film Sequel Extended (2022)","content_path":"/data/torrents/movies/Alpha Film Sequel Extended (2022)"}]' \ + check_filter "hash_a" "history filter keeps live hash even when name is a release" \ + '[{"hash":"hash_a","name":"The Avengers 2012 REPACK BluRay","content_path":"/data/torrents/incomplete/movies/The Avengers 2012 REPACK BluRay"},{"hash":"hash_b","name":"Other Title","content_path":"/data/torrents/movies/Other"}]' \ + hash_a + check_filter "hash_a" "history filter keeps every live candidate hash" \ + '[{"hash":"hash_a","name":"Alpha Film Sequel Extended (2022)","content_path":"/data/torrents/movies/Alpha Film Sequel Extended (2022)"}]' \ hash_a hash_b - check_filter "" "history filter drops all when none match folder" \ - "Alpha Film (2016)" \ - '[{"hash":"hash_b","name":"Alpha Film Sequel Extended (2022)","content_path":"/data/torrents/movies/Alpha Film Sequel Extended (2022)"}]' \ - hash_b + check_filter "" "history filter drops hashes not in qB" \ + '[{"hash":"hash_b","name":"Other","content_path":"/x"}]' \ + hash_a + check "hash_av" "title+year hits release name without parens" \ + '["The Avengers (2012)"]' \ + '[{"hash":"hash_av","name":"The Avengers 2012 REPACK BluRay 1080p DDP 5 1 x264-hallowed","content_path":"/data/torrents/incomplete/movies/www.UIndex.org - The Avengers 2012 REPACK BluRay 1080p DDP 5 1 x264-hallowed"}]' + check "hash_av" "title+year hits indexer-prefixed incomplete path" \ + '["The Avengers (2012)"]' \ + '[{"hash":"hash_av","name":"www.UIndex.org - The Avengers 2012 REPACK","content_path":"/data/torrents/incomplete/movies/www.UIndex.org - The Avengers 2012 REPACK"}]' + check "" "title+year does not hit Age of Ultron" \ + '["The Avengers (2012)"]' \ + '[{"hash":"hash_u","name":"Avengers Age of Ultron 2015 1080p","content_path":"/data/torrents/movies/Avengers Age of Ultron 2015 1080p"}]' + check "" "short It (2017) does not hit It Comes at Night" \ + '["It (2017)"]' \ + '[{"hash":"hash_n","name":"It Comes at Night 2017 1080p","content_path":"/data/torrents/movies/It Comes at Night 2017"}]' + check "hash_it" "It (2017) hits It 2017 release" \ + '["It (2017)"]' \ + '[{"hash":"hash_it","name":"It 2017 1080p BluRay","content_path":"/data/torrents/movies/It 2017 1080p BluRay"}]' + check "hash_bb" "article-stripped title still needs the year next" \ + '["The Avengers (2012)"]' \ + '[{"hash":"hash_bb","name":"Avengers 2012 1080p","content_path":"/data/torrents/movies/Avengers 2012 1080p"}]' check_season() { local want="$1" name="$2" folder="$3" seasons="$4" torrents="$5" local got @@ -937,7 +959,9 @@ if [ "${1:-}" = "--match-self-test" ]; then _old_root="${MEDIA_ROOT:-}" MEDIA_ROOT=$(mktemp -d) mkdir -p "$MEDIA_ROOT/torrents/tv/Loki.S02.COMPLETE.1080p.DSNP.WEB-DL" \ - "$MEDIA_ROOT/torrents/tv/Lost in Space.S01.COMPLETE" + "$MEDIA_ROOT/torrents/tv/Lost in Space.S01.COMPLETE" \ + "$MEDIA_ROOT/torrents/incomplete/movies/www.UIndex.org - The Avengers 2012 REPACK BluRay" \ + "$MEDIA_ROOT/torrents/movies/Avengers Age of Ultron 2015 1080p" DRY_RUN=0 remove_orphan_torrent_dirs "Loki" if [ ! -d "$MEDIA_ROOT/torrents/tv/Loki.S02.COMPLETE.1080p.DSNP.WEB-DL" ]; then @@ -952,6 +976,19 @@ if [ "${1:-}" = "--match-self-test" ]; then echo " FAIL: Lost in Space leftover was removed" failures=$((failures + 1)) fi + remove_orphan_torrent_dirs "The Avengers (2012)" + if [ ! -d "$MEDIA_ROOT/torrents/incomplete/movies/www.UIndex.org - The Avengers 2012 REPACK BluRay" ]; then + echo " PASS: Avengers incomplete leftover removed" + else + echo " FAIL: Avengers incomplete leftover still present" + failures=$((failures + 1)) + fi + if [ -d "$MEDIA_ROOT/torrents/movies/Avengers Age of Ultron 2015 1080p" ]; then + echo " PASS: Age of Ultron leftover left alone" + else + echo " FAIL: Age of Ultron leftover was removed" + failures=$((failures + 1)) + fi rm -rf "$MEDIA_ROOT" MEDIA_ROOT="${_old_root}" diff --git a/scripts/seerr-gateway.py b/scripts/seerr-gateway.py index f728a11..2b2c74e 100755 --- a/scripts/seerr-gateway.py +++ b/scripts/seerr-gateway.py @@ -5,11 +5,13 @@ seerr-gateway — reverse proxy for Seerr with delete-request full purge. Stock Seerr DELETE /api/v1/request/:id only removes the request row. assemblrr intercepts that call (when enabled): - * Movies (and TV requests that cover every remaining season): DELETE + * Movies (and TV requests that cover every remaining season): drain the + *arr download queue (removeFromClient) first, then DELETE /api/v1/media/:id/file → Radarr/Sonarr deleteFiles + purge hooks. - * TV requests for a subset of seasons: delete only those seasons' episode - files via Sonarr and unmonitor them. Other seasons stay. The title-level - Seerr file-delete is skipped so Sonarr does not wipe the series folder. + * TV requests for a subset of seasons: drain only those seasons' queue + items, then delete those seasons' episode files via Sonarr and unmonitor + them. Other seasons stay. The title-level Seerr file-delete is skipped + so Sonarr does not wipe the series folder. Then the original request delete is forwarded. @@ -55,6 +57,9 @@ REQUEST_DELETE_RE = re.compile(r"^/api/v1/request/(\d+)/?$") SONARR_URL = os.environ.get("SONARR_URL", "http://sonarr:8989").rstrip("/") SONARR_API_KEY = os.environ.get("SONARR_API_KEY", "").strip() SONARR_CONFIG = os.environ.get("SONARR_CONFIG", "/config/sonarr/config.xml") +RADARR_URL = os.environ.get("RADARR_URL", "http://radarr:7878").rstrip("/") +RADARR_API_KEY = os.environ.get("RADARR_API_KEY", "").strip() +RADARR_CONFIG = os.environ.get("RADARR_CONFIG", "/config/radarr/config.xml") HOP_BY_HOP = { "connection", @@ -265,28 +270,37 @@ def seasons_from_request(payload: dict) -> List[int]: return uniq -def discover_sonarr_api_key() -> str: - if SONARR_API_KEY: - return SONARR_API_KEY - path = SONARR_CONFIG - if not path or not os.path.isfile(path): +def discover_arr_api_key(explicit: str, config_path: str) -> str: + if explicit: + return explicit + if not config_path or not os.path.isfile(config_path): return "" try: - text = open(path, encoding="utf-8").read() + text = open(config_path, encoding="utf-8").read() except OSError: return "" m = re.search(r"([^<]+)", text) return m.group(1).strip() if m else "" -def _sonarr_request( +def discover_sonarr_api_key() -> str: + return discover_arr_api_key(SONARR_API_KEY, SONARR_CONFIG) + + +def discover_radarr_api_key() -> str: + return discover_arr_api_key(RADARR_API_KEY, RADARR_CONFIG) + + +def _arr_request( + base_url: str, method: str, path: str, api_key: str, body: Optional[dict] = None, timeout: float = 20, + label: str = "arr", ) -> Tuple[int, object]: - url = f"{SONARR_URL}{path}" + url = f"{base_url}{path}" data = None if body is None else json.dumps(body).encode("utf-8") req = urllib.request.Request(url, data=data, method=method) req.add_header("X-Api-Key", api_key) @@ -312,7 +326,177 @@ def _sonarr_request( parsed = raw return e.code, parsed except (urllib.error.URLError, TimeoutError, socket.timeout, OSError) as e: - raise RuntimeError(f"sonarr {method} {path}: {e}") from e + raise RuntimeError(f"{label} {method} {path}: {e}") from e + + +def _sonarr_request( + method: str, + path: str, + api_key: str, + body: Optional[dict] = None, + timeout: float = 20, +) -> Tuple[int, object]: + return _arr_request(SONARR_URL, method, path, api_key, body, timeout, "sonarr") + + +def _radarr_request( + method: str, + path: str, + api_key: str, + body: Optional[dict] = None, + timeout: float = 20, +) -> Tuple[int, object]: + return _arr_request(RADARR_URL, method, path, api_key, body, timeout, "radarr") + + +def request_media(payload: dict) -> dict: + media = payload.get("media") or payload.get("mediaInfo") or {} + return media if isinstance(media, dict) else {} + + +def arr_external_id(payload: dict, is4k: bool) -> Optional[int]: + media = request_media(payload) + raw = media.get("externalServiceId4k") if is4k else media.get("externalServiceId") + if raw is None: + return None + try: + return int(raw) + except (TypeError, ValueError): + return None + + +def radarr_lookup_movie_id(payload: dict, api_key: str) -> Optional[int]: + ext = arr_external_id(payload, bool(payload.get("is4k", False))) + if ext is not None: + return ext + tmdb = request_media(payload).get("tmdbId") + if tmdb is None: + return None + try: + tmdb_i = int(tmdb) + except (TypeError, ValueError): + return None + status, movies = _radarr_request("GET", "/api/v3/movie", api_key) + if status != 200 or not isinstance(movies, list): + return None + for row in movies: + if isinstance(row, dict) and row.get("tmdbId") == tmdb_i: + try: + return int(row["id"]) + except (TypeError, ValueError, KeyError): + return None + return None + + +def queue_records(kind: str, arr_id: int, api_key: str) -> List[dict]: + if kind == "movie": + status, data = _radarr_request( + "GET", f"/api/v3/queue?pageSize=250&movieId={arr_id}", api_key + ) + else: + status, data = _sonarr_request( + "GET", f"/api/v3/queue?pageSize=250&seriesId={arr_id}", api_key + ) + if status != 200: + raise RuntimeError(f"{kind} queue GET HTTP {status}") + if isinstance(data, list): + rows = data + elif isinstance(data, dict): + rows = data.get("records") or [] + else: + rows = [] + return [r for r in rows if isinstance(r, dict)] + + +def queue_record_season(row: dict) -> Optional[int]: + if row.get("seasonNumber") is not None: + try: + return int(row["seasonNumber"]) + except (TypeError, ValueError): + pass + ep = row.get("episode") + if isinstance(ep, dict) and ep.get("seasonNumber") is not None: + try: + return int(ep["seasonNumber"]) + except (TypeError, ValueError): + return None + return None + + +def drain_arr_queue( + kind: str, + arr_id: int, + api_key: str, + seasons: Optional[List[int]] = None, +) -> Tuple[int, int]: + """ + Remove *arr queue items and the matching download-client torrents. + Returns (considered, deleted). Best-effort: HTTP failures on a single + item are logged and skipped. + """ + rows = queue_records(kind, arr_id, api_key) + wanted = set(seasons) if seasons else None + deleted = 0 + considered = 0 + req = _radarr_request if kind == "movie" else _sonarr_request + for row in rows: + if wanted is not None: + sn = queue_record_season(row) + if sn is None or sn not in wanted: + continue + qid = row.get("id") + if qid is None: + continue + considered += 1 + status, _ = req( + "DELETE", + f"/api/v3/queue/{qid}?removeFromClient=true&blocklist=false&skipRedownload=true", + api_key, + ) + if status in (200, 204): + deleted += 1 + else: + log.warning("%s queue delete id=%s HTTP %s", kind, qid, status) + return considered, deleted + + +def _drain_queue_note( + kind: str, + arr_id: int, + api_key: str, + seasons: Optional[List[int]], + request_id: str, +) -> str: + try: + considered, deleted = drain_arr_queue(kind, arr_id, api_key, seasons) + except Exception as exc: + log.warning("request %s: %s queue drain failed: %s", request_id, kind, exc) + return f"queue_{kind}_failed" + log.info( + "request %s: %s queue drain id=%s considered=%s deleted=%s", + request_id, + kind, + arr_id, + considered, + deleted, + ) + return f"queue_{kind}:{considered}:{deleted}" + + +def _drain_movie_queue_note(payload: dict, request_id: str) -> str: + api_key = discover_radarr_api_key() + if not api_key: + log.info("request %s: no Radarr API key; skip queue drain", request_id) + return "queue_movie_skipped" + try: + movie_id = radarr_lookup_movie_id(payload, api_key) + except Exception as exc: + log.warning("request %s: radarr movie lookup failed: %s", request_id, exc) + return "queue_movie_failed" + if movie_id is None: + log.info("request %s: movie not in Radarr; skip queue drain", request_id) + return "queue_movie_missing" + return _drain_queue_note("movie", movie_id, api_key, None, request_id) def sonarr_lookup_series_id(payload: dict, api_key: str) -> Optional[int]: @@ -500,11 +684,12 @@ def purge_then_delete_request( """ Cascade: GET request + drain *arr queue (removeFromClient) while the title still exists movie / TV covering every remaining season → DELETE media/:id/file TV subset of seasons → Sonarr season file delete only DELETE request Auth failure on GET, or any TV-scope / Sonarr uncertainty, returns without - deleting the request (no title-level wipe). + deleting the request (no title-level wipe). Queue drain is best-effort. Returns (status, headers, body, purge_note). """ get_status, _, get_body = proxy_once("GET", f"/api/v1/request/{request_id}", headers, None) @@ -540,9 +725,12 @@ def purge_then_delete_request( else: remaining = sonarr_remaining_seasons(series_id, api_key) if remaining and not seasons_cover_remaining(requested, remaining): + qnote = _drain_queue_note( + "series", series_id, api_key, requested, request_id + ) deleted = sonarr_delete_seasons(series_id, requested, api_key) purge_note = ( - f"seasons_deleted:{media_id}:{series_id}:" + f"{qnote};seasons_deleted:{media_id}:{series_id}:" f"{','.join(str(s) for s in requested)}:{deleted}" ) log.info( @@ -559,12 +747,18 @@ def purge_then_delete_request( requested, remaining, ) + qnote = _drain_queue_note( + "series", series_id, api_key, None, request_id + ) purge_note, _ = _delete_media_file(media_id, is4k, headers) + purge_note = f"{qnote};{purge_note}" except Exception as exc: log.error("request %s: season purge failed: %s", request_id, exc) return _season_purge_fail(str(exc), "season_purge_failed") else: + qnote = _drain_movie_queue_note(payload, request_id) purge_note, _ = _delete_media_file(media_id, is4k, headers) + purge_note = f"{qnote};{purge_note}" if qnote else purge_note else: log.info("request %s: no linked media id; deleting request only", request_id) @@ -839,6 +1033,7 @@ class _MockSonarr(BaseHTTPRequestHandler): calls: List[str] = [] files: List[dict] = [] series: dict = {} + queue: List[dict] = [] episodefile_status: int = 200 delete_episodefile_status: int = 200 @@ -875,6 +1070,9 @@ class _MockSonarr(BaseHTTPRequestHandler): ] self._send(200, json.dumps(eps).encode()) return + if path == "/api/v3/queue": + self._send(200, json.dumps({"records": list(_MockSonarr.queue)}).encode()) + return self._send(404, b"{}") def do_DELETE(self) -> None: # noqa: N802 @@ -889,6 +1087,12 @@ class _MockSonarr(BaseHTTPRequestHandler): _MockSonarr.files = [f for f in _MockSonarr.files if f.get("id") != fid] self._send(200, b"") return + m = re.match(r"^/api/v3/queue/(\d+)$", path) + if m: + qid = int(m.group(1)) + _MockSonarr.queue = [q for q in _MockSonarr.queue if q.get("id") != qid] + self._send(200, b"") + return self._send(404, b"{}") def do_PUT(self) -> None: # noqa: N802 @@ -917,6 +1121,72 @@ def _reset_mock_sonarr() -> None: {"seasonNumber": 2, "monitored": True}, ], } + _MockSonarr.queue = [ + { + "id": 201, + "seriesId": 10, + "downloadId": "aaa", + "episode": {"seasonNumber": 1}, + }, + { + "id": 202, + "seriesId": 10, + "downloadId": "bbb", + "episode": {"seasonNumber": 2}, + }, + ] + + +class _MockRadarr(BaseHTTPRequestHandler): + """Radarr stand-in: movie 1 with one queue item unless deleted.""" + + protocol_version = "HTTP/1.1" + calls: List[str] = [] + movies: List[dict] = [] + queue: List[dict] = [] + + def log_message(self, fmt: str, *args) -> None: + return + + def _send(self, status: int, body: bytes = b"") -> None: + self.send_response(status) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + if self.command != "HEAD" and body: + self.wfile.write(body) + + def do_GET(self) -> None: # noqa: N802 + path = self.path.split("?", 1)[0] + _MockRadarr.calls.append(f"GET {self.path}") + if path == "/api/v3/movie": + self._send(200, json.dumps(list(_MockRadarr.movies)).encode()) + return + if path == "/api/v3/queue": + self._send(200, json.dumps({"records": list(_MockRadarr.queue)}).encode()) + return + self._send(404, b"{}") + + def do_DELETE(self) -> None: # noqa: N802 + path = self.path.split("?", 1)[0] + _MockRadarr.calls.append(f"DELETE {path}") + m = re.match(r"^/api/v3/queue/(\d+)$", path) + if m: + qid = int(m.group(1)) + _MockRadarr.queue = [q for q in _MockRadarr.queue if q.get("id") != qid] + self._send(200, b"") + return + self._send(404, b"{}") + + +def _reset_mock_radarr() -> None: + _MockRadarr.calls = [] + _MockRadarr.movies = [ + {"id": 1, "tmdbId": 10378, "title": "Big Buck Bunny"}, + ] + _MockRadarr.queue = [ + {"id": 99, "movieId": 1, "downloadId": "abc123"}, + ] def _free_port() -> int: @@ -940,13 +1210,19 @@ def self_test() -> int: mock_port = _free_port() sonarr_port = _free_port() + radarr_port = _free_port() gw_port = _free_port() _MockSeerr.calls = [] _MockSeerr.request_payload = { "id": 42, "is4k": False, - "media": {"id": 7, "tmdbId": 10378, "mediaType": "movie"}, + "media": { + "id": 7, + "tmdbId": 10378, + "mediaType": "movie", + "externalServiceId": 1, + }, } _MockSeerr.file_delete_status = 204 _MockSeerr.request_delete_status = 204 @@ -961,16 +1237,25 @@ def self_test() -> int: sonarr_httpd.daemon_threads = True threading.Thread(target=sonarr_httpd.serve_forever, daemon=True).start() + _reset_mock_radarr() + radarr_httpd = ThreadingHTTPServer(("127.0.0.1", radarr_port), _MockRadarr) + radarr_httpd.daemon_threads = True + threading.Thread(target=radarr_httpd.serve_forever, daemon=True).start() + os.environ["SEERR_UPSTREAM"] = f"http://127.0.0.1:{mock_port}" os.environ["SEERR_DELETE_REQUEST_PURGE"] = "1" # Re-bind module globals used by handlers - global UPSTREAM, PURGE_ON_DELETE_REQUEST, LISTEN_HOST, LISTEN_PORT, SONARR_URL, SONARR_API_KEY, SONARR_CONFIG + global UPSTREAM, PURGE_ON_DELETE_REQUEST, LISTEN_HOST, LISTEN_PORT + global SONARR_URL, SONARR_API_KEY, SONARR_CONFIG + global RADARR_URL, RADARR_API_KEY, RADARR_CONFIG UPSTREAM = os.environ["SEERR_UPSTREAM"].rstrip("/") PURGE_ON_DELETE_REQUEST = True LISTEN_HOST = "127.0.0.1" LISTEN_PORT = gw_port SONARR_URL = f"http://127.0.0.1:{sonarr_port}" SONARR_API_KEY = "sonarr-test-key" + RADARR_URL = f"http://127.0.0.1:{radarr_port}" + RADARR_API_KEY = "radarr-test-key" gw = ThreadingHTTPServer((LISTEN_HOST, LISTEN_PORT), GatewayHandler) gw.daemon_threads = True @@ -1011,6 +1296,19 @@ def self_test() -> int: "DELETE media file before request delete", ) check(any(c.startswith("DELETE /api/v1/request/42") for c in calls), "DELETE request last") + radarr_calls = list(_MockRadarr.calls) + check( + any(c.startswith("GET /api/v3/queue") for c in radarr_calls), + "movie queue listed before title delete", + ) + check( + any(c.startswith("DELETE /api/v3/queue/99") for c in radarr_calls), + "movie queue item removed from download client", + ) + check( + "queue_movie:1:1" in (hdrs.get("X-Assemblrr-Purge-Detail") or ""), + f"purge detail notes queue drain ({hdrs.get('X-Assemblrr-Purge-Detail')})", + ) # Order: GET, media file, request delete try: i_get = next(i for i, c in enumerate(calls) if c.startswith("GET /api/v1/request/42")) @@ -1137,6 +1435,14 @@ def self_test() -> int: not any(c == "DELETE /api/v3/episodefile/2" for c in _MockSonarr.calls), "TV S01 does not delete Sonarr S02 episode file", ) + check( + any(c == "DELETE /api/v3/queue/201" for c in _MockSonarr.calls), + "TV S01 removes S01 queue item from download client", + ) + check( + not any(c == "DELETE /api/v3/queue/202" for c in _MockSonarr.calls), + "TV S01 leaves S02 queue item", + ) check( "seasons_deleted" in (hdrs.get("X-Assemblrr-Purge-Detail") or ""), f"purge detail is season-scoped ({hdrs.get('X-Assemblrr-Purge-Detail')})", @@ -1273,6 +1579,11 @@ def self_test() -> int: not any(c.startswith("DELETE /api/v3/episodefile/") for c in _MockSonarr.calls), "TV all-seasons does not piecemeal-delete episode files", ) + check( + any(c == "DELETE /api/v3/queue/201" for c in _MockSonarr.calls) + and any(c == "DELETE /api/v3/queue/202" for c in _MockSonarr.calls), + "TV all-seasons drains every series queue item", + ) tv_s01 = { "id": 8, @@ -1366,6 +1677,7 @@ def self_test() -> int: gw.shutdown() mock_httpd.shutdown() sonarr_httpd.shutdown() + radarr_httpd.shutdown() print("") if failures: diff --git a/tests/integration/media_purge_e2e.sh b/tests/integration/media_purge_e2e.sh index dc5d81b..1eb1d24 100755 --- a/tests/integration/media_purge_e2e.sh +++ b/tests/integration/media_purge_e2e.sh @@ -20,7 +20,7 @@ source "$SCRIPT_DIR/../helpers.sh" TMDB_ID=10378 TITLE="Big Buck Bunny" TITLE_YEAR="Big Buck Bunny (2008)" -TORRENT_DIR_NAME="Big Buck Bunny (2008) [e2e]" +TORRENT_DIR_NAME="Big.Buck.Bunny.2008.1080p.WEB-DL.e2e" COOKIE_JAR="${TMPDIR:-/tmp}/assemblrr_media_purge_e2e_qb.cookies" FAILED=0 diff --git a/tests/unit/test_seerr_gateway.sh b/tests/unit/test_seerr_gateway.sh index 6bfb32d..fba32b7 100755 --- a/tests/unit/test_seerr_gateway.sh +++ b/tests/unit/test_seerr_gateway.sh @@ -35,7 +35,9 @@ gw_block=$(awk '/^ seerr-gateway:/{p=1} p&&/^ [a-z]/{if(!/^ seerr-gateway:/)e assert_contains "$gw_block" "5055:5055" "seerr-gateway publishes 5055" assert_contains "$gw_block" "SEERR_DELETE_REQUEST_PURGE" "purge flag wired in compose" assert_contains "$gw_block" "SONARR_URL" "gateway has Sonarr URL for TV season purge" +assert_contains "$gw_block" "RADARR_URL" "gateway has Radarr URL for movie queue drain" assert_contains "$gw_block" "config/sonarr:/config/sonarr" "gateway mounts Sonarr config dir" +assert_contains "$gw_block" "config/radarr:/config/radarr" "gateway mounts Radarr config dir" assert_not_contains "$gw_block" "config.xml:/config/sonarr/config.xml" "gateway does not file-mount config.xml" # Catalog: Seerr UI is the gateway