I have a plex setup on Ubuntu with Deluge, Jackett, *arr apps, etc. All is gravy.
Except I currently need to manually run rclone copyto
on completed Deluge torrents in order to [reliably] pipe the media to rclone/Plex.
Context:
- Deluge downloads to
~/local/
- rclone is mounted to
~/gdrive/
using:
rclone mount gdrive: ~/gdrive --allow-other --vfs-cache-mode off --bwlimit 15M --tpslimit 4 --tpslimit-burst 4
- Plex serves from
~/gdrive/Media/TV
and~/gdrive/Media/Movies
Initially I was using Deluge Labels to auto-move completed downloads, but this resulted in IO errors (and occasionally illegal disk seeks when I messed with enabling vfs cache writes).
I tried setting up a cron job to monitor ~/local/
for completed files then copying to rclone, but that resulted in IO errors.
The ONLY reliable solution I’ve found is manually running rclone copyto
from a screen
session for completed downloads. It works but it’s a pain in the ass.
My questions for yall:
-
Any idea why I’m getting these IO errors? I’m not pegging the CPU or the disk. There doesn’t seem to be hardware issue, it really feels like the problem relates to rclone and Google’s APIs
-
Any idea how to troubleshoot this and get downloads automatically moving? Any logging mechanism I use (Deluge, rcloone, kernel) just gives generic IO errors. It doesn’t point me in any direction
I ran into the same problem when first setting up a similar program stack. The problem I found was that some programs need to have the rclone mount setup before they first attempt to access the local folders. Just allowing the programs to start randomly would typically result in I/O errors.
My solution was to use the healthcheck feature of docker-compose to ensure that the mount was accessible before anything downstream booted. I placed a small “test” file in the mount and wait until it’s available. I use bash, which means the rclone container needs to have it installed. Below is the dockerfile I use for rclone and my docker-compose file that controls the boot-up sequence.
One thing to note: only docker-compose uses the depends-on argument, so when your server boots-up the default is that docker starts the containers at random. You can get around this by turning off the auto-start using docker and then setting a cron-job that runs the docker-compose up commands at start-up.
Rclone Dockerfile FROM rclone/rclone:latest RUN apk update &&
apk add bashdocker-compose.yml version: ‘3.5’
services: ## rclone rclone-plex-media: build: ./rclone/ container_name: rclone-plex-media environment: - RCLONE_CONFIG=/config/rclone.conf cap_add: - SYS_ADMIN privileged: true devices: - /dev/fuse restart: unless-stopped volumes: - ./rclone/plex-media:/mnt/docker/media/rclone/plex-media-internal:shared - ./rclone/plex-cache:/cache - ./rclone/plex-config:/config command: mount "plex-encrypted:" "/mnt/docker/media/rclone/plex-media-internal" --uid 1000 --gid 1000 --umask 002 --default-permissions --allow-non-empty --allow-other --vfs-cache-mode full --cache-dir "/cache/" --vfs-cache-max-size 50G --dir-cache-time 1000h --vfs-cache-max-age 9999h --vfs-write-back 24h --log-level INFO --poll-interval=15s --stats 1m healthcheck: test: bash -c "[ -f /mnt/docker/media/rclone/plex-media-internal/test.txt ]" interval: 30s retries: 3 start_period: 30s timeout: 5s ## Streaming - Plex plex: image: plexinc/pms-docker container_name: plex environment: - TZ=America/New_York devices: - /dev/dri:/dev/dri restart: unless-stopped network_mode: host volumes: - ./plex/config:/config - ./plex/transcode:/transcode - ./rclone/plex-media:/data depends_on: rclone-plex-media: condition: service_healthy ## Plex Scan Trigger - Autoscan autoscan: image: cloudb0x/autoscan container_name: autoscan environment: - PUID=1000 - PGID=1000 volumes: - ./autoscan/config:/config - ./rclone/plex-media:/mnt/unionfs:ro ports: - 3030:3030 restart: unless-stopped depends_on: plex: condition: service_healthy ## Media Management - Deluge deluge: image: binhex/arch-delugevpn container_name: deluge privileged: true sysctls: - net.ipv4.conf.all.src_valid_mark=1 volumes: - ./deluge/downloads:/downloads - ./deluge/config:/config - /etc/localtime:/etc/localtime:ro environment: - VPN_ENABLED=yes - VPN_USER=REMOVED - VPN_PASS=REMOVED - VPN_PROV=REMOVED - VPN_CLIENT=wireguard - STRICT_PORT_FORWARD=yes - ENABLE_PRIVOXY=yes - LAN_NETWORK=192.168.2.0/24 - NAME_SERVERS=REMOVED - DELUGE_DAEMON_LOG_LEVEL=info - DELUGE_WEB_LOG_LEVEL=info - VPN_INPUT_PORTS=1234 - VPN_OUTPUT_PORTS=5678 - DEBUG=false - UMASK=000 - PUID=1000 - PGID=1000 ports: - 8112:8112 - 8118:8118 - 58846:58846 - 58946:58946 healthcheck: test: "curl -f http://localhost:8112 || exit 1" interval: 10s retries: 3 start_period: 600s timeout: 5s restart: unless-stopped ## Media Management - Prowlarr prowlarr: image: lscr.io/linuxserver/prowlarr:develop container_name: prowlarr environment: - PUID=1000 - PGID=1000 - TZ=America/New_York volumes: - ./prowlarr/config:/config ports: - 9696:9696 restart: unless-stopped healthcheck: test: "curl -f http://localhost:9696 || exit 1" interval: 10s retries: 6 start_period: 5s timeout: 5s depends_on: rclone-plex-media: condition: service_healthy deluge: condition: service_healthy ## Media Management - Radarr radarr: image: lscr.io/linuxserver/radarr:latest container_name: radarr environment: - PUID=1000 - PGID=1000 - TZ=America/New_York volumes: - ./radarr/config:/config - ./rclone/plex-media/Movies:/movies - ./deluge/downloads:/downloads ports: - 7878:7878 restart: unless-stopped healthcheck: test: "curl -f http://localhost:7878 || exit 1" interval: 10s retries: 6 start_period: 5s timeout: 5s depends_on: rclone-plex-media: condition: service_healthy prowlarr: condition: service_healthy ## Media Management - Sonarr sonarr: image: lscr.io/linuxserver/sonarr:latest container_name: sonarr environment: - PUID=1000 - PGID=1000 - TZ=America/New_York volumes: - ./sonarr/config:/config - './rclone/plex-media/TV Shows:/tv' - ./deluge/downloads:/downloads ports: - 8989:8989 restart: unless-stopped healthcheck: test: "curl -f http://localhost:8989 || exit 1" interval: 10s retries: 6 start_period: 5s timeout: 5s depends_on: rclone-plex-media: condition: service_healthy prowlarr: condition: service_healthy
One more thing. I’d highly recommend using vfs-cache for rclone since it takes care of moving the files to your remote source. You can see in my setup rclone will keep a local copy for 24 hrs and then automatically transfer it to the remote.
I was originally using an external script to copy/move files but vfs-cache is so much easier. Just let rclone handle everything in the background.