SQLite on Local Path¶

Apps that store their config in SQLite — Emby, Jellyfin, Kavita, Bazarr — run poorly with config on NFS. SQLite wasn't designed for network filesystems: every read and write crosses the network, which shows up as sluggish UI and slow page loads.

The fix is to move the config PVC to local-path (node-local storage) and back it up to NFS hourly via a CronJob.

graph LR
    App -->|fast local I/O| LocalPVC[local-path PVC\nnode-local]
    CronJob[Hourly CronJob] -->|rsync --delete| LocalPVC
    CronJob --> BackupPVC[NFS backup PVC\nSynology]

Hold "Alt" / "Option" to enable pan & zoom

Trade-offs

The app must be pinned to a specific node with nodeSelector — local-path PVCs have node affinity and cannot move.
The backup CronJob must use the same nodeSelector to reach the PVC.
RPO = 1 hour. If the node fails, restore from NFS backup onto a new local-path PVC on another node (manual step).

PVCs¶

Two PVCs are needed: the local-path primary and an NFS-backed backup target.

pvcs.yaml

# Primary config — local-path, no PV object needed
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: <app>-config
  namespace: <ns>
spec:
  storageClassName: local-path
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: <size>
---
# Backup target — static PV required for NFS CSI
kind: PersistentVolume
apiVersion: v1
metadata:
  name: <app>-config-backup
spec:
  capacity:
    storage: <size>
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: syno-nfs-retain
  csi:
    driver: nfs.csi.k8s.io
    volumeHandle: <app>-config-backup
    volumeAttributes:
      server: "<nas-ip>"
      share: "/volume2/homelab/k8s/pvc-<app>-config-backup"
  mountOptions:
  - nfsvers=4.1
  - hard
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: <app>-config-backup
  namespace: <ns>
spec:
  storageClassName: syno-nfs-retain
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: <size>
  volumeName: <app>-config-backup

Before applying

Create the NFS directory on Synology first — the backup PVC will stay Pending until it exists:

mkdir -p /volume2/homelab/k8s/pvc-<app>-config-backup

Backup CronJob¶

backup-cronjob.yaml

apiVersion: batch/v1
kind: CronJob
metadata:
  name: <app>-config-backup
  namespace: <ns>
spec:
  schedule: "0 * * * *"
  concurrencyPolicy: Forbid
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 3
  jobTemplate:
    spec:
      template:
        spec:
          nodeSelector:
            kubernetes.io/hostname: <same-node-as-app>
          restartPolicy: OnFailure
          containers:
          - name: rsync
            image: alpine:3.21
            command:
            - sh
            - -c
            - |
              apk add --no-cache rsync su-exec && \
              su-exec <uid>:<gid> rsync -av --delete /config/ /backup/
            volumeMounts:
            - name: config
              mountPath: /config
              readOnly: true
            - name: backup
              mountPath: /backup
          volumes:
          - name: config
            persistentVolumeClaim:
              claimName: <app>-config
          - name: backup
            persistentVolumeClaim:
              claimName: <app>-config-backup

The CronJob runs rsync as the app's UID:GID via su-exec — NFS squashes root writes to nobody, so rsync must run as the file owner to write successfully. --delete keeps the backup an exact mirror.

Migration (one-time)¶

Use this when converting an existing app from an NFS config PVC to local-path.

Create the NFS backup directory on Synology:

mkdir -p /volume2/homelab/k8s/pvc-<app>-config-backup

Scale down the app:

kubectl scale deploy <app> -n <ns> --replicas=0

Delete the old NFS PVC and PV — data is safe, the NFS PV uses Retain:

kubectl delete pvc <app>-config -n <ns>
kubectl delete pv <app>-config

Apply the new pvcs.yaml via Flux and wait for both PVCs to reach Bound.

Copy data from the old NFS share into the new local-path PVC. Use a CSI inline volume — plain nfs volume type fails on Talos (no nfs-common kernel modules):

kubectl run <app>-migrate --rm -it --restart=Never -n <ns> \
  --image=alpine \
  --overrides='{
    "spec": {
      "nodeSelector": {"kubernetes.io/hostname": "<target-node>"},
      "securityContext": {
        "runAsUser": <uid>,
        "runAsGroup": <gid>,
        "runAsNonRoot": true,
        "seccompProfile": {"type": "RuntimeDefault"}
      },
      "volumes": [
        {
          "name": "old",
          "csi": {
            "driver": "nfs.csi.k8s.io",
            "volumeAttributes": {
              "server": "<nas-ip>",
              "share": "/volume2/homelab/k8s/pvc-<app>-config"
            }
          }
        },
        {
          "name": "new",
          "persistentVolumeClaim": {"claimName": "<app>-config"}
        }
      ],
      "containers": [{
        "name": "migrate",
        "image": "alpine",
        "command": ["sh", "-c", "apk add --no-cache rsync && rsync -av --progress /old/ /new/ && echo DONE"],
        "securityContext": {
          "allowPrivilegeEscalation": false,
          "capabilities": {"drop": ["ALL"]}
        },
        "volumeMounts": [
          {"name": "old", "mountPath": "/old", "readOnly": true},
          {"name": "new", "mountPath": "/new"}
        ]
      }]
    }
  }'

Scale the app back up and verify it starts healthy.

Trigger a manual backup to confirm the CronJob works end-to-end:

kubectl create job --from=cronjob/<app>-config-backup <app>-config-backup-manual -n <ns>

Troubleshooting¶

Symptom	Cause	Fix
`apk: Permission denied`	Pod securityContext sets non-root UID, blocking apk	Use `su-exec` — let the container start as root, drop for rsync only
`chown: Operation not permitted`	rsync running as root; NFS squashes root to `nobody`	Run rsync via `su-exec <uid>:<gid>`
`rsync: chgrp failed: Operation not permitted`	Backup NFS share owned by wrong group	On the NAS, set the owner of the backup share directory to `docker` via File Station
Migration pod times out with `-it`	Image pull too slow for interactive attach	Drop `-it`, run in background and tail logs separately
Migration pod can't mount local-path PVC	Pod scheduled on wrong node	Add `nodeSelector` matching the node where the PVC was provisioned
Backup PVC stays `Pending`	NFS share doesn't exist on Synology	Create the directory on Synology before applying the PVC
Plain `nfs` volume type fails on Talos	Talos nodes lack `nfs-common` kernel modules	Use CSI inline volume with `driver: nfs.csi.k8s.io`