rosbackup-ng

Next Generation RouterOS Device Fleet Management & Backup

⚠️ Under active development — feature-incomplete until v1.0.
rosbackup-ng is usable today, but behavior, configuration, and APIs may change between releases until v1.0 arrives.

---

Automated, resilient backup utility for MikroTik RouterOS devices. It creates, downloads, and stores both binary (.backup) and plaintext (.rsc) backups — in parallel, with encryption, tmpfs staging, tag/group targeting, pluggable storage destinations, and connectivity built to survive flaky overlay links (e.g. ZeroTier). An optional self-hosted web platform adds a dashboard, HTTP API, schedules, and push-restore.

It runs as a lean CLI (rosbackup) or as a long-running web service (rosbackup-web). The web platform deploys as a Docker stack (see Deploy with Docker); pip, a single-file zipapp, and from-source installs live in doc/INSTALLATION-METHODS.md.

The optional self-hosted web platform — fleet dashboard with reachability, backup health, latency, and recent runs. See Web platform.

At a glance

rosbackup --compose-style renders a live, scale-aware view — a per-row list for small fleets, or an aggregate dashboard (progress bar, success/failure/running counts, throughput ETA, failures tail) at thousands of targets. See Compose-style output.

Highlights

• Both backup formats — binary (.backup) + plaintext (.rsc), in parallel, with encryption and tmpfs staging to spare router flash.
• Resilient connectivity — retry-within-a-window, TCP keepalives, and operation-level reconnect, built to survive flaky overlay links (e.g. ZeroTier).
• Fleet at scale — tags/groups, scoped runs (--group/--tag), and a compose-style live view that scales from 4 routers to 5,000.
• Pluggable storage — mirror to local / sftp / nextcloud (and s3 via an extra), with per-destination retention and group/tag routing.
• Optional web platform — self-hosted UI + /api/v1: dashboard, schedules, push-restore, firmware upgrades, an encrypted Vault for keys/passwords, API tokens, and an audit log.
• Secure by default — SSH key auth, self-contained host-key verification, encrypted backups, 0600 files, and log redaction.

See doc/FEATURES.md for the full (non-exhaustive) feature list.

How it fits together

One backup engine, two front-ends. The rosbackup CLI and the rosbackup-web service both drive the same console-free core (orchestrator.run_backup); the web adds the encrypted Vault, the scheduler, the UI, and the /api/v1 HTTP API on top.

        config dir · global.yaml · targets.yaml · web.db (vault, web only)
                                  │
                 ┌────────────────┴─────────────────┐
                 ▼                                   ▼
        rosbackup (CLI)                     rosbackup-web (service)
        standalone · no server              UI · /api/v1 · scheduler · Vault
        filesystem SSH key                  ◀── browser / API client (Bearer token)
                 │                                   │
                 │                                   │  inject_vault_keys():
                 │                                   │  key_secret → in-memory key
                 └────────────────┐   ┌───────────────┘
                                  ▼   ▼
        core engine · orchestrator.run_backup(config, targets) → BackupRunResult
        parallel SSH · tmpfs staging · retention
                                  │
                 ┌────────────────┴─────────────────┐
                 ▼                                   ▼
        RouterOS fleet (SSH / SCP)          storage: local · sftp · nextcloud · s3

• Shared core — both front-ends load the config and call the same run_backup(config, targets); the web is a thin adapter, not a second implementation.
• Vault is web-only — the web decrypts a target's ssh.key_secret into an in-memory key just before a run (inject_vault_keys); the standalone CLI authenticates with filesystem keys.
• HTTP API — POST /api/v1/runs triggers a run and GET /api/v1/runs/{id} polls results (Bearer-token auth), so backups can be driven programmatically.

Full internals are in doc/DESIGN_REFERENCE.md.

Documentation

• doc/FEATURES.md — Full (non-exhaustive) feature list
• doc/BOOTSTRAP.md — Preparing RouterOS devices for automated backups with the bootstrap utility
• doc/COMMAND_REFERENCE.md — Complete reference of all command-line options
• doc/CONFIG_REFERENCE.md — Detailed reference of all global.yaml / targets.yaml parameters
• doc/STORAGE.md — Storage destinations: inheritance model, routing, and monitoring
• doc/FILESYSTEM_STRUCTURE.md — Project directory structure and organization
• doc/BACKUP_STRUCTURE.md — Backup file formats, naming conventions, and info files
• doc/BACKUP-AND-RESTORE.md — What backups contain and how to restore a device (binary-only, with safety steps)
• doc/TMPFS_FEATURES.md — tmpfs staging feature documentation
• doc/WEB_PLATFORM.md — Self-hosted web UI / API (rosbackup-web), reverse-proxy TLS, and the HTTP API
• doc/NETWORKING.md — Container network profiles (bridge/NAT66 vs macvlan native address) with diagrams
• doc/DEPLOY.md — Docker Compose deploy (registry pull + local build), TLS, profiles
• doc/INSTALLATION-METHODS.md — Non-Docker installs: pip (registry), single-file zipapp, from source
• doc/WEB_STYLE_GUIDE.md — Web UI design system (tokens, components; repo-only, not served)
• doc/DESIGN_REFERENCE.md — Architecture and developer documentation
• doc/TECH_STACK.md — Tech choices and rationale for every layer of the stack
• testlab/README.md — Self-contained libvirt RouterOS test lab (dual-stack virtual routers)

Prerequisites

• Docker Engine + Compose v2 — or Python 3.9+ for the other install methods
• RouterOS devices with SSH access enabled
• An SSH key pair for the backup user (see BOOTSTRAP)
• RouterOS v7.7 or later for tmpfs staging (falls back to flash on older versions)

Deploy with Docker

The supported way to run rosbackup-ng is the Docker Compose stack under deploy/. It pulls a pre-built image from the registry (a local build is the fallback), runs rosbackup-web behind a Caddy TLS front door, and keeps your config + backups on the host. Prerequisite: Docker Engine + Compose v2.

Quickstart

git clone https://git.jdneer.com/jd/rosbackup-ng /opt/rosbackup-ng && cd /opt/rosbackup-ng/deploy
sudo ./deploy.sh up                 # first run scaffolds .env + ./config and pulls the image
$EDITOR config/global.yaml config/targets.yaml   # add your routers + SSH key (see CONFIG_REFERENCE)
sudo ./deploy.sh set-password       # set the web admin password (once)
sudo ./deploy.sh up                 # apply — idempotent, safe to re-run

Open https://<host>/ (self-signed cert by default) and log in. That's the whole setup.

TLS (optional)

sudo ./deploy.sh set-fqdn backup.example.com        # public hostname
sudo ./deploy.sh set-tls-desec <deSEC-token>        # automatic Let's Encrypt via DNS-01
# or bring your own cert:
sudo ./deploy.sh set-cert ./fullchain.pem ./key.pem

Day-to-day

sudo ./deploy.sh status     # health + container state
sudo ./deploy.sh logs       # follow logs
sudo ./deploy.sh update     # pull the latest image + restart
sudo ./deploy.sh down       # stop the stack

Network profiles (bridge/NAT66 vs macvlan native addressing), manual-cert import, and the full deploy.sh reference are in doc/DEPLOY.md and doc/NETWORKING.md.

Prefer not to use Docker? pip (from the package registry), a single-file zipapp, and from-source installs are in doc/INSTALLATION-METHODS.md.

Configuration

The CLI and the web platform read the same config dir (global.yaml + targets.yaml). A minimal global.yaml:

backup_path_parent: backups          # where backups are stored (required)
backup_password: change-me           # used when a target sets `encrypted: true` (optional)

ssh:
  user: rosbackup                    # default SSH username
  connect_retry:                     # resilient connect for flaky/overlay links (ZeroTier), on by default
    total_timeout: 180               # retry window, seconds

tmpfs:
  enabled: true                      # stage binary backups in RAM to spare router flash

A minimal targets.yaml:

targets:
  - name: ROUTER-1
    host: 192.168.88.1
    tags: [site-hq, core]                          # labels for groups / --tag (optional)
    ssh:
      private_key: ./ssh-keys/private/id_rosbackup

See doc/CONFIG_REFERENCE.md for the full reference — retention, encryption, per-target overrides, notifications, groups, storage routing, and web capability toggles.

Usage

# Back up all enabled targets
rosbackup

# A specific target only
rosbackup --target ROUTER-1

# A named group or a tag (selectors are mutually exclusive with --target)
rosbackup --group critical
rosbackup --tag site-hq

# Discover configured groups/tags with device counts
rosbackup --list

# Scale-aware compose-style output
rosbackup --compose-style

# Validate connectivity/access without writing anything
rosbackup --dry-run

Compose-style output

--compose-style (-x) renders a live view that scales from a handful of routers to thousands. Small fleets get a per-row view:

[+] Executing backup for 4 targets ...

    ✔ ROUTER1                  Finished            3.8s
    ✔ ROUTER2                  Finished            3.8s
    ⠹ ROUTER3                  Backing Up          2.1s
    ⠹ ROUTER4                  Connecting          0.6s

Summary:
    Total time: 3.9s
    Total size: 107.0KB
    Success: 4 | Failed: 0 | Total: 4

Large fleets automatically switch to an aggregate dashboard — a live progress bar, success/failure/running counts, throughput-based ETA, the currently-running window, and a failures tail:

[+] Backing up 5000 targets
    ██████████░░░░░░░░░░░░░░░░░░░░ 1640/5000  33%   ✔ 1628  ✘ 12  ⠹ 48 running   2m14s   eta 4m31s

Active:
    ⠹ router-1641             Backing Up          1.2s
    ⠼ router-1642             Getting Info        0.4s
    … and 46 more running

Recent failures:
    router-0044, router-0210

State updates are O(1) and rendering is O(visible rows), so the view stays responsive regardless of fleet size.

Other examples

rosbackup --no-parallel              # sequential
rosbackup --max-parallel 10          # cap concurrency
rosbackup --log-level DEBUG          # verbose
rosbackup --no-tmpfs                 # disable tmpfs staging for this run
rosbackup --tmpfs-size-mb 25         # override tmpfs size for this run
rosbackup --log-file backups.log     # also write a 0600 log file

Bootstrap (provisioning a router)

Create the backup user and install its key on a device (see doc/BOOTSTRAP.md):

# Generate a key pair for the backup user
ssh-keygen -t ed25519 -f ssh-keys/private/id_rosbackup

# Provision a router (prompts for the existing admin password)
rosbackup-bootstrap -H 192.168.88.1 -k ssh-keys/private/id_rosbackup.pub

# See exactly what it would do, without changing anything
rosbackup-bootstrap -H 192.168.88.1 -k ssh-keys/private/id_rosbackup.pub --dry-run

Web platform

rosbackup-web is a self-hosted UI + HTTP API that reads the same config directory as the CLI (dashboard pictured near the top). Deploy it with the Docker stack (doc/DEPLOY.md); to run it without Docker, see doc/INSTALLATION-METHODS.md. Once it's installed:

rosbackup-web -c /etc/rosbackup --set-password   # set the admin password once
rosbackup-web -c /etc/rosbackup                  # serve on 127.0.0.1:8474

It gives you a dashboard, targets inventory (table + cards), run-now with live progress, schedules, a backup browser with download + push-restore, firmware upgrades, an encrypted Vault for SSH keys and passwords, API tokens, and an audit log. Bind it to localhost and terminate TLS at a reverse proxy (sample Caddyfile + systemd unit under doc/examples/).

Full details — auth, the Vault, schedules, push-restore, capability toggles, the API, and the single-file build — are in doc/WEB_PLATFORM.md.

Command completion

Enable bash completion for the current shell session:

source scripts/rosbackup-ng-completion.bash

It provides option, file-path, and target-name completion (from targets.yaml) for the rosbackup and rosbackup-bootstrap commands.

Logging

Console logging is colored via rich. With --log-file, logs are also written to a file (created 0600) including backup status, connection details, retries, and errors. Format: MM-DD-YYYY HH:MM:SS [LEVEL] [target] message.

Development

pip install -e ".[dev,web]"
ruff check src tests && ruff format --check src tests
mypy src
bandit -q -r src
pytest -q

CI runs this same battery on every push via self-hosted Forgejo Actions (.forgejo/workflows/ci.yml), deliberately using no external/marketplace actions — every step is hand-written shell.

Before pushing, run the whole gate in one shot with a venv that matches CI's resolved dependencies — this catches the failures a pytest-only check misses (ruff/format/bandit, and runner-environment flakes):

scripts/ci-local.sh            # ruff -> format -> mypy -> bandit -> pytest -> shiv build + smoke
scripts/ci-local.sh --fresh    # rebuild the .civenv from scratch first

It runs every gate (not just the first failure) and exits non-zero if any fails. The .civenv it creates is gitignored and reused across runs.

Contributing

1. Fork the repository
2. Create your feature branch (git checkout -b feature/AmazingFeature)
3. Run the checks above (they must all pass)
4. Commit and open a Pull Request

License

This project is licensed under the MIT License — see the LICENSE file for details.

Support

Open an issue or send a PR.