Secure Backup Storage Server Setup and Automation

Overview

This guide walks through setting up a secure backup system to automatically encrypt video files from your Reolink cameras on an backup storage server and decrypt them on your macOS client. The setup includes scripts for automatic encryption, disk management, and secure key storage.

Steps to Complete Setup

1. Generate and Backup GPG Keys on macOS (Client)

Generate the GPG key pair on your macOS client, as it will hold the private key for decrypting the files.

Generate GPG Key Pair on macOS

gpg --full-generate-key

Follow the prompts:

• Choose RSA and RSA (default) with a 4096-bit key size.
• Set key expiration as desired or choose "never expire".
• Provide your name and email for key identification.
• Set a strong passphrase.

Export and Backup Your GPG Keys

On your macOS client, export both the public and private keys and securely store them in Bitwarden or another secure password manager.

Export Public Key

gpg --export --armor "Your Name" > publickey.asc

Export Private Key

gpg --export-secret-keys --armor "Your Name" > privatekey.asc

Securely Store Keys

Upload publickey.asc and privatekey.asc to Bitwarden, ensuring your password manager uses a strong master password and 2FA. This backup is critical for key recovery.

2. Transfer and Import the GPG Public Key on Ubuntu Server

Transfer Public Key to Ubuntu Server

From macOS, transfer publickey.asc to the Ubuntu server.

scp publickey.asc yourusername@server_ip:/home/yourusername/

Import Public Key on Ubuntu Server

SSH into the Ubuntu server and import the GPG public key.

ssh yourusername@server_ip
gpg --import publickey.asc

Trust the Public Key on the Server

gpg --edit-key "Your Name"

# At the gpg> prompt:
trust
5
quit

The public key is now ready to encrypt files on the server.

3. Set Up the Encryption Script (encrypt_files.sh) on Ubuntu Server

The encryption script will automatically encrypt files uploaded by the cameras after they have not been modified for 2 minutes. It excludes the log directory.

Create the Encryption Script

Create a file called encrypt_files.sh on the Ubuntu server.

#!/bin/bash

# Directory where camera directories are located
TARGET_DIR="/home/randyjc/reolink"  # Base directory for all camera directories

# List of camera directories to process
CAMERAS=("voorkant" "zijkant" "tuin" "deurbel")

# GPG Recipient (Your Name or Email)
GPG_RECIPIENT="Your Name"

# Log file path
LOG_FILE="$TARGET_DIR/log/encryption_log.txt"

# Create log directory if it doesn't exist
mkdir -p "$(dirname "$LOG_FILE")"

# Function to log messages
log_message() {
    echo "$(date +'%Y-%m-%d %H:%M:%S') - $1" >> "$LOG_FILE"
}

# Function to encrypt a file
encrypt_file() {
    local file="$1"
    gpg --batch --yes --encrypt --recipient "$GPG_RECIPIENT" "$file"
    if [ $? -eq 0 ]; then
        shred -u "$file"  # Securely delete the original file
        log_message "Encrypted and deleted: $file"
    else
        log_message "Encryption failed for: $file"
    fi
}

# Export functions and variables for use in subshells
export -f log_message
export -f encrypt_file
export GPG_RECIPIENT
export LOG_FILE

# Loop through each camera directory
for camera in "${CAMERAS[@]}"; do
    CAMERA_DIR="$TARGET_DIR/$camera"
    # Check if the directory exists
    if [ -d "$CAMERA_DIR" ]; then
        # Find and encrypt files not modified in the last 2 minutes, excluding the log directory
        find "$CAMERA_DIR" \
            -type d -name "log" -prune -o \
            -type f ! -name "*.gpg" -mmin +2 -exec bash -c 'encrypt_file "$0"' {} \;
    else
        log_message "Camera directory does not exist: $CAMERA_DIR"
    fi
done

Make the Script Executable

chmod +x /path/to/encrypt_files.sh

Schedule the Script with Cron

Open crontab to run the script every 5 minutes.

crontab -e

Add this line:

*/5 * * * * /path/to/encrypt_files.sh

4. Set Up the Cleanup Script (cleanup_script.sh) on Ubuntu Server

The cleanup script manages disk usage by deleting old encrypted files when disk usage exceeds 900GB.

Create the Cleanup Script

Create a file called cleanup_script.sh on the Ubuntu server.

#!/bin/bash
set -euo pipefail

# Set variables
TARGET_DIR="/home/randyjc/reolink"  # Base directory for all camera directories
LIMIT=900000000000  # 900GB in bytes
CAMERAS=("voorkant" "zijkant" "tuin" "deurbel")  # List of camera directories
LOG_DIR="$TARGET_DIR/log"  # Directory for logs
LOG_FILE="$LOG_DIR/cleanup_log.txt"  # Log file path

# Create log directory if it doesn't exist
mkdir -p "$LOG_DIR"

# Function to log messages
log_message() {
    echo "$(date +'%Y-%m-%d %H:%M:%S') - $1" >> "$LOG_FILE"
}

# Start script logging
log_message "Starting cleanup process. Checking current storage usage."

# Check current usage
current_usage=$(du -sb "$TARGET_DIR" | awk '{print $1}')

# Log current usage
log_message "Current usage: $(awk "BEGIN {printf \"%.2f\", $current_usage/1000000000}") GB"

# If usage exceeds limit, begin cleanup
if (( current_usage > LIMIT )); then
    log_message "Usage exceeds 900GB. Starting file deletion."

    # Initialize a flag to track if any files were deleted
    files_deleted=false

    # Loop through each camera directory
    for camera in "${CAMERAS[@]}"; do
        CAMERA_DIR="$TARGET_DIR/$camera"

        # Check if the camera directory exists
        if [ -d "$CAMERA_DIR" ]; then
            # Find all day directories sorted by date (oldest first)
            find "$CAMERA_DIR" \
                -type d -name "log" -prune -o \
                -mindepth 3 -maxdepth 3 -type d -print | sort | while read -r day_dir; do
                # Delete encrypted files (.gpg) in the day directory
                find "$day_dir" -type f -name "*.gpg" -delete

                # Log the deletion
                log_message "Deleted encrypted files in directory: $day_dir"

                # Remove empty directories
                if [ -z "$(find "$day_dir" -mindepth 1 -print -quit)" ]; then
                    rmdir "$day_dir"
                    log_message "Deleted empty directory: $day_dir"
                fi

                # Set the flag to true since files were deleted
                files_deleted=true

                # Recalculate usage after each day deletion
                current_usage=$(du -sb "$TARGET_DIR" | awk '{print $1}')
                log_message "Recalculated usage: $(awk "BEGIN {printf \"%.2f\", $current_usage/1000000000}") GB"

                # Check if usage is now under the limit
                if (( current_usage <= LIMIT )); then
                    log_message "Usage is now under 900GB. Stopping cleanup."
                    exit 0
                fi
            done
        else
            log_message "Camera directory does not exist: $CAMERA_DIR"
        fi
    done

    # If no files were deleted, log a message
    if [ "$files_deleted" = false ]; then
        log_message "No files were deleted. Check if directories contain files or if usage calculations are correct."
    fi
else
    log_message "Current usage is within limit. No cleanup necessary."
fi

log_message "Cleanup process completed."

Make the Script Executable

chmod +x /path/to/cleanup_script.sh

Schedule the Cleanup Script with Cron

Run crontab -e and add:

0 3 * * * /path/to/cleanup_script.sh

This schedules the script to run daily at 3 AM.

5. Decrypting Files on macOS Client

With encrypted files on the Ubuntu server, you’ll use your macOS machine to decrypt them using your private GPG key.

1. Install GnuPG on macOS

brew install gnupg

1. Import Your Private Key

Transfer privatekey.asc from Bitwarden or other secure storage to your macOS machine, then import it:

gpg --import /path/to/privatekey.asc

1. Decrypt Files

To decrypt a file:

gpg --output decryptedfile --decrypt encryptedfile.gpg

To decrypt multiple files:

for file in *.gpg; do
    gpg --output "${file%.gpg}" --decrypt "$file"
done

Also:
The following can also be done, but will not be covered in this guide.

• Set up monitoring and alerts for script errors or disk space.
• Regularly back up and rotate keys and logs.