cmc-sales/scripts/VAULT_DEPLOYMENT.md

Go Vault Deployment Guide

Running the Go Vault in Docker Production

Prerequisites

• The vault binary is built into the Docker image
• Vault directories are mounted from /home/cmc/files/vaultmsgs/
• Container has access to the database

Step 1: Rebuild the Docker Image

The Dockerfile has been updated to include the vault binary. Rebuild the image:

# On the production server
cd ~/src/cmc-sales

# Rebuild the Go container
docker compose -f docker-compose.prod.yml build cmc-prod-go

# Restart the container
docker compose -f docker-compose.prod.yml up -d cmc-prod-go

Step 2: Verify Vault Binary Exists

# Check if vault binary is in the container
docker exec -t cmc-prod-go ls -lh /root/vault

# Test running vault (dry run)
docker exec -t cmc-prod-go ./vault --mode=local \
  --vaultdir=/var/www/vaultmsgs/new \
  --processeddir=/var/www/vaultmsgs/cur \
  --emaildir=/var/www/emails \
  --dbhost=cmc-prod-db \
  --dbuser=cmc \
  --dbpass="xVRQI&cA?7AU=hqJ!%au" \
  --dbname=cmc

Step 3: Set Up Cron Job

# Copy the cron script to the server
scp scripts/vault-cron-prod.sh cmc@sales.cmctechnologies.com.au:~/scripts/

# On the server, make it executable
chmod +x ~/scripts/vault-cron-prod.sh

# Create log directory
mkdir -p ~/logs

# Test the script manually
~/scripts/vault-cron-prod.sh

# Check the log
tail -f ~/logs/vault-go.log

Step 4: Add to Crontab

# Edit crontab
crontab -e

# Add this line to run every 5 minutes
*/5 * * * * /home/cmc/scripts/vault-cron-prod.sh

Step 5: Monitor

# Watch logs in real-time
tail -f ~/logs/vault-go.log

# Check cron execution
grep vault /var/log/syslog

# List cron jobs
crontab -l

Directory Structure

The following directories need to exist on the host:

/home/cmc/files/
├── vaultmsgs/
│   ├── new/       # Incoming emails to process
│   └── cur/       # Processed emails archive
├── emails/        # Email storage
└── vault/         # Additional vault data

Troubleshooting

Vault binary not found

# Rebuild the image
docker compose -f docker-compose.prod.yml build cmc-prod-go
docker compose -f docker-compose.prod.yml up -d cmc-prod-go

Permission errors

# Check directory permissions on host
ls -ld /home/cmc/files/vaultmsgs/new
ls -ld /home/cmc/files/vaultmsgs/cur

# Should be readable/writable by the user running Docker

Database connection errors

# Verify database container is running
docker ps | grep cmc-prod-db

# Test database connection from Go container
docker exec -it cmc-prod-go sh
nc -zv cmc-prod-db 3306

No emails being processed

# Check if there are emails to process
ls -la /home/cmc/files/vaultmsgs/new/

# Check if getmail or other email fetcher is running
ps aux | grep getmail

Switching from PHP to Go Vault

Current PHP Setup

The PHP vault runs via:

/var/www/cmc-sales/cake/console/cake -app /var/www/cmc-sales/app vault

Migration Steps

1. Run both in parallel (testing phase):

   # Keep PHP vault running
   */5 * * * * /path/to/old/vault_cron.sh
   
   # Add Go vault (different schedule)
   */10 * * * * /home/cmc/scripts/vault-cron-prod.sh
   

2. Compare results:

   • Check database for duplicate emails
   • Verify all business entities are linked correctly
   • Compare attachment handling

3. Switch completely:

   # Disable PHP vault
   crontab -e  # Comment out the PHP vault line
   
   # Enable Go vault only
   */5 * * * * /home/cmc/scripts/vault-cron-prod.sh
   

Advanced: Gmail Mode

For better performance, consider switching to Gmail API mode:

Setup Gmail OAuth (one-time)

# Copy credentials to the server
scp credentials.json cmc@sales.cmctechnologies.com.au:~/

# Copy to container
docker cp ~/credentials.json cmc-prod-go:/root/credentials.json

# Run initial authorization (interactive)
docker exec -it cmc-prod-go ./vault --mode=index \
  --credentials=credentials.json \
  --token=token.json \
  --gmail-query="is:unread newer_than:1d" \
  --dbhost=cmc-prod-db \
  --dbuser=cmc \
  --dbpass="xVRQI&cA?7AU=hqJ!%au" \
  --dbname=cmc

# This will generate a URL - open in browser and authorize
# Token will be saved in container

# Copy token back to host for backup
docker cp cmc-prod-go:/root/token.json ~/token.json

Update Cron for Gmail Mode

Edit scripts/vault-cron-prod.sh and change:

docker exec -t "$CONTAINER_NAME" ./vault --mode=index \
  --credentials=credentials.json \
  --token=token.json \
  --gmail-query="is:unread newer_than:1d" \
  --dbhost=cmc-prod-db \
  --dbuser=cmc \
  --dbpass="xVRQI&cA?7AU=hqJ!%au" \
  --dbname=cmc \
  >> "$LOG_FILE" 2>&1

Performance Comparison

Mode	Speed	Storage	Dependencies
Local	Moderate	High (stores files)	getmail, filesystem
Gmail Index	Fast	Low (metadata only)	Gmail API, OAuth

Related Files

• docker-compose.prod.yml - Production compose file with volume mounts
• Dockerfile.prod.go - Builds both server and vault binaries
• vault-cron-prod.sh - Cron script for running vault
• go-app/cmd/vault/README.md - Vault program documentation