Work on staging and vault
This commit is contained in:
Karl Cordes
parent
f6eef99d47
commit
687739e9d6
356
DEPLOYMENT.md
Normal file
356
DEPLOYMENT.md
Normal file
|
|
@ -0,0 +1,356 @@
|
|||
# CMC Sales Deployment Guide
|
||||
|
||||
## Overview
|
||||
|
||||
This guide covers deploying the CMC Sales application to a Debian 12 VM at `cmc.springupsoftware.com` with both staging and production environments.
|
||||
|
||||
## Architecture
|
||||
|
||||
- **Production**: `https://cmc.springupsoftware.com`
|
||||
- **Staging**: `https://staging.cmc.springupsoftware.com`
|
||||
- **Components**: CakePHP legacy app, Go modern app, MariaDB, Nginx reverse proxy
|
||||
- **SSL**: Let's Encrypt certificates
|
||||
- **Authentication**: Basic auth for both environments
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Server Setup (Debian 12)
|
||||
|
||||
```bash
|
||||
# Update system
|
||||
sudo apt update && sudo apt upgrade -y
|
||||
|
||||
# Install Docker and Docker Compose
|
||||
sudo apt install -y docker.io docker-compose-plugin
|
||||
sudo systemctl enable docker
|
||||
sudo systemctl start docker
|
||||
|
||||
# Add user to docker group
|
||||
sudo usermod -aG docker $USER
|
||||
# Log out and back in
|
||||
|
||||
# Create backup directory
|
||||
sudo mkdir -p /var/backups/cmc-sales
|
||||
sudo chown $USER:$USER /var/backups/cmc-sales
|
||||
```
|
||||
|
||||
### DNS Configuration
|
||||
|
||||
Ensure these DNS records point to your server:
|
||||
- `cmc.springupsoftware.com` → Server IP
|
||||
- `staging.cmc.springupsoftware.com` → Server IP
|
||||
|
||||
## Initial Deployment
|
||||
|
||||
### 1. Clone Repository
|
||||
|
||||
```bash
|
||||
cd /opt
|
||||
sudo git clone <repository-url> cmc-sales
|
||||
sudo chown -R $USER:$USER cmc-sales
|
||||
cd cmc-sales
|
||||
```
|
||||
|
||||
### 2. Environment Configuration
|
||||
|
||||
```bash
|
||||
# Copy environment files
|
||||
cp .env.staging go-app/.env.staging
|
||||
cp .env.production go-app/.env.production
|
||||
|
||||
# Edit with actual passwords
|
||||
nano .env.staging
|
||||
nano .env.production
|
||||
|
||||
# Create credential directories
|
||||
mkdir -p credentials/staging credentials/production
|
||||
```
|
||||
|
||||
### 3. Gmail OAuth Setup
|
||||
|
||||
For each environment (staging/production):
|
||||
|
||||
1. Go to [Google Cloud Console](https://console.cloud.google.com)
|
||||
2. Create/select project
|
||||
3. Enable Gmail API
|
||||
4. Create OAuth 2.0 credentials
|
||||
5. Download `credentials.json`
|
||||
6. Place in appropriate directory:
|
||||
- Staging: `credentials/staging/credentials.json`
|
||||
- Production: `credentials/production/credentials.json`
|
||||
|
||||
Generate tokens (run on local machine first):
|
||||
```bash
|
||||
cd go-app
|
||||
go run cmd/auth/main.go
|
||||
# Follow OAuth flow
|
||||
# Copy token.json to appropriate credential directory
|
||||
```
|
||||
|
||||
### 4. SSL Certificates
|
||||
|
||||
```bash
|
||||
# Start proxy services (includes Lego container)
|
||||
docker compose -f docker-compose.proxy.yml up -d
|
||||
|
||||
# Setup SSL certificates using Lego
|
||||
./scripts/setup-lego-certs.sh admin@springupsoftware.com
|
||||
|
||||
# Verify certificates
|
||||
./scripts/lego-list-certs.sh
|
||||
```
|
||||
|
||||
### 5. Database Initialization
|
||||
|
||||
```bash
|
||||
# Start database containers first
|
||||
docker compose -f docker-compose.staging.yml up -d db-staging
|
||||
docker compose -f docker-compose.production.yml up -d db-production
|
||||
|
||||
# Wait for databases to be ready
|
||||
sleep 30
|
||||
|
||||
# Restore production database (if you have a backup)
|
||||
./scripts/restore-db.sh production /path/to/backup.sql.gz
|
||||
|
||||
# Or initialize empty database and run migrations
|
||||
# (implementation specific)
|
||||
```
|
||||
|
||||
## Deployment Commands
|
||||
|
||||
### Starting Services
|
||||
|
||||
```bash
|
||||
# Start staging environment
|
||||
docker compose -f docker-compose.staging.yml up -d
|
||||
|
||||
# Start production environment
|
||||
docker compose -f docker-compose.production.yml up -d
|
||||
|
||||
# Start reverse proxy (after both environments are running)
|
||||
docker compose -f docker-compose.proxy.yml up -d
|
||||
```
|
||||
|
||||
### Updating Applications
|
||||
|
||||
```bash
|
||||
# Pull latest code
|
||||
git pull origin main
|
||||
|
||||
# Rebuild and restart staging
|
||||
docker compose -f docker-compose.staging.yml down
|
||||
docker compose -f docker-compose.staging.yml build --no-cache
|
||||
docker compose -f docker-compose.staging.yml up -d
|
||||
|
||||
# Test staging thoroughly, then update production
|
||||
docker compose -f docker-compose.production.yml down
|
||||
docker compose -f docker-compose.production.yml build --no-cache
|
||||
docker compose -f docker-compose.production.yml up -d
|
||||
```
|
||||
|
||||
## Monitoring and Maintenance
|
||||
|
||||
### Health Checks
|
||||
|
||||
```bash
|
||||
# Check all containers
|
||||
docker ps
|
||||
|
||||
# Check logs
|
||||
docker compose -f docker-compose.production.yml logs -f
|
||||
|
||||
# Check application health
|
||||
curl https://cmc.springupsoftware.com/health
|
||||
curl https://staging.cmc.springupsoftware.com/health
|
||||
```
|
||||
|
||||
### Database Backups
|
||||
|
||||
```bash
|
||||
# Manual backup
|
||||
./scripts/backup-db.sh production
|
||||
./scripts/backup-db.sh staging
|
||||
|
||||
# Set up automated backups (cron)
|
||||
sudo crontab -e
|
||||
# Add: 0 2 * * * /opt/cmc-sales/scripts/backup-db.sh production
|
||||
# Add: 0 3 * * * /opt/cmc-sales/scripts/backup-db.sh staging
|
||||
```
|
||||
|
||||
### Log Management
|
||||
|
||||
```bash
|
||||
# View nginx logs
|
||||
sudo tail -f /var/log/nginx/access.log
|
||||
sudo tail -f /var/log/nginx/error.log
|
||||
|
||||
# View application logs
|
||||
docker compose -f docker-compose.production.yml logs -f cmc-go-production
|
||||
docker compose -f docker-compose.staging.yml logs -f cmc-go-staging
|
||||
```
|
||||
|
||||
### SSL Certificate Renewal
|
||||
|
||||
```bash
|
||||
# Manual renewal
|
||||
./scripts/lego-renew-cert.sh all
|
||||
|
||||
# Renew specific domain
|
||||
./scripts/lego-renew-cert.sh cmc.springupsoftware.com
|
||||
|
||||
# Set up auto-renewal (cron)
|
||||
sudo crontab -e
|
||||
# Add: 0 2 * * * /opt/cmc-sales/scripts/lego-renew-cert.sh all
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Basic Authentication
|
||||
|
||||
Update passwords in `userpasswd` file:
|
||||
```bash
|
||||
# Generate new password hash
|
||||
sudo apt install apache2-utils
|
||||
htpasswd -c userpasswd username
|
||||
|
||||
# Restart nginx containers
|
||||
docker compose -f docker-compose.proxy.yml restart nginx-proxy
|
||||
```
|
||||
|
||||
### Database Security
|
||||
|
||||
- Use strong passwords in environment files
|
||||
- Database containers are not exposed externally in production
|
||||
- Regular backups with encryption at rest
|
||||
|
||||
### Network Security
|
||||
|
||||
- All traffic encrypted with SSL/TLS
|
||||
- Rate limiting configured in nginx
|
||||
- Security headers enabled
|
||||
- Docker networks isolate environments
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **Containers won't start**
|
||||
```bash
|
||||
# Check logs
|
||||
docker compose logs container-name
|
||||
|
||||
# Check system resources
|
||||
df -h
|
||||
free -h
|
||||
```
|
||||
|
||||
2. **SSL issues**
|
||||
```bash
|
||||
# Check certificate status
|
||||
./scripts/lego-list-certs.sh
|
||||
|
||||
# Test SSL configuration
|
||||
curl -I https://cmc.springupsoftware.com
|
||||
|
||||
# Manually renew certificates
|
||||
./scripts/lego-renew-cert.sh all
|
||||
```
|
||||
|
||||
3. **Database connection issues**
|
||||
```bash
|
||||
# Test database connectivity
|
||||
docker exec -it cmc-db-production mysql -u cmc -p
|
||||
```
|
||||
|
||||
4. **Gmail API issues**
|
||||
```bash
|
||||
# Check credentials are mounted
|
||||
docker exec -it cmc-go-production ls -la /root/credentials/
|
||||
|
||||
# Check logs for OAuth errors
|
||||
docker compose logs cmc-go-production | grep -i gmail
|
||||
```
|
||||
|
||||
### Emergency Procedures
|
||||
|
||||
1. **Quick rollback**
|
||||
```bash
|
||||
# Stop current containers
|
||||
docker compose -f docker-compose.production.yml down
|
||||
|
||||
# Restore from backup
|
||||
./scripts/restore-db.sh production /var/backups/cmc-sales/latest_backup.sql.gz
|
||||
|
||||
# Start previous version
|
||||
git checkout previous-commit
|
||||
docker compose -f docker-compose.production.yml up -d
|
||||
```
|
||||
|
||||
2. **Database corruption**
|
||||
```bash
|
||||
# Stop application
|
||||
docker compose -f docker-compose.production.yml stop cmc-go-production cmc-php-production
|
||||
|
||||
# Restore from backup
|
||||
./scripts/restore-db.sh production /var/backups/cmc-sales/backup_production_YYYYMMDD-HHMMSS.sql.gz
|
||||
|
||||
# Restart application
|
||||
docker compose -f docker-compose.production.yml start cmc-go-production cmc-php-production
|
||||
```
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
/opt/cmc-sales/
|
||||
├── docker-compose.staging.yml
|
||||
├── docker-compose.production.yml
|
||||
├── docker-compose.proxy.yml
|
||||
├── conf/
|
||||
│ ├── nginx-staging.conf
|
||||
│ ├── nginx-production.conf
|
||||
│ └── nginx-proxy.conf
|
||||
├── credentials/
|
||||
│ ├── staging/
|
||||
│ │ ├── credentials.json
|
||||
│ │ └── token.json
|
||||
│ └── production/
|
||||
│ ├── credentials.json
|
||||
│ └── token.json
|
||||
├── scripts/
|
||||
│ ├── backup-db.sh
|
||||
│ ├── restore-db.sh
|
||||
│ ├── lego-obtain-cert.sh
|
||||
│ ├── lego-renew-cert.sh
|
||||
│ ├── lego-list-certs.sh
|
||||
│ └── setup-lego-certs.sh
|
||||
└── .env files
|
||||
```
|
||||
|
||||
## Performance Tuning
|
||||
|
||||
### Resource Limits
|
||||
|
||||
Resource limits are configured in the Docker Compose files:
|
||||
- Production: 2 CPU cores, 2-4GB RAM per service
|
||||
- Staging: More relaxed limits for testing
|
||||
|
||||
### Database Optimization
|
||||
|
||||
```sql
|
||||
-- Monitor slow queries
|
||||
SHOW VARIABLES LIKE 'slow_query_log';
|
||||
SET GLOBAL slow_query_log = 'ON';
|
||||
SET GLOBAL long_query_time = 2;
|
||||
|
||||
-- Check database performance
|
||||
SHOW PROCESSLIST;
|
||||
SHOW ENGINE INNODB STATUS;
|
||||
```
|
||||
|
||||
### Nginx Optimization
|
||||
|
||||
- Gzip compression enabled
|
||||
- Static file caching
|
||||
- Connection keep-alive
|
||||
- Rate limiting configured
|
||||
|
|
@ -40,6 +40,13 @@ COPY --from=builder /app/server .
|
|||
COPY go-app/templates ./templates
|
||||
COPY go-app/static ./static
|
||||
|
||||
# Copy Gmail OAuth credentials if they exist
|
||||
# Note: These files contain sensitive data and should not be committed to version control
|
||||
# Consider using Docker secrets or environment variables for production
|
||||
# Using conditional copy pattern to handle missing files gracefully
|
||||
COPY go-app/credentials.jso[n] ./
|
||||
COPY go-app/token.jso[n] ./
|
||||
|
||||
# Copy .env file if needed
|
||||
COPY go-app/.env.example .env
|
||||
|
||||
|
|
|
|||
63
Dockerfile.go.production
Normal file
63
Dockerfile.go.production
Normal file
|
|
@ -0,0 +1,63 @@
|
|||
# Build stage
|
||||
FROM golang:1.23-alpine AS builder
|
||||
|
||||
# Install build dependencies
|
||||
RUN apk add --no-cache git
|
||||
|
||||
# Set working directory
|
||||
WORKDIR /app
|
||||
|
||||
# Copy go mod files
|
||||
COPY go-app/go.mod go-app/go.sum ./
|
||||
|
||||
# Download dependencies
|
||||
RUN go mod download
|
||||
|
||||
# Copy source code
|
||||
COPY go-app/ .
|
||||
|
||||
# Install sqlc (compatible with Go 1.23+)
|
||||
RUN go install github.com/sqlc-dev/sqlc/cmd/sqlc@latest
|
||||
|
||||
# Generate sqlc code
|
||||
RUN sqlc generate
|
||||
|
||||
# Build the application with production optimizations
|
||||
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags="-w -s" -tags production -o server cmd/server/main.go
|
||||
|
||||
# Runtime stage - minimal image for production
|
||||
FROM alpine:latest
|
||||
|
||||
# Install only essential runtime dependencies
|
||||
RUN apk --no-cache add ca-certificates && \
|
||||
addgroup -g 1001 -S appgroup && \
|
||||
adduser -u 1001 -S appuser -G appgroup
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Copy the binary from builder
|
||||
COPY --from=builder /app/server .
|
||||
|
||||
# Copy templates and static files
|
||||
COPY go-app/templates ./templates
|
||||
COPY go-app/static ./static
|
||||
|
||||
# Copy production environment file
|
||||
COPY go-app/.env.production .env
|
||||
|
||||
# Create credentials directory with proper permissions
|
||||
RUN mkdir -p ./credentials && \
|
||||
chown -R appuser:appgroup /app
|
||||
|
||||
# Switch to non-root user
|
||||
USER appuser
|
||||
|
||||
# Expose port
|
||||
EXPOSE 8080
|
||||
|
||||
# Health check
|
||||
HEALTHCHECK --interval=60s --timeout=10s --start-period=10s --retries=3 \
|
||||
CMD wget --no-verbose --tries=1 --spider http://localhost:8080/api/v1/health || exit 1
|
||||
|
||||
# Run the application
|
||||
CMD ["./server"]
|
||||
57
Dockerfile.go.staging
Normal file
57
Dockerfile.go.staging
Normal file
|
|
@ -0,0 +1,57 @@
|
|||
# Build stage
|
||||
FROM golang:1.23-alpine AS builder
|
||||
|
||||
# Install build dependencies
|
||||
RUN apk add --no-cache git
|
||||
|
||||
# Set working directory
|
||||
WORKDIR /app
|
||||
|
||||
# Copy go mod files
|
||||
COPY go-app/go.mod go-app/go.sum ./
|
||||
|
||||
# Download dependencies
|
||||
RUN go mod download
|
||||
|
||||
# Copy source code
|
||||
COPY go-app/ .
|
||||
|
||||
# Install sqlc (compatible with Go 1.23+)
|
||||
RUN go install github.com/sqlc-dev/sqlc/cmd/sqlc@latest
|
||||
|
||||
# Generate sqlc code
|
||||
RUN sqlc generate
|
||||
|
||||
# Build the application with staging tags
|
||||
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -tags staging -o server cmd/server/main.go
|
||||
|
||||
# Runtime stage
|
||||
FROM alpine:latest
|
||||
|
||||
# Install runtime dependencies and debugging tools for staging
|
||||
RUN apk --no-cache add ca-certificates curl net-tools
|
||||
|
||||
WORKDIR /root/
|
||||
|
||||
# Copy the binary from builder
|
||||
COPY --from=builder /app/server .
|
||||
|
||||
# Copy templates and static files
|
||||
COPY go-app/templates ./templates
|
||||
COPY go-app/static ./static
|
||||
|
||||
# Copy staging environment file
|
||||
COPY go-app/.env.staging .env
|
||||
|
||||
# Create credentials directory
|
||||
RUN mkdir -p ./credentials
|
||||
|
||||
# Expose port
|
||||
EXPOSE 8080
|
||||
|
||||
# Health check
|
||||
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
|
||||
CMD curl -f http://localhost:8080/api/v1/health || exit 1
|
||||
|
||||
# Run the application
|
||||
CMD ["./server"]
|
||||
394
TESTING_DOCKER.md
Normal file
394
TESTING_DOCKER.md
Normal file
|
|
@ -0,0 +1,394 @@
|
|||
# Running CMC Django Tests in Docker
|
||||
|
||||
This guide explains how to run the comprehensive CMC Django test suite using Docker for consistent, isolated testing.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# 1. Setup test environment (one-time)
|
||||
./run-tests-docker.sh setup
|
||||
|
||||
# 2. Run all tests
|
||||
./run-tests-docker.sh run
|
||||
|
||||
# 3. Run tests with coverage
|
||||
./run-tests-docker.sh coverage
|
||||
```
|
||||
|
||||
## Test Environment Overview
|
||||
|
||||
The Docker test environment includes:
|
||||
- **Isolated test database** (MariaDB on port 3307)
|
||||
- **Django test container** with all dependencies
|
||||
- **Coverage reporting** with HTML and XML output
|
||||
- **PDF generation testing** with WeasyPrint/ReportLab
|
||||
- **Parallel test execution** support
|
||||
|
||||
## Available Commands
|
||||
|
||||
### Setup and Management
|
||||
|
||||
```bash
|
||||
# Build containers and setup test database
|
||||
./run-tests-docker.sh setup
|
||||
|
||||
# Clean up all test containers and data
|
||||
./run-tests-docker.sh clean
|
||||
|
||||
# View test container logs
|
||||
./run-tests-docker.sh logs
|
||||
|
||||
# Open shell in test container
|
||||
./run-tests-docker.sh shell
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
./run-tests-docker.sh run
|
||||
|
||||
# Run specific test suites
|
||||
./run-tests-docker.sh run models # Model tests only
|
||||
./run-tests-docker.sh run services # Service layer tests
|
||||
./run-tests-docker.sh run auth # Authentication tests
|
||||
./run-tests-docker.sh run views # View and URL tests
|
||||
./run-tests-docker.sh run pdf # PDF generation tests
|
||||
./run-tests-docker.sh run integration # Integration tests
|
||||
|
||||
# Run quick tests (models + services)
|
||||
./run-tests-docker.sh quick
|
||||
|
||||
# Run tests with coverage reporting
|
||||
./run-tests-docker.sh coverage
|
||||
```
|
||||
|
||||
## Advanced Test Options
|
||||
|
||||
### Using Docker Compose Directly
|
||||
|
||||
```bash
|
||||
# Run specific test with custom options
|
||||
docker-compose -f docker-compose.test.yml run --rm cmc-django-test \
|
||||
python cmcsales/manage.py test cmc.tests.test_models --verbosity=2 --keepdb
|
||||
|
||||
# Run tests with coverage
|
||||
docker-compose -f docker-compose.test.yml run --rm cmc-django-test \
|
||||
coverage run --source='.' cmcsales/manage.py test cmc.tests
|
||||
|
||||
# Generate coverage report
|
||||
docker-compose -f docker-compose.test.yml run --rm cmc-django-test \
|
||||
coverage report --show-missing
|
||||
```
|
||||
|
||||
### Using the Test Script Directly
|
||||
|
||||
```bash
|
||||
# Inside the container, you can use the test script with advanced options
|
||||
docker-compose -f docker-compose.test.yml run --rm cmc-django-test \
|
||||
/app/scripts/run-tests.sh --coverage --keepdb --failfast models
|
||||
|
||||
# Script options:
|
||||
# -c, --coverage Enable coverage reporting
|
||||
# -k, --keepdb Keep test database between runs
|
||||
# -p, --parallel NUM Run tests in parallel
|
||||
# -f, --failfast Stop on first failure
|
||||
# -v, --verbosity NUM Verbosity level 0-3
|
||||
```
|
||||
|
||||
## Test Suite Structure
|
||||
|
||||
### 1. Model Tests (`test_models.py`)
|
||||
Tests all Django models including:
|
||||
- Customer, Enquiry, Job, Document models
|
||||
- Model validation and constraints
|
||||
- Relationships and cascade behavior
|
||||
- Financial calculations
|
||||
|
||||
```bash
|
||||
./run-tests-docker.sh run models
|
||||
```
|
||||
|
||||
### 2. Service Tests (`test_services.py`)
|
||||
Tests business logic layer:
|
||||
- Number generation service
|
||||
- Financial calculation service
|
||||
- Document service workflows
|
||||
- Validation service
|
||||
|
||||
```bash
|
||||
./run-tests-docker.sh run services
|
||||
```
|
||||
|
||||
### 3. Authentication Tests (`test_authentication.py`)
|
||||
Tests authentication system:
|
||||
- Multiple authentication backends
|
||||
- Permission decorators and middleware
|
||||
- User management workflows
|
||||
- Security features
|
||||
|
||||
```bash
|
||||
./run-tests-docker.sh run auth
|
||||
```
|
||||
|
||||
### 4. View Tests (`test_views.py`)
|
||||
Tests web interface:
|
||||
- CRUD operations for all entities
|
||||
- AJAX endpoints
|
||||
- Permission enforcement
|
||||
- URL routing
|
||||
|
||||
```bash
|
||||
./run-tests-docker.sh run views
|
||||
```
|
||||
|
||||
### 5. PDF Tests (`test_pdf.py`)
|
||||
Tests PDF generation:
|
||||
- WeasyPrint and ReportLab engines
|
||||
- Template rendering
|
||||
- Document formatting
|
||||
- Security and performance
|
||||
|
||||
```bash
|
||||
./run-tests-docker.sh run pdf
|
||||
```
|
||||
|
||||
### 6. Integration Tests (`test_integration.py`)
|
||||
Tests complete workflows:
|
||||
- End-to-end business processes
|
||||
- Multi-user collaboration
|
||||
- System integration scenarios
|
||||
- Performance and security
|
||||
|
||||
```bash
|
||||
./run-tests-docker.sh run integration
|
||||
```
|
||||
|
||||
## Test Reports and Coverage
|
||||
|
||||
### Coverage Reports
|
||||
|
||||
After running tests with coverage, reports are available in:
|
||||
- **HTML Report**: `./coverage-reports/html/index.html`
|
||||
- **XML Report**: `./coverage-reports/coverage.xml`
|
||||
- **Console**: Displayed after test run
|
||||
|
||||
```bash
|
||||
# Run tests with coverage
|
||||
./run-tests-docker.sh coverage
|
||||
|
||||
# View HTML report
|
||||
open coverage-reports/html/index.html
|
||||
```
|
||||
|
||||
### Test Artifacts
|
||||
|
||||
Test outputs are saved to:
|
||||
- **Test Reports**: `./test-reports/`
|
||||
- **Coverage Reports**: `./coverage-reports/`
|
||||
- **Logs**: `./logs/`
|
||||
- **PDF Test Files**: `./test-reports/pdf/`
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
The test environment uses these key variables:
|
||||
|
||||
```yaml
|
||||
# Database configuration
|
||||
DATABASE_HOST: test-db
|
||||
DATABASE_NAME: test_cmc
|
||||
DATABASE_USER: test_cmc
|
||||
DATABASE_PASSWORD: testPassword123
|
||||
|
||||
# Django settings
|
||||
DJANGO_SETTINGS_MODULE: cmcsales.settings
|
||||
TESTING: 1
|
||||
DEBUG: 0
|
||||
|
||||
# PDF generation
|
||||
PDF_GENERATION_ENGINE: weasyprint
|
||||
PDF_SAVE_DIRECTORY: /app/test-reports/pdf
|
||||
```
|
||||
|
||||
### Test Database
|
||||
|
||||
- **Isolated database** separate from development/production
|
||||
- **Runs on port 3307** to avoid conflicts
|
||||
- **Optimized for testing** with reduced buffer sizes
|
||||
- **Automatically reset** between test runs (unless `--keepdb` used)
|
||||
|
||||
## Performance Optimization
|
||||
|
||||
### Parallel Test Execution
|
||||
|
||||
```bash
|
||||
# Run tests in parallel (faster execution)
|
||||
docker-compose -f docker-compose.test.yml run --rm cmc-django-test \
|
||||
/app/scripts/run-tests.sh --parallel=4 all
|
||||
```
|
||||
|
||||
### Keeping Test Database
|
||||
|
||||
```bash
|
||||
# Keep database between runs for faster subsequent tests
|
||||
./run-tests-docker.sh run models --keepdb
|
||||
```
|
||||
|
||||
### Quick Test Suite
|
||||
|
||||
```bash
|
||||
# Run only essential tests for rapid feedback
|
||||
./run-tests-docker.sh quick
|
||||
```
|
||||
|
||||
## Continuous Integration
|
||||
|
||||
### GitHub Actions Example
|
||||
|
||||
```yaml
|
||||
name: Test CMC Django
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
|
||||
- name: Setup test environment
|
||||
run: ./run-tests-docker.sh setup
|
||||
|
||||
- name: Run tests with coverage
|
||||
run: ./run-tests-docker.sh coverage
|
||||
|
||||
- name: Upload coverage reports
|
||||
uses: codecov/codecov-action@v3
|
||||
with:
|
||||
file: ./coverage-reports/coverage.xml
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Database Connection Issues
|
||||
|
||||
```bash
|
||||
# Check database status
|
||||
docker-compose -f docker-compose.test.yml ps
|
||||
|
||||
# View database logs
|
||||
docker-compose -f docker-compose.test.yml logs test-db
|
||||
|
||||
# Restart database
|
||||
docker-compose -f docker-compose.test.yml restart test-db
|
||||
```
|
||||
|
||||
### Test Failures
|
||||
|
||||
```bash
|
||||
# Run with maximum verbosity for debugging
|
||||
docker-compose -f docker-compose.test.yml run --rm cmc-django-test \
|
||||
python cmcsales/manage.py test cmc.tests.test_models --verbosity=3
|
||||
|
||||
# Use failfast to stop on first error
|
||||
./run-tests-docker.sh run models --failfast
|
||||
|
||||
# Open shell to investigate
|
||||
./run-tests-docker.sh shell
|
||||
```
|
||||
|
||||
### Permission Issues
|
||||
|
||||
```bash
|
||||
# Fix file permissions
|
||||
sudo chown -R $USER:$USER test-reports coverage-reports logs
|
||||
|
||||
# Check Docker permissions
|
||||
docker-compose -f docker-compose.test.yml run --rm cmc-django-test whoami
|
||||
```
|
||||
|
||||
### Memory Issues
|
||||
|
||||
```bash
|
||||
# Run tests with reduced parallel workers
|
||||
docker-compose -f docker-compose.test.yml run --rm cmc-django-test \
|
||||
/app/scripts/run-tests.sh --parallel=1 all
|
||||
|
||||
# Monitor resource usage
|
||||
docker stats
|
||||
```
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### Recommended Testing Workflow
|
||||
|
||||
1. **Initial Setup** (one-time):
|
||||
```bash
|
||||
./run-tests-docker.sh setup
|
||||
```
|
||||
|
||||
2. **During Development** (fast feedback):
|
||||
```bash
|
||||
./run-tests-docker.sh quick --keepdb
|
||||
```
|
||||
|
||||
3. **Before Commit** (comprehensive):
|
||||
```bash
|
||||
./run-tests-docker.sh coverage
|
||||
```
|
||||
|
||||
4. **Debugging Issues**:
|
||||
```bash
|
||||
./run-tests-docker.sh shell
|
||||
# Inside container:
|
||||
python cmcsales/manage.py test cmc.tests.test_models.CustomerModelTest.test_customer_creation --verbosity=3
|
||||
```
|
||||
|
||||
### Adding New Tests
|
||||
|
||||
1. Create test file in appropriate module
|
||||
2. Follow existing test patterns and base classes
|
||||
3. Test locally:
|
||||
```bash
|
||||
./run-tests-docker.sh run models --keepdb
|
||||
```
|
||||
4. Run full suite before committing:
|
||||
```bash
|
||||
./run-tests-docker.sh coverage
|
||||
```
|
||||
|
||||
## Integration with IDE
|
||||
|
||||
### PyCharm/IntelliJ
|
||||
|
||||
Configure remote interpreter using Docker:
|
||||
1. Go to Settings → Project → Python Interpreter
|
||||
2. Add Docker Compose interpreter
|
||||
3. Use `docker-compose.test.yml` configuration
|
||||
4. Set service to `cmc-django-test`
|
||||
|
||||
### VS Code
|
||||
|
||||
Use Dev Containers extension:
|
||||
1. Create `.devcontainer/devcontainer.json`
|
||||
2. Configure to use test Docker environment
|
||||
3. Run tests directly in integrated terminal
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Always run tests in Docker** for consistency
|
||||
2. **Use `--keepdb` during development** for speed
|
||||
3. **Run coverage reports before commits**
|
||||
4. **Clean up regularly** to free disk space
|
||||
5. **Monitor test performance** and optimize slow tests
|
||||
6. **Use parallel execution** for large test suites
|
||||
7. **Keep test data realistic** but minimal
|
||||
8. **Test error conditions** as well as happy paths
|
||||
|
||||
## Resources
|
||||
|
||||
- **Django Testing Documentation**: https://docs.djangoproject.com/en/5.1/topics/testing/
|
||||
- **Coverage.py Documentation**: https://coverage.readthedocs.io/
|
||||
- **Docker Compose Reference**: https://docs.docker.com/compose/
|
||||
- **CMC Test Suite Documentation**: See individual test modules for detailed information
|
||||
152
conf/nginx-production.conf
Normal file
152
conf/nginx-production.conf
Normal file
|
|
@ -0,0 +1,152 @@
|
|||
# Production environment configuration
|
||||
upstream cmc_php_production {
|
||||
server cmc-php-production:80;
|
||||
keepalive 32;
|
||||
}
|
||||
|
||||
upstream cmc_go_production {
|
||||
server cmc-go-production:8080;
|
||||
keepalive 32;
|
||||
}
|
||||
|
||||
# Rate limiting
|
||||
limit_req_zone $binary_remote_addr zone=login:10m rate=5r/m;
|
||||
limit_req_zone $binary_remote_addr zone=api:10m rate=30r/m;
|
||||
|
||||
server {
|
||||
server_name cmc.springupsoftware.com;
|
||||
|
||||
# Basic auth for production
|
||||
auth_basic_user_file /etc/nginx/userpasswd;
|
||||
auth_basic "CMC Sales - Restricted Access";
|
||||
|
||||
# Security headers
|
||||
add_header X-Frame-Options DENY;
|
||||
add_header X-Content-Type-Options nosniff;
|
||||
add_header X-XSS-Protection "1; mode=block";
|
||||
add_header Referrer-Policy "strict-origin-when-cross-origin";
|
||||
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
|
||||
|
||||
# Hide server information
|
||||
server_tokens off;
|
||||
|
||||
# Request size limits
|
||||
client_max_body_size 50M;
|
||||
client_body_timeout 30s;
|
||||
client_header_timeout 30s;
|
||||
|
||||
# Compression
|
||||
gzip on;
|
||||
gzip_vary on;
|
||||
gzip_min_length 1024;
|
||||
gzip_types text/plain text/css text/xml text/javascript application/javascript application/xml+rss application/json;
|
||||
|
||||
# CakePHP legacy app routes
|
||||
location / {
|
||||
limit_req zone=api burst=10 nodelay;
|
||||
|
||||
proxy_pass http://cmc_php_production;
|
||||
proxy_read_timeout 300s;
|
||||
proxy_connect_timeout 10s;
|
||||
proxy_send_timeout 30s;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# Buffer settings for better performance
|
||||
proxy_buffering on;
|
||||
proxy_buffer_size 128k;
|
||||
proxy_buffers 4 256k;
|
||||
proxy_busy_buffers_size 256k;
|
||||
}
|
||||
|
||||
# Go API routes
|
||||
location /api/ {
|
||||
limit_req zone=api burst=20 nodelay;
|
||||
|
||||
proxy_pass http://cmc_go_production;
|
||||
proxy_read_timeout 300s;
|
||||
proxy_connect_timeout 10s;
|
||||
proxy_send_timeout 30s;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# Buffer settings for better performance
|
||||
proxy_buffering on;
|
||||
proxy_buffer_size 128k;
|
||||
proxy_buffers 4 256k;
|
||||
proxy_busy_buffers_size 256k;
|
||||
}
|
||||
|
||||
# Go page routes for emails
|
||||
location ~ ^/(emails|customers|products|purchase-orders|enquiries|documents) {
|
||||
limit_req zone=api burst=15 nodelay;
|
||||
|
||||
proxy_pass http://cmc_go_production;
|
||||
proxy_read_timeout 300s;
|
||||
proxy_connect_timeout 10s;
|
||||
proxy_send_timeout 30s;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# Buffer settings for better performance
|
||||
proxy_buffering on;
|
||||
proxy_buffer_size 128k;
|
||||
proxy_buffers 4 256k;
|
||||
proxy_busy_buffers_size 256k;
|
||||
}
|
||||
|
||||
# Static files from Go app with aggressive caching
|
||||
location /static/ {
|
||||
proxy_pass http://cmc_go_production;
|
||||
proxy_cache_valid 200 24h;
|
||||
add_header Cache-Control "public, max-age=86400";
|
||||
expires 1d;
|
||||
}
|
||||
|
||||
# PDF files with caching
|
||||
location /pdf/ {
|
||||
proxy_pass http://cmc_go_production;
|
||||
proxy_cache_valid 200 1h;
|
||||
add_header Cache-Control "public, max-age=3600";
|
||||
expires 1h;
|
||||
}
|
||||
|
||||
# Health check endpoints (no rate limiting)
|
||||
location /health {
|
||||
proxy_pass http://cmc_go_production/api/v1/health;
|
||||
access_log off;
|
||||
}
|
||||
|
||||
# Block common attack patterns
|
||||
location ~ /\. {
|
||||
deny all;
|
||||
access_log off;
|
||||
log_not_found off;
|
||||
}
|
||||
|
||||
location ~ ~$ {
|
||||
deny all;
|
||||
access_log off;
|
||||
log_not_found off;
|
||||
}
|
||||
|
||||
# Error pages
|
||||
error_page 502 503 504 /50x.html;
|
||||
location = /50x.html {
|
||||
root /usr/share/nginx/html;
|
||||
}
|
||||
|
||||
# Custom error page for rate limiting
|
||||
error_page 429 /429.html;
|
||||
location = /429.html {
|
||||
root /usr/share/nginx/html;
|
||||
}
|
||||
|
||||
listen 80;
|
||||
}
|
||||
137
conf/nginx-proxy.conf
Normal file
137
conf/nginx-proxy.conf
Normal file
|
|
@ -0,0 +1,137 @@
|
|||
user nginx;
|
||||
worker_processes auto;
|
||||
error_log /var/log/nginx/error.log warn;
|
||||
pid /var/run/nginx.pid;
|
||||
|
||||
events {
|
||||
worker_connections 1024;
|
||||
use epoll;
|
||||
multi_accept on;
|
||||
}
|
||||
|
||||
http {
|
||||
include /etc/nginx/mime.types;
|
||||
default_type application/octet-stream;
|
||||
|
||||
# Logging
|
||||
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
|
||||
'$status $body_bytes_sent "$http_referer" '
|
||||
'"$http_user_agent" "$http_x_forwarded_for"';
|
||||
|
||||
access_log /var/log/nginx/access.log main;
|
||||
|
||||
# Performance
|
||||
sendfile on;
|
||||
tcp_nopush on;
|
||||
tcp_nodelay on;
|
||||
keepalive_timeout 65;
|
||||
types_hash_max_size 2048;
|
||||
server_tokens off;
|
||||
|
||||
# Gzip
|
||||
gzip on;
|
||||
gzip_vary on;
|
||||
gzip_min_length 1024;
|
||||
gzip_types text/plain text/css text/xml text/javascript application/javascript application/xml+rss application/json;
|
||||
|
||||
# Rate limiting
|
||||
limit_req_zone $binary_remote_addr zone=global:10m rate=10r/s;
|
||||
|
||||
# SSL configuration
|
||||
ssl_protocols TLSv1.2 TLSv1.3;
|
||||
ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384;
|
||||
ssl_prefer_server_ciphers off;
|
||||
ssl_session_cache shared:SSL:10m;
|
||||
ssl_session_timeout 10m;
|
||||
|
||||
# Upstream servers
|
||||
upstream cmc_staging {
|
||||
server nginx-staging:80;
|
||||
keepalive 32;
|
||||
}
|
||||
|
||||
upstream cmc_production {
|
||||
server nginx-production:80;
|
||||
keepalive 32;
|
||||
}
|
||||
|
||||
# Redirect HTTP to HTTPS
|
||||
server {
|
||||
listen 80;
|
||||
server_name cmc.springupsoftware.com staging.cmc.springupsoftware.com;
|
||||
|
||||
# ACME challenge for Lego
|
||||
location /.well-known/acme-challenge/ {
|
||||
root /var/www/acme-challenge;
|
||||
try_files $uri =404;
|
||||
}
|
||||
|
||||
# Redirect all other traffic to HTTPS
|
||||
location / {
|
||||
return 301 https://$server_name$request_uri;
|
||||
}
|
||||
}
|
||||
|
||||
# Production HTTPS
|
||||
server {
|
||||
listen 443 ssl http2;
|
||||
server_name cmc.springupsoftware.com;
|
||||
|
||||
ssl_certificate /etc/ssl/certs/cmc.springupsoftware.com.crt;
|
||||
ssl_certificate_key /etc/ssl/certs/cmc.springupsoftware.com.key;
|
||||
|
||||
# Security headers
|
||||
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
|
||||
add_header X-Frame-Options DENY;
|
||||
add_header X-Content-Type-Options nosniff;
|
||||
add_header X-XSS-Protection "1; mode=block";
|
||||
|
||||
# Rate limiting
|
||||
limit_req zone=global burst=20 nodelay;
|
||||
|
||||
location / {
|
||||
proxy_pass http://cmc_production;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# Buffer settings
|
||||
proxy_buffering on;
|
||||
proxy_buffer_size 128k;
|
||||
proxy_buffers 4 256k;
|
||||
proxy_busy_buffers_size 256k;
|
||||
}
|
||||
}
|
||||
|
||||
# Staging HTTPS
|
||||
server {
|
||||
listen 443 ssl http2;
|
||||
server_name staging.cmc.springupsoftware.com;
|
||||
|
||||
ssl_certificate /etc/ssl/certs/staging.cmc.springupsoftware.com.crt;
|
||||
ssl_certificate_key /etc/ssl/certs/staging.cmc.springupsoftware.com.key;
|
||||
|
||||
# Security headers (less strict for staging)
|
||||
add_header X-Frame-Options DENY;
|
||||
add_header X-Content-Type-Options nosniff;
|
||||
add_header X-Environment "STAGING";
|
||||
|
||||
# Rate limiting (more lenient for staging)
|
||||
limit_req zone=global burst=50 nodelay;
|
||||
|
||||
location / {
|
||||
proxy_pass http://cmc_staging;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# Buffer settings
|
||||
proxy_buffering on;
|
||||
proxy_buffer_size 128k;
|
||||
proxy_buffers 4 256k;
|
||||
proxy_busy_buffers_size 256k;
|
||||
}
|
||||
}
|
||||
}
|
||||
89
conf/nginx-staging.conf
Normal file
89
conf/nginx-staging.conf
Normal file
|
|
@ -0,0 +1,89 @@
|
|||
# Staging environment configuration
|
||||
upstream cmc_php_staging {
|
||||
server cmc-php-staging:80;
|
||||
}
|
||||
|
||||
upstream cmc_go_staging {
|
||||
server cmc-go-staging:8080;
|
||||
}
|
||||
|
||||
server {
|
||||
server_name staging.cmc.springupsoftware.com;
|
||||
|
||||
# Basic auth for staging
|
||||
auth_basic_user_file /etc/nginx/userpasswd;
|
||||
auth_basic "CMC Sales Staging - Restricted Access";
|
||||
|
||||
# Security headers
|
||||
add_header X-Frame-Options DENY;
|
||||
add_header X-Content-Type-Options nosniff;
|
||||
add_header X-XSS-Protection "1; mode=block";
|
||||
add_header Referrer-Policy "strict-origin-when-cross-origin";
|
||||
|
||||
# Staging banner
|
||||
add_header X-Environment "STAGING";
|
||||
|
||||
# CakePHP legacy app routes
|
||||
location / {
|
||||
proxy_pass http://cmc_php_staging;
|
||||
proxy_read_timeout 300s;
|
||||
proxy_connect_timeout 10s;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_set_header X-Environment "staging";
|
||||
}
|
||||
|
||||
# Go API routes
|
||||
location /api/ {
|
||||
proxy_pass http://cmc_go_staging;
|
||||
proxy_read_timeout 300s;
|
||||
proxy_connect_timeout 10s;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_set_header X-Environment "staging";
|
||||
}
|
||||
|
||||
# Go page routes for emails
|
||||
location ~ ^/(emails|customers|products|purchase-orders|enquiries|documents) {
|
||||
proxy_pass http://cmc_go_staging;
|
||||
proxy_read_timeout 300s;
|
||||
proxy_connect_timeout 10s;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_set_header X-Environment "staging";
|
||||
}
|
||||
|
||||
# Static files from Go app
|
||||
location /static/ {
|
||||
proxy_pass http://cmc_go_staging;
|
||||
proxy_cache_valid 200 1h;
|
||||
add_header Cache-Control "public, max-age=3600";
|
||||
}
|
||||
|
||||
# PDF files
|
||||
location /pdf/ {
|
||||
proxy_pass http://cmc_go_staging;
|
||||
proxy_cache_valid 200 1h;
|
||||
add_header Cache-Control "public, max-age=3600";
|
||||
}
|
||||
|
||||
# Health check endpoints
|
||||
location /health {
|
||||
proxy_pass http://cmc_go_staging/api/v1/health;
|
||||
access_log off;
|
||||
}
|
||||
|
||||
# Error pages
|
||||
error_page 502 503 504 /50x.html;
|
||||
location = /50x.html {
|
||||
root /usr/share/nginx/html;
|
||||
}
|
||||
|
||||
listen 80;
|
||||
}
|
||||
110
docker-compose.production.yml
Normal file
110
docker-compose.production.yml
Normal file
|
|
@ -0,0 +1,110 @@
|
|||
version: '3.8'
|
||||
|
||||
services:
|
||||
nginx-production:
|
||||
image: nginx:latest
|
||||
hostname: nginx-production
|
||||
container_name: cmc-nginx-production
|
||||
ports:
|
||||
- "8080:80" # Internal port for production
|
||||
volumes:
|
||||
- ./conf/nginx-production.conf:/etc/nginx/conf.d/cmc-production.conf
|
||||
- ./userpasswd:/etc/nginx/userpasswd:ro
|
||||
depends_on:
|
||||
- cmc-php-production
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- cmc-production-network
|
||||
environment:
|
||||
- NGINX_ENVSUBST_TEMPLATE_SUFFIX=.template
|
||||
|
||||
cmc-php-production:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
platform: linux/amd64
|
||||
container_name: cmc-php-production
|
||||
depends_on:
|
||||
- db-production
|
||||
volumes:
|
||||
- production_pdf_data:/var/www/cmc-sales/app/webroot/pdf
|
||||
- production_attachments_data:/var/www/cmc-sales/app/webroot/attachments_files
|
||||
networks:
|
||||
- cmc-production-network
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
- APP_ENV=production
|
||||
# Remove development features
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: '2.0'
|
||||
memory: 2G
|
||||
reservations:
|
||||
cpus: '0.5'
|
||||
memory: 512M
|
||||
|
||||
db-production:
|
||||
image: mariadb:latest
|
||||
container_name: cmc-db-production
|
||||
environment:
|
||||
MYSQL_ROOT_PASSWORD: ${DB_ROOT_PASSWORD_PRODUCTION}
|
||||
MYSQL_DATABASE: cmc
|
||||
MYSQL_USER: cmc
|
||||
MYSQL_PASSWORD: ${DB_PASSWORD_PRODUCTION}
|
||||
volumes:
|
||||
- production_db_data:/var/lib/mysql
|
||||
- ./backups:/backups:ro # Backup restore directory
|
||||
networks:
|
||||
- cmc-production-network
|
||||
restart: unless-stopped
|
||||
# No external port exposure for security
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: '2.0'
|
||||
memory: 4G
|
||||
reservations:
|
||||
cpus: '0.5'
|
||||
memory: 1G
|
||||
|
||||
cmc-go-production:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile.go.production
|
||||
container_name: cmc-go-production
|
||||
environment:
|
||||
DB_HOST: db-production
|
||||
DB_PORT: 3306
|
||||
DB_USER: cmc
|
||||
DB_PASSWORD: ${DB_PASSWORD_PRODUCTION}
|
||||
DB_NAME: cmc
|
||||
PORT: 8080
|
||||
APP_ENV: production
|
||||
depends_on:
|
||||
db-production:
|
||||
condition: service_started
|
||||
# No external port exposure - only through nginx
|
||||
volumes:
|
||||
- production_pdf_data:/root/webroot/pdf
|
||||
- ./credentials/production:/root/credentials:ro # Production Gmail credentials
|
||||
networks:
|
||||
- cmc-production-network
|
||||
restart: unless-stopped
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: '2.0'
|
||||
memory: 2G
|
||||
reservations:
|
||||
cpus: '0.5'
|
||||
memory: 512M
|
||||
|
||||
volumes:
|
||||
production_db_data:
|
||||
production_pdf_data:
|
||||
production_attachments_data:
|
||||
|
||||
networks:
|
||||
cmc-production-network:
|
||||
driver: bridge
|
||||
68
docker-compose.proxy.yml
Normal file
68
docker-compose.proxy.yml
Normal file
|
|
@ -0,0 +1,68 @@
|
|||
# Main reverse proxy for both staging and production
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
nginx-proxy:
|
||||
image: nginx:latest
|
||||
container_name: cmc-nginx-proxy
|
||||
ports:
|
||||
- "80:80"
|
||||
- "443:443"
|
||||
volumes:
|
||||
- ./conf/nginx-proxy.conf:/etc/nginx/nginx.conf
|
||||
- lego_certificates:/etc/ssl/certs:ro
|
||||
- lego_acme_challenge:/var/www/acme-challenge:ro
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
- nginx-staging
|
||||
- nginx-production
|
||||
networks:
|
||||
- proxy-network
|
||||
- cmc-staging-network
|
||||
- cmc-production-network
|
||||
|
||||
lego:
|
||||
image: goacme/lego:latest
|
||||
container_name: cmc-lego
|
||||
volumes:
|
||||
- lego_certificates:/data/certificates
|
||||
- lego_accounts:/data/accounts
|
||||
- lego_acme_challenge:/data/acme-challenge
|
||||
- ./scripts:/scripts:ro
|
||||
environment:
|
||||
- LEGO_DISABLE_CNAME=true
|
||||
command: sleep infinity
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- proxy-network
|
||||
|
||||
# Import staging services
|
||||
nginx-staging:
|
||||
extends:
|
||||
file: docker-compose.staging.yml
|
||||
service: nginx-staging
|
||||
networks:
|
||||
- proxy-network
|
||||
- cmc-staging-network
|
||||
|
||||
# Import production services
|
||||
nginx-production:
|
||||
extends:
|
||||
file: docker-compose.production.yml
|
||||
service: nginx-production
|
||||
networks:
|
||||
- proxy-network
|
||||
- cmc-production-network
|
||||
|
||||
volumes:
|
||||
lego_certificates:
|
||||
lego_accounts:
|
||||
lego_acme_challenge:
|
||||
|
||||
networks:
|
||||
proxy-network:
|
||||
driver: bridge
|
||||
cmc-staging-network:
|
||||
external: true
|
||||
cmc-production-network:
|
||||
external: true
|
||||
86
docker-compose.staging.yml
Normal file
86
docker-compose.staging.yml
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
version: '3.8'
|
||||
|
||||
services:
|
||||
nginx-staging:
|
||||
image: nginx:latest
|
||||
hostname: nginx-staging
|
||||
container_name: cmc-nginx-staging
|
||||
ports:
|
||||
- "8081:80" # Internal port for staging
|
||||
volumes:
|
||||
- ./conf/nginx-staging.conf:/etc/nginx/conf.d/cmc-staging.conf
|
||||
- ./userpasswd:/etc/nginx/userpasswd:ro
|
||||
depends_on:
|
||||
- cmc-php-staging
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- cmc-staging-network
|
||||
environment:
|
||||
- NGINX_ENVSUBST_TEMPLATE_SUFFIX=.template
|
||||
|
||||
cmc-php-staging:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
platform: linux/amd64
|
||||
container_name: cmc-php-staging
|
||||
depends_on:
|
||||
- db-staging
|
||||
volumes:
|
||||
- staging_pdf_data:/var/www/cmc-sales/app/webroot/pdf
|
||||
- staging_attachments_data:/var/www/cmc-sales/app/webroot/attachments_files
|
||||
networks:
|
||||
- cmc-staging-network
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
- APP_ENV=staging
|
||||
|
||||
db-staging:
|
||||
image: mariadb:latest
|
||||
container_name: cmc-db-staging
|
||||
environment:
|
||||
MYSQL_ROOT_PASSWORD: ${DB_ROOT_PASSWORD_STAGING}
|
||||
MYSQL_DATABASE: cmc_staging
|
||||
MYSQL_USER: cmc_staging
|
||||
MYSQL_PASSWORD: ${DB_PASSWORD_STAGING}
|
||||
volumes:
|
||||
- staging_db_data:/var/lib/mysql
|
||||
networks:
|
||||
- cmc-staging-network
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "3307:3306" # Different port for staging DB access
|
||||
|
||||
cmc-go-staging:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile.go.staging
|
||||
container_name: cmc-go-staging
|
||||
environment:
|
||||
DB_HOST: db-staging
|
||||
DB_PORT: 3306
|
||||
DB_USER: cmc_staging
|
||||
DB_PASSWORD: ${DB_PASSWORD_STAGING}
|
||||
DB_NAME: cmc_staging
|
||||
PORT: 8080
|
||||
APP_ENV: staging
|
||||
depends_on:
|
||||
db-staging:
|
||||
condition: service_started
|
||||
ports:
|
||||
- "8082:8080" # Direct access for testing
|
||||
volumes:
|
||||
- staging_pdf_data:/root/webroot/pdf
|
||||
- ./credentials/staging:/root/credentials:ro # Staging Gmail credentials
|
||||
networks:
|
||||
- cmc-staging-network
|
||||
restart: unless-stopped
|
||||
|
||||
volumes:
|
||||
staging_db_data:
|
||||
staging_pdf_data:
|
||||
staging_attachments_data:
|
||||
|
||||
networks:
|
||||
cmc-staging-network:
|
||||
driver: bridge
|
||||
9
go-app/.gitignore
vendored
9
go-app/.gitignore
vendored
|
|
@ -30,4 +30,11 @@ vendor/
|
|||
|
||||
# OS specific files
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
Thumbs.db
|
||||
|
||||
# Goose database migration config
|
||||
goose.env
|
||||
|
||||
# Gmail OAuth credentials - NEVER commit these!
|
||||
credentials.json
|
||||
token.json
|
||||
|
|
|
|||
70
go-app/MIGRATIONS.md
Normal file
70
go-app/MIGRATIONS.md
Normal file
|
|
@ -0,0 +1,70 @@
|
|||
# Database Migrations with Goose
|
||||
|
||||
This document explains how to use goose for database migrations in the CMC Sales Go application.
|
||||
|
||||
## Setup
|
||||
|
||||
1. **Install goose**:
|
||||
```bash
|
||||
make install
|
||||
```
|
||||
|
||||
2. **Configure database connection**:
|
||||
```bash
|
||||
cp goose.env.example goose.env
|
||||
# Edit goose.env with your database credentials
|
||||
```
|
||||
|
||||
## Migration Commands
|
||||
|
||||
### Run Migrations
|
||||
```bash
|
||||
# Run all pending migrations
|
||||
make migrate
|
||||
|
||||
# Check migration status
|
||||
make migrate-status
|
||||
```
|
||||
|
||||
### Rollback Migrations
|
||||
```bash
|
||||
# Rollback the last migration
|
||||
make migrate-down
|
||||
```
|
||||
|
||||
### Create New Migrations
|
||||
```bash
|
||||
# Create a new migration file
|
||||
make migrate-create name=add_new_feature
|
||||
```
|
||||
|
||||
## Migration Structure
|
||||
|
||||
Migrations are stored in `sql/migrations/` and follow this naming convention:
|
||||
- `001_add_gmail_fields.sql`
|
||||
- `002_add_new_feature.sql`
|
||||
|
||||
Each migration file contains:
|
||||
```sql
|
||||
-- +goose Up
|
||||
-- Your upgrade SQL here
|
||||
|
||||
-- +goose Down
|
||||
-- Your rollback SQL here
|
||||
```
|
||||
|
||||
## Configuration Files
|
||||
|
||||
- `goose.env` - Database connection settings (gitignored)
|
||||
- `goose.env.example` - Template for goose.env
|
||||
|
||||
## Current Migrations
|
||||
|
||||
1. **001_add_gmail_fields.sql** - Adds Gmail integration fields to emails and email_attachments tables
|
||||
|
||||
## Tips
|
||||
|
||||
- Always test migrations on a backup database first
|
||||
- Use `make migrate-status` to check current state
|
||||
- Migrations are atomic - if one fails, none are applied
|
||||
- Each migration should be reversible with a corresponding Down section
|
||||
|
|
@ -11,6 +11,7 @@ install: ## Install dependencies
|
|||
go env -w GOPRIVATE=code.springupsoftware.com
|
||||
go mod download
|
||||
go install github.com/sqlc-dev/sqlc/cmd/sqlc@latest
|
||||
go install github.com/pressly/goose/v3/cmd/goose@latest
|
||||
|
||||
.PHONY: sqlc
|
||||
sqlc: ## Generate Go code from SQL queries
|
||||
|
|
@ -76,4 +77,48 @@ dbshell-root: ## Connect to MariaDB as root user
|
|||
exit 1; \
|
||||
else \
|
||||
docker compose exec -e MYSQL_PWD="$$DB_ROOT_PASSWORD" db mariadb -u root; \
|
||||
fi
|
||||
|
||||
.PHONY: migrate
|
||||
migrate: ## Run database migrations
|
||||
@echo "Running database migrations..."
|
||||
@if [ -f goose.env ]; then \
|
||||
export $$(cat goose.env | xargs) && goose up; \
|
||||
else \
|
||||
echo "Error: goose.env file not found"; \
|
||||
exit 1; \
|
||||
fi
|
||||
|
||||
.PHONY: migrate-down
|
||||
migrate-down: ## Rollback last migration
|
||||
@echo "Rolling back last migration..."
|
||||
@if [ -f goose.env ]; then \
|
||||
export $$(cat goose.env | xargs) && goose down; \
|
||||
else \
|
||||
echo "Error: goose.env file not found"; \
|
||||
exit 1; \
|
||||
fi
|
||||
|
||||
.PHONY: migrate-status
|
||||
migrate-status: ## Show migration status
|
||||
@echo "Migration status:"
|
||||
@if [ -f goose.env ]; then \
|
||||
export $$(cat goose.env | xargs) && goose status; \
|
||||
else \
|
||||
echo "Error: goose.env file not found"; \
|
||||
exit 1; \
|
||||
fi
|
||||
|
||||
.PHONY: migrate-create
|
||||
migrate-create: ## Create a new migration file (use: make migrate-create name=add_new_table)
|
||||
@if [ -z "$(name)" ]; then \
|
||||
echo "Error: Please provide a migration name. Usage: make migrate-create name=add_new_table"; \
|
||||
exit 1; \
|
||||
fi
|
||||
@echo "Creating new migration: $(name)"
|
||||
@if [ -f goose.env ]; then \
|
||||
export $$(cat goose.env | xargs) && goose create $(name) sql; \
|
||||
else \
|
||||
echo "Error: goose.env file not found"; \
|
||||
exit 1; \
|
||||
fi
|
||||
|
|
@ -58,12 +58,13 @@ func main() {
|
|||
purchaseOrderHandler := handlers.NewPurchaseOrderHandler(queries)
|
||||
enquiryHandler := handlers.NewEnquiryHandler(queries)
|
||||
documentHandler := handlers.NewDocumentHandler(queries)
|
||||
pageHandler := handlers.NewPageHandler(queries, tmpl)
|
||||
pageHandler := handlers.NewPageHandler(queries, tmpl, database)
|
||||
addressHandler := handlers.NewAddressHandler(queries)
|
||||
attachmentHandler := handlers.NewAttachmentHandler(queries)
|
||||
countryHandler := handlers.NewCountryHandler(queries)
|
||||
statusHandler := handlers.NewStatusHandler(queries)
|
||||
lineItemHandler := handlers.NewLineItemHandler(queries)
|
||||
emailHandler := handlers.NewEmailHandler(queries, database)
|
||||
|
||||
// Setup routes
|
||||
r := mux.NewRouter()
|
||||
|
|
@ -161,6 +162,14 @@ func main() {
|
|||
api.HandleFunc("/line-items/{id}", lineItemHandler.Delete).Methods("DELETE")
|
||||
api.HandleFunc("/line-items/document/{documentID}/table", lineItemHandler.GetTable).Methods("GET")
|
||||
|
||||
// Email routes
|
||||
api.HandleFunc("/emails", emailHandler.List).Methods("GET")
|
||||
api.HandleFunc("/emails/{id}", emailHandler.Get).Methods("GET")
|
||||
api.HandleFunc("/emails/{id}/content", emailHandler.StreamContent).Methods("GET")
|
||||
api.HandleFunc("/emails/{id}/attachments", emailHandler.ListAttachments).Methods("GET")
|
||||
api.HandleFunc("/emails/{id}/attachments/{attachmentId}/stream", emailHandler.StreamAttachment).Methods("GET")
|
||||
api.HandleFunc("/emails/search", emailHandler.Search).Methods("GET")
|
||||
|
||||
// Health check
|
||||
api.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
|
|
@ -208,6 +217,12 @@ func main() {
|
|||
r.HandleFunc("/documents/{id}", pageHandler.DocumentsShow).Methods("GET")
|
||||
r.HandleFunc("/documents/pdf/{id}", documentHandler.GeneratePDF).Methods("GET")
|
||||
|
||||
// Email pages
|
||||
r.HandleFunc("/emails", pageHandler.EmailsIndex).Methods("GET")
|
||||
r.HandleFunc("/emails/search", pageHandler.EmailsSearch).Methods("GET")
|
||||
r.HandleFunc("/emails/{id}", pageHandler.EmailsShow).Methods("GET")
|
||||
r.HandleFunc("/emails/{id}/attachments", pageHandler.EmailsAttachments).Methods("GET")
|
||||
|
||||
// Address routes (matching CakePHP)
|
||||
r.HandleFunc("/addresses", addressHandler.List).Methods("GET")
|
||||
r.HandleFunc("/addresses/view/{id}", addressHandler.Get).Methods("GET")
|
||||
|
|
|
|||
|
|
@ -1,28 +1,23 @@
|
|||
# Vault Email Processor
|
||||
# Vault Email Processor - Smart Proxy
|
||||
|
||||
This is a Go rewrite of the PHP vault.php script that processes emails for the CMC Sales system.
|
||||
This is a Go rewrite of the PHP vault.php script that processes emails for the CMC Sales system. It now supports three modes: local file processing, Gmail indexing, and HTTP streaming proxy.
|
||||
|
||||
## Key Changes from PHP Version
|
||||
## Key Features
|
||||
|
||||
1. **No ripmime dependency**: Uses the enmime Go library for MIME parsing instead of external ripmime binary
|
||||
2. **Better error handling**: Proper error handling and database transactions
|
||||
3. **Type safety**: Strongly typed Go structures
|
||||
4. **Modern email parsing**: Uses enmime for robust email parsing
|
||||
1. **Gmail Integration**: Index Gmail emails without downloading
|
||||
2. **Smart Proxy**: Stream email content on-demand without storing to disk
|
||||
3. **No ripmime dependency**: Uses the enmime Go library for MIME parsing
|
||||
4. **Better error handling**: Proper error handling and database transactions
|
||||
5. **Type safety**: Strongly typed Go structures
|
||||
6. **Modern email parsing**: Uses enmime for robust email parsing
|
||||
|
||||
## Features
|
||||
## Operating Modes
|
||||
|
||||
- Processes emails from vault directory
|
||||
- Parses email headers and extracts recipients
|
||||
- Identifies and creates users as needed
|
||||
- Extracts attachments and email body parts
|
||||
- Matches document identifiers in subjects (enquiries, invoices, POs, jobs)
|
||||
- Saves emails with all associations to database
|
||||
- Moves processed emails to processed directory
|
||||
|
||||
## Usage
|
||||
### 1. Local Mode (Original functionality)
|
||||
Processes emails from local filesystem directories.
|
||||
|
||||
```bash
|
||||
go run cmd/vault/main.go \
|
||||
./vault --mode=local \
|
||||
--emaildir=/var/www/emails \
|
||||
--vaultdir=/var/www/vaultmsgs/new \
|
||||
--processeddir=/var/www/vaultmsgs/cur \
|
||||
|
|
@ -32,6 +27,83 @@ go run cmd/vault/main.go \
|
|||
--dbname=cmc
|
||||
```
|
||||
|
||||
### 2. Gmail Index Mode
|
||||
Indexes Gmail emails without downloading content. Creates database references only.
|
||||
|
||||
```bash
|
||||
./vault --mode=index \
|
||||
--gmail-query="is:unread" \
|
||||
--credentials=credentials.json \
|
||||
--token=token.json \
|
||||
--dbhost=127.0.0.1 \
|
||||
--dbuser=cmc \
|
||||
--dbpass="xVRQI&cA?7AU=hqJ!%au" \
|
||||
--dbname=cmc
|
||||
```
|
||||
|
||||
### 3. HTTP Server Mode
|
||||
Runs an HTTP server that streams Gmail content on-demand.
|
||||
|
||||
```bash
|
||||
./vault --mode=serve \
|
||||
--port=8080 \
|
||||
--credentials=credentials.json \
|
||||
--token=token.json \
|
||||
--dbhost=127.0.0.1 \
|
||||
--dbuser=cmc \
|
||||
--dbpass="xVRQI&cA?7AU=hqJ!%au" \
|
||||
--dbname=cmc
|
||||
```
|
||||
|
||||
## Gmail Setup
|
||||
|
||||
1. Enable Gmail API in Google Cloud Console
|
||||
2. Create OAuth 2.0 credentials
|
||||
3. Download credentials as `credentials.json`
|
||||
4. Run vault in any Gmail mode - it will prompt for authorization
|
||||
5. Token will be saved as `token.json` for future use
|
||||
|
||||
## API Endpoints (Server Mode)
|
||||
|
||||
- `GET /api/emails` - List indexed emails (metadata only)
|
||||
- `GET /api/emails/:id` - Get email metadata
|
||||
- `GET /api/emails/:id/content` - Stream email HTML/text from Gmail
|
||||
- `GET /api/emails/:id/attachments` - List attachment metadata
|
||||
- `GET /api/emails/:id/attachments/:attachmentId` - Stream attachment from Gmail
|
||||
- `GET /api/emails/:id/raw` - Stream raw email (for email clients)
|
||||
|
||||
## Database Schema Changes
|
||||
|
||||
Required migrations for Gmail support:
|
||||
|
||||
```sql
|
||||
ALTER TABLE emails
|
||||
ADD COLUMN gmail_message_id VARCHAR(255) UNIQUE,
|
||||
ADD COLUMN gmail_thread_id VARCHAR(255),
|
||||
ADD COLUMN is_downloaded BOOLEAN DEFAULT FALSE,
|
||||
ADD COLUMN raw_headers TEXT;
|
||||
|
||||
CREATE INDEX idx_gmail_message_id ON emails(gmail_message_id);
|
||||
|
||||
ALTER TABLE email_attachments
|
||||
ADD COLUMN gmail_attachment_id VARCHAR(255),
|
||||
ADD COLUMN gmail_message_id VARCHAR(255);
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
### Smart Proxy Benefits
|
||||
- **No Disk Storage**: Emails/attachments streamed directly from Gmail
|
||||
- **Low Storage Footprint**: Only metadata stored in database
|
||||
- **Fresh Content**: Always serves latest version from Gmail
|
||||
- **Scalable**: No file management overhead
|
||||
- **On-Demand**: Content fetched only when requested
|
||||
|
||||
### Processing Flow
|
||||
1. **Index Mode**: Scans Gmail, stores metadata, creates associations
|
||||
2. **Server Mode**: Receives HTTP requests, fetches from Gmail, streams to client
|
||||
3. **Local Mode**: Original file-based processing (backwards compatible)
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
|
|
@ -41,17 +113,28 @@ go build -o vault cmd/vault/main.go
|
|||
## Dependencies
|
||||
|
||||
- github.com/jhillyerd/enmime - MIME email parsing
|
||||
- github.com/google/uuid - UUID generation for unique filenames
|
||||
- github.com/google/uuid - UUID generation
|
||||
- github.com/go-sql-driver/mysql - MySQL driver
|
||||
- github.com/gorilla/mux - HTTP router
|
||||
- golang.org/x/oauth2 - OAuth2 support
|
||||
- google.golang.org/api/gmail/v1 - Gmail API client
|
||||
|
||||
## Database Tables Used
|
||||
|
||||
- emails - Main email records
|
||||
- emails - Main email records with Gmail metadata
|
||||
- email_recipients - To/CC recipients
|
||||
- email_attachments - File attachments
|
||||
- email_attachments - Attachment metadata (no file storage)
|
||||
- emails_enquiries - Email to enquiry associations
|
||||
- emails_invoices - Email to invoice associations
|
||||
- emails_purchase_orders - Email to PO associations
|
||||
- emails_jobs - Email to job associations
|
||||
- users - System users
|
||||
- enquiries, invoices, purchase_orders, jobs - For identifier matching
|
||||
- enquiries, invoices, purchase_orders, jobs - For identifier matching
|
||||
|
||||
## Gmail Query Examples
|
||||
|
||||
- `is:unread` - Unread emails
|
||||
- `newer_than:1d` - Emails from last 24 hours
|
||||
- `from:customer@example.com` - From specific sender
|
||||
- `subject:invoice` - Subject contains "invoice"
|
||||
- `has:attachment` - Emails with attachments
|
||||
|
|
@ -2,11 +2,15 @@ package main
|
|||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"log"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
|
|
@ -15,39 +19,82 @@ import (
|
|||
|
||||
_ "github.com/go-sql-driver/mysql"
|
||||
"github.com/google/uuid"
|
||||
"github.com/gorilla/mux"
|
||||
"github.com/jhillyerd/enmime"
|
||||
"golang.org/x/oauth2"
|
||||
"golang.org/x/oauth2/google"
|
||||
"google.golang.org/api/gmail/v1"
|
||||
"google.golang.org/api/option"
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
EmailDir string
|
||||
VaultDir string
|
||||
ProcessedDir string
|
||||
DBHost string
|
||||
DBUser string
|
||||
DBPassword string
|
||||
DBName string
|
||||
Mode string
|
||||
EmailDir string
|
||||
VaultDir string
|
||||
ProcessedDir string
|
||||
DBHost string
|
||||
DBUser string
|
||||
DBPassword string
|
||||
DBName string
|
||||
Port string
|
||||
CredentialsFile string
|
||||
TokenFile string
|
||||
GmailQuery string
|
||||
}
|
||||
|
||||
type VaultService struct {
|
||||
db *sql.DB
|
||||
config Config
|
||||
gmailService *gmail.Service
|
||||
indexer *GmailIndexer
|
||||
processor *EmailProcessor
|
||||
server *HTTPServer
|
||||
}
|
||||
|
||||
type EmailProcessor struct {
|
||||
db *sql.DB
|
||||
config Config
|
||||
enquiryMap map[string]int
|
||||
invoiceMap map[string]int
|
||||
poMap map[string]int
|
||||
userMap map[string]int
|
||||
jobMap map[string]int
|
||||
db *sql.DB
|
||||
config Config
|
||||
enquiryMap map[string]int
|
||||
invoiceMap map[string]int
|
||||
poMap map[string]int
|
||||
userMap map[string]int
|
||||
jobMap map[string]int
|
||||
}
|
||||
|
||||
type Attachment struct {
|
||||
Type string
|
||||
Name string
|
||||
Filename string
|
||||
Size int64
|
||||
IsMessageBody int
|
||||
type GmailIndexer struct {
|
||||
db *sql.DB
|
||||
gmailService *gmail.Service
|
||||
processor *EmailProcessor
|
||||
}
|
||||
|
||||
type HTTPServer struct {
|
||||
db *sql.DB
|
||||
gmailService *gmail.Service
|
||||
processor *EmailProcessor
|
||||
}
|
||||
|
||||
type EmailMetadata struct {
|
||||
Subject string
|
||||
From []string
|
||||
To []string
|
||||
CC []string
|
||||
Date time.Time
|
||||
GmailMessageID string
|
||||
GmailThreadID string
|
||||
AttachmentCount int
|
||||
Attachments []AttachmentMeta
|
||||
}
|
||||
|
||||
type AttachmentMeta struct {
|
||||
Filename string
|
||||
ContentType string
|
||||
Size int
|
||||
GmailAttachmentID string
|
||||
}
|
||||
|
||||
func main() {
|
||||
var config Config
|
||||
flag.StringVar(&config.Mode, "mode", "serve", "Mode: index, serve, or local")
|
||||
flag.StringVar(&config.EmailDir, "emaildir", "/var/www/emails", "Email storage directory")
|
||||
flag.StringVar(&config.VaultDir, "vaultdir", "/var/www/vaultmsgs/new", "Vault messages directory")
|
||||
flag.StringVar(&config.ProcessedDir, "processeddir", "/var/www/vaultmsgs/cur", "Processed messages directory")
|
||||
|
|
@ -55,17 +102,23 @@ func main() {
|
|||
flag.StringVar(&config.DBUser, "dbuser", "cmc", "Database user")
|
||||
flag.StringVar(&config.DBPassword, "dbpass", "xVRQI&cA?7AU=hqJ!%au", "Database password")
|
||||
flag.StringVar(&config.DBName, "dbname", "cmc", "Database name")
|
||||
flag.StringVar(&config.Port, "port", "8080", "HTTP server port")
|
||||
flag.StringVar(&config.CredentialsFile, "credentials", "credentials.json", "Gmail credentials file")
|
||||
flag.StringVar(&config.TokenFile, "token", "token.json", "Gmail token file")
|
||||
flag.StringVar(&config.GmailQuery, "gmail-query", "is:unread", "Gmail search query")
|
||||
flag.Parse()
|
||||
|
||||
// Connect to database
|
||||
dsn := fmt.Sprintf("%s:%s@tcp(%s)/%s?parseTime=true",
|
||||
config.DBUser, config.DBPassword, config.DBHost, config.DBName)
|
||||
|
||||
|
||||
db, err := sql.Open("mysql", dsn)
|
||||
if err != nil {
|
||||
log.Fatal("Failed to connect to database:", err)
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
// Create processor
|
||||
processor := &EmailProcessor{
|
||||
db: db,
|
||||
config: config,
|
||||
|
|
@ -75,11 +128,623 @@ func main() {
|
|||
log.Fatal("Failed to load maps:", err)
|
||||
}
|
||||
|
||||
if err := processor.processEmails(); err != nil {
|
||||
log.Fatal("Failed to process emails:", err)
|
||||
// Create vault service
|
||||
service := &VaultService{
|
||||
db: db,
|
||||
config: config,
|
||||
processor: processor,
|
||||
}
|
||||
|
||||
switch config.Mode {
|
||||
case "index":
|
||||
// Initialize Gmail service
|
||||
gmailService, err := getGmailService(config.CredentialsFile, config.TokenFile)
|
||||
if err != nil {
|
||||
log.Fatal("Failed to get Gmail service:", err)
|
||||
}
|
||||
service.gmailService = gmailService
|
||||
|
||||
// Create and run indexer
|
||||
service.indexer = &GmailIndexer{
|
||||
db: db,
|
||||
gmailService: gmailService,
|
||||
processor: processor,
|
||||
}
|
||||
|
||||
if err := service.indexer.IndexEmails(config.GmailQuery); err != nil {
|
||||
log.Fatal("Failed to index emails:", err)
|
||||
}
|
||||
|
||||
case "serve":
|
||||
// Initialize Gmail service
|
||||
gmailService, err := getGmailService(config.CredentialsFile, config.TokenFile)
|
||||
if err != nil {
|
||||
log.Fatal("Failed to get Gmail service:", err)
|
||||
}
|
||||
service.gmailService = gmailService
|
||||
|
||||
// Create and start HTTP server
|
||||
service.server = &HTTPServer{
|
||||
db: db,
|
||||
gmailService: gmailService,
|
||||
processor: processor,
|
||||
}
|
||||
|
||||
log.Printf("Starting HTTP server on port %s", config.Port)
|
||||
service.server.Start(config.Port)
|
||||
|
||||
case "local":
|
||||
// Original file-based processing
|
||||
if err := processor.processEmails(); err != nil {
|
||||
log.Fatal("Failed to process emails:", err)
|
||||
}
|
||||
|
||||
default:
|
||||
log.Fatal("Invalid mode. Use: index, serve, or local")
|
||||
}
|
||||
}
|
||||
|
||||
// Gmail OAuth2 functions
|
||||
func getGmailService(credentialsFile, tokenFile string) (*gmail.Service, error) {
|
||||
ctx := context.Background()
|
||||
|
||||
b, err := ioutil.ReadFile(credentialsFile)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("unable to read client secret file: %v", err)
|
||||
}
|
||||
|
||||
config, err := google.ConfigFromJSON(b, gmail.GmailReadonlyScope)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("unable to parse client secret file to config: %v", err)
|
||||
}
|
||||
|
||||
client := getClient(config, tokenFile)
|
||||
srv, err := gmail.NewService(ctx, option.WithHTTPClient(client))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("unable to retrieve Gmail client: %v", err)
|
||||
}
|
||||
|
||||
return srv, nil
|
||||
}
|
||||
|
||||
func getClient(config *oauth2.Config, tokFile string) *http.Client {
|
||||
tok, err := tokenFromFile(tokFile)
|
||||
if err != nil {
|
||||
tok = getTokenFromWeb(config)
|
||||
saveToken(tokFile, tok)
|
||||
}
|
||||
return config.Client(context.Background(), tok)
|
||||
}
|
||||
|
||||
func getTokenFromWeb(config *oauth2.Config) *oauth2.Token {
|
||||
authURL := config.AuthCodeURL("state-token", oauth2.AccessTypeOffline)
|
||||
fmt.Printf("Go to the following link in your browser then type the authorization code: \n%v\n", authURL)
|
||||
|
||||
var authCode string
|
||||
if _, err := fmt.Scan(&authCode); err != nil {
|
||||
log.Fatalf("Unable to read authorization code: %v", err)
|
||||
}
|
||||
|
||||
tok, err := config.Exchange(context.TODO(), authCode)
|
||||
if err != nil {
|
||||
log.Fatalf("Unable to retrieve token from web: %v", err)
|
||||
}
|
||||
return tok
|
||||
}
|
||||
|
||||
func tokenFromFile(file string) (*oauth2.Token, error) {
|
||||
f, err := os.Open(file)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer f.Close()
|
||||
tok := &oauth2.Token{}
|
||||
err = json.NewDecoder(f).Decode(tok)
|
||||
return tok, err
|
||||
}
|
||||
|
||||
func saveToken(path string, token *oauth2.Token) {
|
||||
fmt.Printf("Saving credential file to: %s\n", path)
|
||||
f, err := os.OpenFile(path, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0600)
|
||||
if err != nil {
|
||||
log.Fatalf("Unable to cache oauth token: %v", err)
|
||||
}
|
||||
defer f.Close()
|
||||
json.NewEncoder(f).Encode(token)
|
||||
}
|
||||
|
||||
// GmailIndexer methods
|
||||
func (g *GmailIndexer) IndexEmails(query string) error {
|
||||
user := "me"
|
||||
var pageToken string
|
||||
|
||||
for {
|
||||
// List messages with query
|
||||
call := g.gmailService.Users.Messages.List(user).Q(query).MaxResults(500)
|
||||
if pageToken != "" {
|
||||
call = call.PageToken(pageToken)
|
||||
}
|
||||
|
||||
response, err := call.Do()
|
||||
if err != nil {
|
||||
return fmt.Errorf("unable to retrieve messages: %v", err)
|
||||
}
|
||||
|
||||
// Process each message
|
||||
for _, msg := range response.Messages {
|
||||
if err := g.indexMessage(msg.Id); err != nil {
|
||||
log.Printf("Error indexing message %s: %v", msg.Id, err)
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
// Check for more pages
|
||||
pageToken = response.NextPageToken
|
||||
if pageToken == "" {
|
||||
break
|
||||
}
|
||||
|
||||
log.Printf("Processed %d messages, continuing with next page...", len(response.Messages))
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (g *GmailIndexer) indexMessage(messageID string) error {
|
||||
// Get message metadata only
|
||||
message, err := g.gmailService.Users.Messages.Get("me", messageID).
|
||||
Format("METADATA").
|
||||
MetadataHeaders("From", "To", "Cc", "Subject", "Date").
|
||||
Do()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Extract headers
|
||||
headers := make(map[string]string)
|
||||
for _, header := range message.Payload.Headers {
|
||||
headers[header.Name] = header.Value
|
||||
}
|
||||
|
||||
// Parse email addresses
|
||||
toRecipients := g.processor.parseEnmimeAddresses(headers["To"])
|
||||
fromRecipients := g.processor.parseEnmimeAddresses(headers["From"])
|
||||
ccRecipients := g.processor.parseEnmimeAddresses(headers["Cc"])
|
||||
|
||||
// Check if we should save this email
|
||||
saveThis := false
|
||||
fromKnownUser := false
|
||||
|
||||
for _, email := range toRecipients {
|
||||
if g.processor.userExists(email) {
|
||||
saveThis = true
|
||||
}
|
||||
}
|
||||
|
||||
for _, email := range fromRecipients {
|
||||
if g.processor.userExists(email) {
|
||||
saveThis = true
|
||||
fromKnownUser = true
|
||||
}
|
||||
}
|
||||
|
||||
for _, email := range ccRecipients {
|
||||
if g.processor.userExists(email) {
|
||||
saveThis = true
|
||||
}
|
||||
}
|
||||
|
||||
subject := headers["Subject"]
|
||||
if subject == "" {
|
||||
return nil // Skip emails without subject
|
||||
}
|
||||
|
||||
// Check for identifiers in subject
|
||||
foundEnquiries := g.processor.checkIdentifier(subject, g.processor.enquiryMap, "enquiry")
|
||||
foundInvoices := g.processor.checkIdentifier(subject, g.processor.invoiceMap, "invoice")
|
||||
foundPOs := g.processor.checkIdentifier(subject, g.processor.poMap, "purchaseorder")
|
||||
foundJobs := g.processor.checkIdentifier(subject, g.processor.jobMap, "job")
|
||||
|
||||
foundIdent := len(foundEnquiries) > 0 || len(foundInvoices) > 0 ||
|
||||
len(foundPOs) > 0 || len(foundJobs) > 0
|
||||
|
||||
if fromKnownUser || saveThis || foundIdent {
|
||||
// Parse date
|
||||
unixTime := time.Now().Unix()
|
||||
if dateStr := headers["Date"]; dateStr != "" {
|
||||
if t, err := time.Parse(time.RFC1123Z, dateStr); err == nil {
|
||||
unixTime = t.Unix()
|
||||
}
|
||||
}
|
||||
|
||||
// Get recipient user IDs
|
||||
recipientIDs := make(map[string][]int)
|
||||
recipientIDs["to"] = g.processor.getUserIDs(toRecipients)
|
||||
recipientIDs["from"] = g.processor.getUserIDs(fromRecipients)
|
||||
recipientIDs["cc"] = g.processor.getUserIDs(ccRecipients)
|
||||
|
||||
if len(recipientIDs["from"]) == 0 {
|
||||
return nil // Skip if no from recipient
|
||||
}
|
||||
|
||||
// Marshal headers for storage
|
||||
headerJSON, _ := json.Marshal(headers)
|
||||
|
||||
// Count attachments (from message metadata)
|
||||
attachmentCount := 0
|
||||
if message.Payload != nil {
|
||||
attachmentCount = countAttachments(message.Payload)
|
||||
}
|
||||
|
||||
fmt.Printf("Indexing message: %s - Subject: %s\n", messageID, subject)
|
||||
|
||||
// Save to database
|
||||
if err := g.saveEmailMetadata(messageID, message.ThreadId, subject, string(headerJSON),
|
||||
unixTime, recipientIDs, attachmentCount, message.Payload,
|
||||
foundEnquiries, foundInvoices, foundPOs, foundJobs); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func countAttachments(payload *gmail.MessagePart) int {
|
||||
count := 0
|
||||
if payload.Body != nil && payload.Body.AttachmentId != "" {
|
||||
count++
|
||||
}
|
||||
for _, part := range payload.Parts {
|
||||
count += countAttachments(part)
|
||||
}
|
||||
return count
|
||||
}
|
||||
|
||||
func (g *GmailIndexer) saveEmailMetadata(gmailMessageID, threadID, subject, headers string,
|
||||
unixTime int64, recipientIDs map[string][]int, attachmentCount int,
|
||||
payload *gmail.MessagePart, foundEnquiries, foundInvoices, foundPOs, foundJobs []int) error {
|
||||
|
||||
tx, err := g.db.Begin()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer tx.Rollback()
|
||||
|
||||
// Insert email
|
||||
result, err := tx.Exec(
|
||||
`INSERT INTO emails (user_id, udate, created, subject, gmail_message_id,
|
||||
gmail_thread_id, raw_headers, is_downloaded, email_attachment_count)
|
||||
VALUES (?, ?, NOW(), ?, ?, ?, ?, FALSE, ?)`,
|
||||
recipientIDs["from"][0], unixTime, subject, gmailMessageID,
|
||||
threadID, headers, attachmentCount)
|
||||
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
emailID, err := result.LastInsertId()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Insert recipients
|
||||
for recipType, userIDs := range recipientIDs {
|
||||
for _, userID := range userIDs {
|
||||
if recipType == "from" {
|
||||
continue // From is already stored in emails.user_id
|
||||
}
|
||||
_, err = tx.Exec(
|
||||
"INSERT INTO email_recipients (email_id, user_id, type) VALUES (?, ?, ?)",
|
||||
emailID, userID, recipType)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Index attachment metadata
|
||||
if payload != nil {
|
||||
if err := g.indexAttachments(tx, emailID, gmailMessageID, payload); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
// Insert associations
|
||||
for _, jobID := range foundJobs {
|
||||
_, err = tx.Exec("INSERT INTO emails_jobs (email_id, job_id) VALUES (?, ?)", emailID, jobID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
for _, poID := range foundPOs {
|
||||
_, err = tx.Exec("INSERT INTO emails_purchase_orders (email_id, purchase_order_id) VALUES (?, ?)", emailID, poID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
for _, enqID := range foundEnquiries {
|
||||
_, err = tx.Exec("INSERT INTO emails_enquiries (email_id, enquiry_id) VALUES (?, ?)", emailID, enqID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
for _, invID := range foundInvoices {
|
||||
_, err = tx.Exec("INSERT INTO emails_invoices (email_id, invoice_id) VALUES (?, ?)", emailID, invID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return tx.Commit()
|
||||
}
|
||||
|
||||
func (g *GmailIndexer) indexAttachments(tx *sql.Tx, emailID int64, gmailMessageID string, part *gmail.MessagePart) error {
|
||||
// Check if this part is an attachment
|
||||
if part.Body != nil && part.Body.AttachmentId != "" {
|
||||
filename := part.Filename
|
||||
if filename == "" {
|
||||
filename = "attachment"
|
||||
}
|
||||
|
||||
_, err := tx.Exec(
|
||||
`INSERT INTO email_attachments
|
||||
(email_id, gmail_attachment_id, gmail_message_id, type, size, filename, is_message_body, created)
|
||||
VALUES (?, ?, ?, ?, ?, ?, 0, NOW())`,
|
||||
emailID, part.Body.AttachmentId, gmailMessageID, part.MimeType, part.Body.Size, filename)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
// Process sub-parts
|
||||
for _, subPart := range part.Parts {
|
||||
if err := g.indexAttachments(tx, emailID, gmailMessageID, subPart); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// HTTPServer methods
|
||||
func (s *HTTPServer) Start(port string) {
|
||||
router := mux.NewRouter()
|
||||
|
||||
// API routes
|
||||
router.HandleFunc("/api/emails", s.ListEmails).Methods("GET")
|
||||
router.HandleFunc("/api/emails/{id:[0-9]+}", s.GetEmail).Methods("GET")
|
||||
router.HandleFunc("/api/emails/{id:[0-9]+}/content", s.StreamEmailContent).Methods("GET")
|
||||
router.HandleFunc("/api/emails/{id:[0-9]+}/attachments", s.ListAttachments).Methods("GET")
|
||||
router.HandleFunc("/api/emails/{id:[0-9]+}/attachments/{attachmentId:[0-9]+}", s.StreamAttachment).Methods("GET")
|
||||
router.HandleFunc("/api/emails/{id:[0-9]+}/raw", s.StreamRawEmail).Methods("GET")
|
||||
|
||||
log.Fatal(http.ListenAndServe(":"+port, router))
|
||||
}
|
||||
|
||||
func (s *HTTPServer) ListEmails(w http.ResponseWriter, r *http.Request) {
|
||||
// TODO: Add pagination
|
||||
rows, err := s.db.Query(`
|
||||
SELECT id, subject, user_id, created, gmail_message_id, email_attachment_count
|
||||
FROM emails
|
||||
ORDER BY created DESC
|
||||
LIMIT 100`)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var emails []map[string]interface{}
|
||||
for rows.Next() {
|
||||
var id, userID, attachmentCount int
|
||||
var subject, gmailMessageID string
|
||||
var created time.Time
|
||||
|
||||
if err := rows.Scan(&id, &subject, &userID, &created, &gmailMessageID, &attachmentCount); err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
emails = append(emails, map[string]interface{}{
|
||||
"id": id,
|
||||
"subject": subject,
|
||||
"user_id": userID,
|
||||
"created": created,
|
||||
"gmail_message_id": gmailMessageID,
|
||||
"attachment_count": attachmentCount,
|
||||
})
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(emails)
|
||||
}
|
||||
|
||||
func (s *HTTPServer) GetEmail(w http.ResponseWriter, r *http.Request) {
|
||||
vars := mux.Vars(r)
|
||||
emailID := vars["id"]
|
||||
|
||||
var gmailMessageID, subject, rawHeaders string
|
||||
var created time.Time
|
||||
|
||||
err := s.db.QueryRow(`
|
||||
SELECT gmail_message_id, subject, created, raw_headers
|
||||
FROM emails WHERE id = ?`, emailID).
|
||||
Scan(&gmailMessageID, &subject, &created, &rawHeaders)
|
||||
|
||||
if err != nil {
|
||||
http.Error(w, "Email not found", http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
|
||||
response := map[string]interface{}{
|
||||
"id": emailID,
|
||||
"gmail_message_id": gmailMessageID,
|
||||
"subject": subject,
|
||||
"created": created,
|
||||
"headers": rawHeaders,
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(response)
|
||||
}
|
||||
|
||||
func (s *HTTPServer) StreamEmailContent(w http.ResponseWriter, r *http.Request) {
|
||||
vars := mux.Vars(r)
|
||||
emailID := vars["id"]
|
||||
|
||||
// Get Gmail message ID from database
|
||||
var gmailMessageID string
|
||||
err := s.db.QueryRow("SELECT gmail_message_id FROM emails WHERE id = ?", emailID).
|
||||
Scan(&gmailMessageID)
|
||||
if err != nil {
|
||||
http.Error(w, "Email not found", http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
|
||||
// Fetch from Gmail
|
||||
message, err := s.gmailService.Users.Messages.Get("me", gmailMessageID).
|
||||
Format("RAW").Do()
|
||||
if err != nil {
|
||||
http.Error(w, "Failed to fetch email from Gmail", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
// Decode message
|
||||
rawEmail, err := base64.URLEncoding.DecodeString(message.Raw)
|
||||
if err != nil {
|
||||
http.Error(w, "Failed to decode email", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
// Parse with enmime
|
||||
env, err := enmime.ReadEnvelope(bytes.NewReader(rawEmail))
|
||||
if err != nil {
|
||||
http.Error(w, "Failed to parse email", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
// Stream HTML or Text directly to client
|
||||
if env.HTML != "" {
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
w.Write([]byte(env.HTML))
|
||||
} else if env.Text != "" {
|
||||
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
|
||||
w.Write([]byte(env.Text))
|
||||
} else {
|
||||
http.Error(w, "No content found in email", http.StatusNotFound)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *HTTPServer) ListAttachments(w http.ResponseWriter, r *http.Request) {
|
||||
vars := mux.Vars(r)
|
||||
emailID := vars["id"]
|
||||
|
||||
rows, err := s.db.Query(`
|
||||
SELECT id, filename, type, size, gmail_attachment_id
|
||||
FROM email_attachments
|
||||
WHERE email_id = ?`, emailID)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var attachments []map[string]interface{}
|
||||
for rows.Next() {
|
||||
var id, size int
|
||||
var filename, contentType, gmailAttachmentID string
|
||||
|
||||
if err := rows.Scan(&id, &filename, &contentType, &size, &gmailAttachmentID); err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
attachments = append(attachments, map[string]interface{}{
|
||||
"id": id,
|
||||
"filename": filename,
|
||||
"content_type": contentType,
|
||||
"size": size,
|
||||
"gmail_attachment_id": gmailAttachmentID,
|
||||
})
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(attachments)
|
||||
}
|
||||
|
||||
func (s *HTTPServer) StreamAttachment(w http.ResponseWriter, r *http.Request) {
|
||||
vars := mux.Vars(r)
|
||||
attachmentID := vars["attachmentId"]
|
||||
|
||||
// Get attachment info from database
|
||||
var gmailMessageID, gmailAttachmentID, filename, contentType string
|
||||
err := s.db.QueryRow(`
|
||||
SELECT gmail_message_id, gmail_attachment_id, filename, type
|
||||
FROM email_attachments
|
||||
WHERE id = ?`, attachmentID).
|
||||
Scan(&gmailMessageID, &gmailAttachmentID, &filename, &contentType)
|
||||
|
||||
if err != nil {
|
||||
http.Error(w, "Attachment not found", http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
|
||||
// Fetch from Gmail
|
||||
attachment, err := s.gmailService.Users.Messages.Attachments.
|
||||
Get("me", gmailMessageID, gmailAttachmentID).Do()
|
||||
if err != nil {
|
||||
http.Error(w, "Failed to fetch attachment from Gmail", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
// Decode base64
|
||||
data, err := base64.URLEncoding.DecodeString(attachment.Data)
|
||||
if err != nil {
|
||||
http.Error(w, "Failed to decode attachment", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
// Set headers and stream
|
||||
w.Header().Set("Content-Type", contentType)
|
||||
w.Header().Set("Content-Disposition", fmt.Sprintf("inline; filename=\"%s\"", filename))
|
||||
w.Header().Set("Content-Length", fmt.Sprintf("%d", len(data)))
|
||||
w.Write(data)
|
||||
}
|
||||
|
||||
func (s *HTTPServer) StreamRawEmail(w http.ResponseWriter, r *http.Request) {
|
||||
vars := mux.Vars(r)
|
||||
emailID := vars["id"]
|
||||
|
||||
// Get Gmail message ID
|
||||
var gmailMessageID string
|
||||
err := s.db.QueryRow("SELECT gmail_message_id FROM emails WHERE id = ?", emailID).
|
||||
Scan(&gmailMessageID)
|
||||
if err != nil {
|
||||
http.Error(w, "Email not found", http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
|
||||
// Fetch from Gmail
|
||||
message, err := s.gmailService.Users.Messages.Get("me", gmailMessageID).
|
||||
Format("RAW").Do()
|
||||
if err != nil {
|
||||
http.Error(w, "Failed to fetch email from Gmail", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
// Decode and stream
|
||||
rawEmail, err := base64.URLEncoding.DecodeString(message.Raw)
|
||||
if err != nil {
|
||||
http.Error(w, "Failed to decode email", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "message/rfc822")
|
||||
w.Write(rawEmail)
|
||||
}
|
||||
|
||||
// Original EmailProcessor methods (kept for local mode and shared logic)
|
||||
func (p *EmailProcessor) loadMaps() error {
|
||||
p.enquiryMap = make(map[string]int)
|
||||
p.invoiceMap = make(map[string]int)
|
||||
|
|
@ -243,7 +908,7 @@ func (p *EmailProcessor) processEmail(filename string) error {
|
|||
foundPOs := p.checkIdentifier(subject, p.poMap, "purchaseorder")
|
||||
foundJobs := p.checkIdentifier(subject, p.jobMap, "job")
|
||||
|
||||
foundIdent := len(foundEnquiries) > 0 || len(foundInvoices) > 0 ||
|
||||
foundIdent := len(foundEnquiries) > 0 || len(foundInvoices) > 0 ||
|
||||
len(foundPOs) > 0 || len(foundJobs) > 0
|
||||
|
||||
if fromKnownUser || saveThis || foundIdent {
|
||||
|
|
@ -327,11 +992,14 @@ func (p *EmailProcessor) getUserIDs(emails []string) []int {
|
|||
|
||||
func (p *EmailProcessor) createUser(email string) int {
|
||||
fmt.Printf("Making a new User for: '%s'\n", email)
|
||||
|
||||
|
||||
result, err := p.db.Exec(
|
||||
"INSERT INTO users (type, email, by_vault, created, modified) VALUES (?, ?, ?, NOW(), NOW())",
|
||||
"contact", strings.ToLower(email), 1)
|
||||
|
||||
`INSERT INTO users (principle_id, customer_id, type, access_level, username, password,
|
||||
first_name, last_name, email, job_title, phone, mobile, fax, phone_extension,
|
||||
direct_phone, notes, by_vault, blacklisted, enabled, primary_contact)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
0, 0, "contact", "none", "", "", "", "", strings.ToLower(email), "", "", "", "", "", "", "", 1, 0, 1, 0)
|
||||
|
||||
if err != nil {
|
||||
fmt.Printf("Serious Error: Unable to create user for email '%s': %v\n", email, err)
|
||||
return 0
|
||||
|
|
@ -407,7 +1075,7 @@ func (p *EmailProcessor) extractEnmimeAttachments(env *enmime.Envelope, relative
|
|||
fileName := "texthtml"
|
||||
newFileName := uuid + "-" + fileName
|
||||
filePath := filepath.Join(outputDir, newFileName)
|
||||
|
||||
|
||||
if err := ioutil.WriteFile(filePath, htmlData, 0644); err == nil {
|
||||
att := Attachment{
|
||||
Type: "text/html",
|
||||
|
|
@ -428,7 +1096,7 @@ func (p *EmailProcessor) extractEnmimeAttachments(env *enmime.Envelope, relative
|
|||
fileName := "textplain"
|
||||
newFileName := uuid + "-" + fileName
|
||||
filePath := filepath.Join(outputDir, newFileName)
|
||||
|
||||
|
||||
if err := ioutil.WriteFile(filePath, textData, 0644); err == nil {
|
||||
att := Attachment{
|
||||
Type: "text/plain",
|
||||
|
|
@ -449,10 +1117,10 @@ func (p *EmailProcessor) extractEnmimeAttachments(env *enmime.Envelope, relative
|
|||
if fileName == "" {
|
||||
fileName = "attachment"
|
||||
}
|
||||
|
||||
|
||||
newFileName := uuid + "-" + fileName
|
||||
filePath := filepath.Join(outputDir, newFileName)
|
||||
|
||||
|
||||
if err := ioutil.WriteFile(filePath, part.Content, 0644); err != nil {
|
||||
log.Printf("Failed to save attachment %s: %v", fileName, err)
|
||||
continue
|
||||
|
|
@ -485,10 +1153,10 @@ func (p *EmailProcessor) extractEnmimeAttachments(env *enmime.Envelope, relative
|
|||
if fileName == "" {
|
||||
fileName = "inline"
|
||||
}
|
||||
|
||||
|
||||
newFileName := uuid + "-" + fileName
|
||||
filePath := filepath.Join(outputDir, newFileName)
|
||||
|
||||
|
||||
if err := ioutil.WriteFile(filePath, part.Content, 0644); err != nil {
|
||||
log.Printf("Failed to save inline part %s: %v", fileName, err)
|
||||
continue
|
||||
|
|
@ -525,7 +1193,7 @@ func (p *EmailProcessor) extractEnmimeAttachments(env *enmime.Envelope, relative
|
|||
return attachments
|
||||
}
|
||||
|
||||
func (p *EmailProcessor) saveEmail(filename, subject string, unixTime int64,
|
||||
func (p *EmailProcessor) saveEmail(filename, subject string, unixTime int64,
|
||||
recipientIDs map[string][]int, attachments []Attachment,
|
||||
foundEnquiries, foundInvoices, foundPOs, foundJobs []int) error {
|
||||
|
||||
|
|
@ -537,9 +1205,9 @@ func (p *EmailProcessor) saveEmail(filename, subject string, unixTime int64,
|
|||
|
||||
// Insert email
|
||||
result, err := tx.Exec(
|
||||
"INSERT INTO emails (user_id, udate, created, subject, filename) VALUES (?, ?, NOW(), ?, ?)",
|
||||
recipientIDs["from"][0], unixTime, subject, filename)
|
||||
|
||||
"INSERT INTO emails (user_id, udate, created, subject) VALUES (?, ?, NOW(), ?)",
|
||||
recipientIDs["from"][0], unixTime, subject)
|
||||
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
|
@ -621,4 +1289,12 @@ func (p *EmailProcessor) moveEmail(filename string) error {
|
|||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
type Attachment struct {
|
||||
Type string
|
||||
Name string
|
||||
Filename string
|
||||
Size int64
|
||||
IsMessageBody int
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,6 +1,8 @@
|
|||
module code.springupsoftware.com/cmc/cmc-sales
|
||||
|
||||
go 1.23
|
||||
go 1.23.0
|
||||
|
||||
toolchain go1.24.4
|
||||
|
||||
require (
|
||||
github.com/go-sql-driver/mysql v1.7.1
|
||||
|
|
@ -9,17 +11,38 @@ require (
|
|||
github.com/jhillyerd/enmime v1.3.0
|
||||
github.com/joho/godotenv v1.5.1
|
||||
github.com/jung-kurt/gofpdf v1.16.2
|
||||
golang.org/x/oauth2 v0.30.0
|
||||
google.golang.org/api v0.244.0
|
||||
)
|
||||
|
||||
require (
|
||||
cloud.google.com/go/auth v0.16.3 // indirect
|
||||
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
|
||||
cloud.google.com/go/compute/metadata v0.7.0 // indirect
|
||||
github.com/cention-sany/utf7 v0.0.0-20170124080048-26cad61bd60a // indirect
|
||||
github.com/felixge/httpsnoop v1.0.4 // indirect
|
||||
github.com/go-logr/logr v1.4.3 // indirect
|
||||
github.com/go-logr/stdr v1.2.2 // indirect
|
||||
github.com/gogs/chardet v0.0.0-20211120154057-b7413eaefb8f // indirect
|
||||
github.com/google/s2a-go v0.1.9 // indirect
|
||||
github.com/googleapis/enterprise-certificate-proxy v0.3.6 // indirect
|
||||
github.com/googleapis/gax-go/v2 v2.15.0 // indirect
|
||||
github.com/jaytaylor/html2text v0.0.0-20230321000545-74c2419ad056 // indirect
|
||||
github.com/mattn/go-runewidth v0.0.15 // indirect
|
||||
github.com/olekukonko/tablewriter v0.0.5 // indirect
|
||||
github.com/pkg/errors v0.9.1 // indirect
|
||||
github.com/rivo/uniseg v0.4.4 // indirect
|
||||
github.com/ssor/bom v0.0.0-20170718123548-6386211fdfcf // indirect
|
||||
golang.org/x/net v0.23.0 // indirect
|
||||
golang.org/x/text v0.14.0 // indirect
|
||||
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
|
||||
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect
|
||||
go.opentelemetry.io/otel v1.36.0 // indirect
|
||||
go.opentelemetry.io/otel/metric v1.36.0 // indirect
|
||||
go.opentelemetry.io/otel/trace v1.36.0 // indirect
|
||||
golang.org/x/crypto v0.40.0 // indirect
|
||||
golang.org/x/net v0.42.0 // indirect
|
||||
golang.org/x/sys v0.34.0 // indirect
|
||||
golang.org/x/text v0.27.0 // indirect
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20250728155136-f173205681a0 // indirect
|
||||
google.golang.org/grpc v1.74.2 // indirect
|
||||
google.golang.org/protobuf v1.36.6 // indirect
|
||||
)
|
||||
|
|
|
|||
|
|
@ -1,13 +1,40 @@
|
|||
cloud.google.com/go/auth v0.16.3 h1:kabzoQ9/bobUmnseYnBO6qQG7q4a/CffFRlJSxv2wCc=
|
||||
cloud.google.com/go/auth v0.16.3/go.mod h1:NucRGjaXfzP1ltpcQ7On/VTZ0H4kWB5Jy+Y9Dnm76fA=
|
||||
cloud.google.com/go/auth/oauth2adapt v0.2.8 h1:keo8NaayQZ6wimpNSmW5OPc283g65QNIiLpZnkHRbnc=
|
||||
cloud.google.com/go/auth/oauth2adapt v0.2.8/go.mod h1:XQ9y31RkqZCcwJWNSx2Xvric3RrU88hAYYbjDWYDL+c=
|
||||
cloud.google.com/go/compute/metadata v0.7.0 h1:PBWF+iiAerVNe8UCHxdOt6eHLVc3ydFeOCw78U8ytSU=
|
||||
cloud.google.com/go/compute/metadata v0.7.0/go.mod h1:j5MvL9PprKL39t166CoB1uVHfQMs4tFQZZcKwksXUjo=
|
||||
github.com/boombuler/barcode v1.0.0/go.mod h1:paBWMcWSl3LHKBqUq+rly7CNSldXjb2rDl3JlRe0mD8=
|
||||
github.com/cention-sany/utf7 v0.0.0-20170124080048-26cad61bd60a h1:MISbI8sU/PSK/ztvmWKFcI7UGb5/HQT7B+i3a2myKgI=
|
||||
github.com/cention-sany/utf7 v0.0.0-20170124080048-26cad61bd60a/go.mod h1:2GxOXOlEPAMFPfp014mK1SWq8G8BN8o7/dfYqJrVGn8=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
|
||||
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
|
||||
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
|
||||
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
|
||||
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
|
||||
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
|
||||
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
|
||||
github.com/go-sql-driver/mysql v1.7.1 h1:lUIinVbN1DY0xBg0eMOzmmtGoHwWBbvnWubQUrtU8EI=
|
||||
github.com/go-sql-driver/mysql v1.7.1/go.mod h1:OXbVy3sEdcQ2Doequ6Z5BW6fXNQTmx+9S1MCJN5yJMI=
|
||||
github.com/go-test/deep v1.1.0 h1:WOcxcdHcvdgThNXjw0t76K42FXTU7HpNQWHpA2HHNlg=
|
||||
github.com/go-test/deep v1.1.0/go.mod h1:5C2ZWiW0ErCdrYzpqxLbTX7MG14M9iiw8DgHncVwcsE=
|
||||
github.com/gogs/chardet v0.0.0-20211120154057-b7413eaefb8f h1:3BSP1Tbs2djlpprl7wCLuiqMaUh5SJkkzI2gDs+FgLs=
|
||||
github.com/gogs/chardet v0.0.0-20211120154057-b7413eaefb8f/go.mod h1:Pcatq5tYkCW2Q6yrR2VRHlbHpZ/R4/7qyL1TCF7vl14=
|
||||
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
|
||||
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
|
||||
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
||||
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
|
||||
github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0=
|
||||
github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM=
|
||||
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
||||
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/googleapis/enterprise-certificate-proxy v0.3.6 h1:GW/XbdyBFQ8Qe+YAmFU9uHLo7OnF5tL52HFAgMmyrf4=
|
||||
github.com/googleapis/enterprise-certificate-proxy v0.3.6/go.mod h1:MkHOF77EYAE7qfSuSS9PU6g4Nt4e11cnsDUowfwewLA=
|
||||
github.com/googleapis/gax-go/v2 v2.15.0 h1:SyjDc1mGgZU5LncH8gimWo9lW1DtIfPibOG81vgd/bo=
|
||||
github.com/googleapis/gax-go/v2 v2.15.0/go.mod h1:zVVkkxAQHa1RQpg9z2AUCMnKhi0Qld9rcmyfL1OZhoc=
|
||||
github.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY=
|
||||
github.com/gorilla/mux v1.8.1/go.mod h1:AKf9I4AEqPTmMytcMc0KkNouC66V3BtZ4qD5fmWSiMQ=
|
||||
github.com/jaytaylor/html2text v0.0.0-20230321000545-74c2419ad056 h1:iCHtR9CQyktQ5+f3dMVZfwD2KWJUgm7M0gdL9NGr8KA=
|
||||
|
|
@ -28,6 +55,7 @@ github.com/phpdave11/gofpdi v1.0.7/go.mod h1:vBmVV0Do6hSBHC8uKUQ71JGW+ZGQq74llk/
|
|||
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
|
||||
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
|
||||
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
|
||||
github.com/rivo/uniseg v0.4.4 h1:8TfxU8dW6PdqD27gjM8MVNuicgxIjxpm4K7x4jp8sis=
|
||||
|
|
@ -36,9 +64,47 @@ github.com/ruudk/golang-pdf417 v0.0.0-20181029194003-1af4ab5afa58/go.mod h1:6lfF
|
|||
github.com/ssor/bom v0.0.0-20170718123548-6386211fdfcf h1:pvbZ0lM0XWPBqUKqFU8cmavspvIl9nulOYwdy6IFRRo=
|
||||
github.com/ssor/bom v0.0.0-20170718123548-6386211fdfcf/go.mod h1:RJID2RhlZKId02nZ62WenDCkgHFerpIOmW0iT7GKmXM=
|
||||
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
|
||||
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
|
||||
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
||||
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
|
||||
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
|
||||
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 h1:F7Jx+6hwnZ41NSFTO5q4LYDtJRXBf2PD0rNBkeB/lus=
|
||||
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0/go.mod h1:UHB22Z8QsdRDrnAtX4PntOl36ajSxcdUMt1sF7Y6E7Q=
|
||||
go.opentelemetry.io/otel v1.36.0 h1:UumtzIklRBY6cI/lllNZlALOF5nNIzJVb16APdvgTXg=
|
||||
go.opentelemetry.io/otel v1.36.0/go.mod h1:/TcFMXYjyRNh8khOAO9ybYkqaDBb/70aVwkNML4pP8E=
|
||||
go.opentelemetry.io/otel/metric v1.36.0 h1:MoWPKVhQvJ+eeXWHFBOPoBOi20jh6Iq2CcCREuTYufE=
|
||||
go.opentelemetry.io/otel/metric v1.36.0/go.mod h1:zC7Ks+yeyJt4xig9DEw9kuUFe5C3zLbVjV2PzT6qzbs=
|
||||
go.opentelemetry.io/otel/sdk v1.36.0 h1:b6SYIuLRs88ztox4EyrvRti80uXIFy+Sqzoh9kFULbs=
|
||||
go.opentelemetry.io/otel/sdk v1.36.0/go.mod h1:+lC+mTgD+MUWfjJubi2vvXWcVxyr9rmlshZni72pXeY=
|
||||
go.opentelemetry.io/otel/sdk/metric v1.36.0 h1:r0ntwwGosWGaa0CrSt8cuNuTcccMXERFwHX4dThiPis=
|
||||
go.opentelemetry.io/otel/sdk/metric v1.36.0/go.mod h1:qTNOhFDfKRwX0yXOqJYegL5WRaW376QbB7P4Pb0qva4=
|
||||
go.opentelemetry.io/otel/trace v1.36.0 h1:ahxWNuqZjpdiFAyrIoQ4GIiAIhxAunQR6MUoKrsNd4w=
|
||||
go.opentelemetry.io/otel/trace v1.36.0/go.mod h1:gQ+OnDZzrybY4k4seLzPAWNwVBBVlF2szhehOBB/tGA=
|
||||
golang.org/x/crypto v0.40.0 h1:r4x+VvoG5Fm+eJcxMaY8CQM7Lb0l1lsmjGBQ6s8BfKM=
|
||||
golang.org/x/crypto v0.40.0/go.mod h1:Qr1vMER5WyS2dfPHAlsOj01wgLbsyWtFn/aY+5+ZdxY=
|
||||
golang.org/x/image v0.0.0-20190910094157-69e4b8554b2a/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
|
||||
golang.org/x/net v0.23.0 h1:7EYJ93RZ9vYSZAIb2x3lnuvqO5zneoD6IvWjuhfxjTs=
|
||||
golang.org/x/net v0.23.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg=
|
||||
golang.org/x/net v0.42.0 h1:jzkYrhi3YQWD6MLBJcsklgQsoAcw89EcZbJw8Z614hs=
|
||||
golang.org/x/net v0.42.0/go.mod h1:FF1RA5d3u7nAYA4z2TkclSCKh68eSXtiFwcWQpPXdt8=
|
||||
golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI=
|
||||
golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU=
|
||||
golang.org/x/sync v0.16.0 h1:ycBJEhp9p4vXvUZNszeOq0kGTPghopOL8q0fq3vstxw=
|
||||
golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
|
||||
golang.org/x/sys v0.34.0 h1:H5Y5sJ2L2JRdyv7ROF1he/lPdvFsd0mJHFw2ThKHxLA=
|
||||
golang.org/x/sys v0.34.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
|
||||
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||
golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
|
||||
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
|
||||
golang.org/x/text v0.27.0 h1:4fGWRpyh641NLlecmyl4LOe6yDdfaYNrGb2zdfo4JV4=
|
||||
golang.org/x/text v0.27.0/go.mod h1:1D28KMCvyooCX9hBiosv5Tz/+YLxj0j7XhWjpSUF7CU=
|
||||
google.golang.org/api v0.244.0 h1:lpkP8wVibSKr++NCD36XzTk/IzeKJ3klj7vbj+XU5pE=
|
||||
google.golang.org/api v0.244.0/go.mod h1:dMVhVcylamkirHdzEBAIQWUCgqY885ivNeZYd7VAVr8=
|
||||
google.golang.org/genproto v0.0.0-20250603155806-513f23925822 h1:rHWScKit0gvAPuOnu87KpaYtjK5zBMLcULh7gxkCXu4=
|
||||
google.golang.org/genproto v0.0.0-20250603155806-513f23925822/go.mod h1:HubltRL7rMh0LfnQPkMH4NPDFEWp0jw3vixw7jEM53s=
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20250603155806-513f23925822 h1:oWVWY3NzT7KJppx2UKhKmzPq4SRe0LdCijVRwvGeikY=
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20250603155806-513f23925822/go.mod h1:h3c4v36UTKzUiuaOKQ6gr3S+0hovBtUrXzTG/i3+XEc=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20250728155136-f173205681a0 h1:MAKi5q709QWfnkkpNQ0M12hYJ1+e8qYVDyowc4U1XZM=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20250728155136-f173205681a0/go.mod h1:qQ0YXyHHx3XkvlzUtpXDkS29lDSafHMZBAZDc03LQ3A=
|
||||
google.golang.org/grpc v1.74.2 h1:WoosgB65DlWVC9FqI82dGsZhWFNBSLjQ84bjROOpMu4=
|
||||
google.golang.org/grpc v1.74.2/go.mod h1:CtQ+BGjaAIXHs/5YS3i473GqwBBa1zGQNevxdeBEXrM=
|
||||
google.golang.org/protobuf v1.36.6 h1:z1NpPI8ku2WgiWnf+t9wTPsn6eP1L7ksHUlkfLvd9xY=
|
||||
google.golang.org/protobuf v1.36.6/go.mod h1:jduwjTPXsFjZGTmRluh+L6NjiWu7pchiJ2/5YcXBHnY=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
|
|
|
|||
3
go-app/goose.env.example
Normal file
3
go-app/goose.env.example
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
GOOSE_DRIVER=mysql
|
||||
GOOSE_DBSTRING=username:password@tcp(localhost:3306)/database?parseTime=true
|
||||
GOOSE_MIGRATION_DIR=sql/migrations
|
||||
870
go-app/internal/cmc/handlers/emails.go
Normal file
870
go-app/internal/cmc/handlers/emails.go
Normal file
|
|
@ -0,0 +1,870 @@
|
|||
package handlers
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"net/http"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"code.springupsoftware.com/cmc/cmc-sales/internal/cmc/db"
|
||||
"github.com/gorilla/mux"
|
||||
"github.com/jhillyerd/enmime"
|
||||
"golang.org/x/oauth2"
|
||||
"golang.org/x/oauth2/google"
|
||||
"google.golang.org/api/gmail/v1"
|
||||
"google.golang.org/api/option"
|
||||
)
|
||||
|
||||
type EmailHandler struct {
|
||||
queries *db.Queries
|
||||
db *sql.DB
|
||||
gmailService *gmail.Service
|
||||
}
|
||||
|
||||
type EmailResponse struct {
|
||||
ID int32 `json:"id"`
|
||||
Subject string `json:"subject"`
|
||||
UserID int32 `json:"user_id"`
|
||||
Created time.Time `json:"created"`
|
||||
GmailMessageID *string `json:"gmail_message_id,omitempty"`
|
||||
AttachmentCount int32 `json:"attachment_count"`
|
||||
IsDownloaded *bool `json:"is_downloaded,omitempty"`
|
||||
}
|
||||
|
||||
type EmailDetailResponse struct {
|
||||
ID int32 `json:"id"`
|
||||
Subject string `json:"subject"`
|
||||
UserID int32 `json:"user_id"`
|
||||
Created time.Time `json:"created"`
|
||||
GmailMessageID *string `json:"gmail_message_id,omitempty"`
|
||||
GmailThreadID *string `json:"gmail_thread_id,omitempty"`
|
||||
RawHeaders *string `json:"raw_headers,omitempty"`
|
||||
IsDownloaded *bool `json:"is_downloaded,omitempty"`
|
||||
Enquiries []int32 `json:"enquiries,omitempty"`
|
||||
Invoices []int32 `json:"invoices,omitempty"`
|
||||
PurchaseOrders []int32 `json:"purchase_orders,omitempty"`
|
||||
Jobs []int32 `json:"jobs,omitempty"`
|
||||
}
|
||||
|
||||
type EmailAttachmentResponse struct {
|
||||
ID int32 `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Type string `json:"type"`
|
||||
Size int32 `json:"size"`
|
||||
Filename string `json:"filename"`
|
||||
IsMessageBody bool `json:"is_message_body"`
|
||||
GmailAttachmentID *string `json:"gmail_attachment_id,omitempty"`
|
||||
Created time.Time `json:"created"`
|
||||
}
|
||||
|
||||
func NewEmailHandler(queries *db.Queries, database *sql.DB) *EmailHandler {
|
||||
// Try to initialize Gmail service
|
||||
gmailService, err := getGmailService("credentials.json", "token.json")
|
||||
if err != nil {
|
||||
// Log the error but continue without Gmail service
|
||||
fmt.Printf("Warning: Gmail service not available: %v\n", err)
|
||||
}
|
||||
|
||||
return &EmailHandler{
|
||||
queries: queries,
|
||||
db: database,
|
||||
gmailService: gmailService,
|
||||
}
|
||||
}
|
||||
|
||||
// List emails with pagination and filtering
|
||||
func (h *EmailHandler) List(w http.ResponseWriter, r *http.Request) {
|
||||
// Parse query parameters
|
||||
limitStr := r.URL.Query().Get("limit")
|
||||
offsetStr := r.URL.Query().Get("offset")
|
||||
search := r.URL.Query().Get("search")
|
||||
userID := r.URL.Query().Get("user_id")
|
||||
|
||||
// Set defaults
|
||||
limit := 50
|
||||
offset := 0
|
||||
|
||||
if limitStr != "" {
|
||||
if l, err := strconv.Atoi(limitStr); err == nil && l > 0 && l <= 100 {
|
||||
limit = l
|
||||
}
|
||||
}
|
||||
|
||||
if offsetStr != "" {
|
||||
if o, err := strconv.Atoi(offsetStr); err == nil && o >= 0 {
|
||||
offset = o
|
||||
}
|
||||
}
|
||||
|
||||
// Build query
|
||||
query := `
|
||||
SELECT e.id, e.subject, e.user_id, e.created, e.gmail_message_id, e.email_attachment_count, e.is_downloaded
|
||||
FROM emails e`
|
||||
|
||||
var args []interface{}
|
||||
var conditions []string
|
||||
|
||||
if search != "" {
|
||||
conditions = append(conditions, "e.subject LIKE ?")
|
||||
args = append(args, "%"+search+"%")
|
||||
}
|
||||
|
||||
if userID != "" {
|
||||
conditions = append(conditions, "e.user_id = ?")
|
||||
args = append(args, userID)
|
||||
}
|
||||
|
||||
if len(conditions) > 0 {
|
||||
query += " WHERE " + joinConditions(conditions, " AND ")
|
||||
}
|
||||
|
||||
query += " ORDER BY e.id DESC LIMIT ? OFFSET ?"
|
||||
args = append(args, limit, offset)
|
||||
|
||||
rows, err := h.db.Query(query, args...)
|
||||
if err != nil {
|
||||
http.Error(w, fmt.Sprintf("Database error: %v", err), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var emails []EmailResponse
|
||||
for rows.Next() {
|
||||
var email EmailResponse
|
||||
var gmailMessageID sql.NullString
|
||||
var isDownloaded sql.NullBool
|
||||
|
||||
err := rows.Scan(
|
||||
&email.ID,
|
||||
&email.Subject,
|
||||
&email.UserID,
|
||||
&email.Created,
|
||||
&gmailMessageID,
|
||||
&email.AttachmentCount,
|
||||
&isDownloaded,
|
||||
)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
if gmailMessageID.Valid {
|
||||
email.GmailMessageID = &gmailMessageID.String
|
||||
}
|
||||
if isDownloaded.Valid {
|
||||
email.IsDownloaded = &isDownloaded.Bool
|
||||
}
|
||||
|
||||
emails = append(emails, email)
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(emails)
|
||||
}
|
||||
|
||||
// Get a specific email with details
|
||||
func (h *EmailHandler) Get(w http.ResponseWriter, r *http.Request) {
|
||||
vars := mux.Vars(r)
|
||||
emailID, err := strconv.Atoi(vars["id"])
|
||||
if err != nil {
|
||||
http.Error(w, "Invalid email ID", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
// Get email details
|
||||
query := `
|
||||
SELECT e.id, e.subject, e.user_id, e.created, e.gmail_message_id,
|
||||
e.gmail_thread_id, e.raw_headers, e.is_downloaded
|
||||
FROM emails e
|
||||
WHERE e.id = ?`
|
||||
|
||||
var email EmailDetailResponse
|
||||
var gmailMessageID, gmailThreadID, rawHeaders sql.NullString
|
||||
var isDownloaded sql.NullBool
|
||||
|
||||
err = h.db.QueryRow(query, emailID).Scan(
|
||||
&email.ID,
|
||||
&email.Subject,
|
||||
&email.UserID,
|
||||
&email.Created,
|
||||
&gmailMessageID,
|
||||
&gmailThreadID,
|
||||
&rawHeaders,
|
||||
&isDownloaded,
|
||||
)
|
||||
|
||||
if err != nil {
|
||||
if err == sql.ErrNoRows {
|
||||
http.Error(w, "Email not found", http.StatusNotFound)
|
||||
} else {
|
||||
http.Error(w, fmt.Sprintf("Database error: %v", err), http.StatusInternalServerError)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
if gmailMessageID.Valid {
|
||||
email.GmailMessageID = &gmailMessageID.String
|
||||
}
|
||||
if gmailThreadID.Valid {
|
||||
email.GmailThreadID = &gmailThreadID.String
|
||||
}
|
||||
if rawHeaders.Valid {
|
||||
email.RawHeaders = &rawHeaders.String
|
||||
}
|
||||
if isDownloaded.Valid {
|
||||
email.IsDownloaded = &isDownloaded.Bool
|
||||
}
|
||||
|
||||
// Get associated enquiries
|
||||
enquiryRows, err := h.db.Query("SELECT enquiry_id FROM emails_enquiries WHERE email_id = ?", emailID)
|
||||
if err == nil {
|
||||
defer enquiryRows.Close()
|
||||
for enquiryRows.Next() {
|
||||
var enquiryID int32
|
||||
if enquiryRows.Scan(&enquiryID) == nil {
|
||||
email.Enquiries = append(email.Enquiries, enquiryID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Get associated invoices
|
||||
invoiceRows, err := h.db.Query("SELECT invoice_id FROM emails_invoices WHERE email_id = ?", emailID)
|
||||
if err == nil {
|
||||
defer invoiceRows.Close()
|
||||
for invoiceRows.Next() {
|
||||
var invoiceID int32
|
||||
if invoiceRows.Scan(&invoiceID) == nil {
|
||||
email.Invoices = append(email.Invoices, invoiceID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Get associated purchase orders
|
||||
poRows, err := h.db.Query("SELECT purchase_order_id FROM emails_purchase_orders WHERE email_id = ?", emailID)
|
||||
if err == nil {
|
||||
defer poRows.Close()
|
||||
for poRows.Next() {
|
||||
var poID int32
|
||||
if poRows.Scan(&poID) == nil {
|
||||
email.PurchaseOrders = append(email.PurchaseOrders, poID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Get associated jobs
|
||||
jobRows, err := h.db.Query("SELECT job_id FROM emails_jobs WHERE email_id = ?", emailID)
|
||||
if err == nil {
|
||||
defer jobRows.Close()
|
||||
for jobRows.Next() {
|
||||
var jobID int32
|
||||
if jobRows.Scan(&jobID) == nil {
|
||||
email.Jobs = append(email.Jobs, jobID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(email)
|
||||
}
|
||||
|
||||
// List attachments for an email
|
||||
func (h *EmailHandler) ListAttachments(w http.ResponseWriter, r *http.Request) {
|
||||
vars := mux.Vars(r)
|
||||
emailID, err := strconv.Atoi(vars["id"])
|
||||
if err != nil {
|
||||
http.Error(w, "Invalid email ID", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
// First check if attachments are already in database
|
||||
query := `
|
||||
SELECT id, name, type, size, filename, is_message_body, gmail_attachment_id, created
|
||||
FROM email_attachments
|
||||
WHERE email_id = ?
|
||||
ORDER BY is_message_body DESC, created ASC`
|
||||
|
||||
rows, err := h.db.Query(query, emailID)
|
||||
if err != nil {
|
||||
http.Error(w, fmt.Sprintf("Database error: %v", err), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var attachments []EmailAttachmentResponse
|
||||
hasStoredAttachments := false
|
||||
|
||||
for rows.Next() {
|
||||
hasStoredAttachments = true
|
||||
var attachment EmailAttachmentResponse
|
||||
var gmailAttachmentID sql.NullString
|
||||
|
||||
err := rows.Scan(
|
||||
&attachment.ID,
|
||||
&attachment.Name,
|
||||
&attachment.Type,
|
||||
&attachment.Size,
|
||||
&attachment.Filename,
|
||||
&attachment.IsMessageBody,
|
||||
&gmailAttachmentID,
|
||||
&attachment.Created,
|
||||
)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
if gmailAttachmentID.Valid {
|
||||
attachment.GmailAttachmentID = &gmailAttachmentID.String
|
||||
}
|
||||
|
||||
attachments = append(attachments, attachment)
|
||||
}
|
||||
|
||||
// If no stored attachments and this is a Gmail email, try to fetch from Gmail
|
||||
if !hasStoredAttachments && h.gmailService != nil {
|
||||
// Get Gmail message ID
|
||||
var gmailMessageID sql.NullString
|
||||
err := h.db.QueryRow("SELECT gmail_message_id FROM emails WHERE id = ?", emailID).Scan(&gmailMessageID)
|
||||
|
||||
if err == nil && gmailMessageID.Valid {
|
||||
// Fetch message metadata from Gmail
|
||||
message, err := h.gmailService.Users.Messages.Get("me", gmailMessageID.String).
|
||||
Format("FULL").Do()
|
||||
|
||||
if err == nil && message.Payload != nil {
|
||||
// Extract attachment info from Gmail message
|
||||
attachmentIndex := int32(1)
|
||||
h.extractGmailAttachments(message.Payload, &attachments, &attachmentIndex)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Check if this is an HTMX request
|
||||
if r.Header.Get("HX-Request") == "true" {
|
||||
// Return HTML for HTMX
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
|
||||
if len(attachments) == 0 {
|
||||
// No attachments found
|
||||
html := `<div class="notification is-light">
|
||||
<p class="has-text-grey">No attachments found for this email.</p>
|
||||
</div>`
|
||||
w.Write([]byte(html))
|
||||
return
|
||||
}
|
||||
|
||||
// Build HTML table for attachments
|
||||
var htmlBuilder strings.Builder
|
||||
htmlBuilder.WriteString(`<div class="box">
|
||||
<h3 class="title is-5">Attachments</h3>
|
||||
<div class="table-container">
|
||||
<table class="table is-fullwidth">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
<th>Type</th>
|
||||
<th>Size</th>
|
||||
<th>Actions</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>`)
|
||||
|
||||
for _, att := range attachments {
|
||||
icon := `<i class="fas fa-paperclip"></i>`
|
||||
if att.IsMessageBody {
|
||||
icon = `<i class="fas fa-envelope"></i>`
|
||||
}
|
||||
|
||||
downloadURL := fmt.Sprintf("/api/v1/emails/%d/attachments/%d", emailID, att.ID)
|
||||
if att.GmailAttachmentID != nil {
|
||||
downloadURL = fmt.Sprintf("/api/v1/emails/%d/attachments/%d/stream", emailID, att.ID)
|
||||
}
|
||||
|
||||
htmlBuilder.WriteString(fmt.Sprintf(`
|
||||
<tr>
|
||||
<td>
|
||||
<span class="icon has-text-grey">%s</span>
|
||||
%s
|
||||
</td>
|
||||
<td><span class="tag is-light">%s</span></td>
|
||||
<td>%d bytes</td>
|
||||
<td>
|
||||
<a href="%s" target="_blank" class="button is-small is-info">
|
||||
<span class="icon"><i class="fas fa-download"></i></span>
|
||||
<span>Download</span>
|
||||
</a>
|
||||
</td>
|
||||
</tr>`, icon, att.Name, att.Type, att.Size, downloadURL))
|
||||
}
|
||||
|
||||
htmlBuilder.WriteString(`
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>`)
|
||||
|
||||
w.Write([]byte(htmlBuilder.String()))
|
||||
return
|
||||
}
|
||||
|
||||
// Return JSON for API requests
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(attachments)
|
||||
}
|
||||
|
||||
// Helper function to extract attachment info from Gmail message parts
|
||||
func (h *EmailHandler) extractGmailAttachments(part *gmail.MessagePart, attachments *[]EmailAttachmentResponse, index *int32) {
|
||||
// Check if this part is an attachment
|
||||
// Some attachments may not have filenames or may be inline
|
||||
if part.Body != nil && part.Body.AttachmentId != "" {
|
||||
filename := part.Filename
|
||||
if filename == "" {
|
||||
// Try to generate a filename from content type
|
||||
switch part.MimeType {
|
||||
case "application/pdf":
|
||||
filename = "attachment.pdf"
|
||||
case "image/png":
|
||||
filename = "image.png"
|
||||
case "image/jpeg":
|
||||
filename = "image.jpg"
|
||||
case "text/plain":
|
||||
filename = "text.txt"
|
||||
default:
|
||||
filename = "attachment"
|
||||
}
|
||||
}
|
||||
|
||||
attachment := EmailAttachmentResponse{
|
||||
ID: *index,
|
||||
Name: filename,
|
||||
Type: part.MimeType,
|
||||
Size: int32(part.Body.Size),
|
||||
Filename: filename,
|
||||
IsMessageBody: false,
|
||||
GmailAttachmentID: &part.Body.AttachmentId,
|
||||
Created: time.Now(), // Use current time as placeholder
|
||||
}
|
||||
*attachments = append(*attachments, attachment)
|
||||
*index++
|
||||
}
|
||||
|
||||
// Process sub-parts
|
||||
for _, subPart := range part.Parts {
|
||||
h.extractGmailAttachments(subPart, attachments, index)
|
||||
}
|
||||
}
|
||||
|
||||
// Search emails
|
||||
func (h *EmailHandler) Search(w http.ResponseWriter, r *http.Request) {
|
||||
query := r.URL.Query().Get("q")
|
||||
if query == "" {
|
||||
http.Error(w, "Search query is required", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
// Parse optional parameters
|
||||
limitStr := r.URL.Query().Get("limit")
|
||||
limit := 20
|
||||
|
||||
if limitStr != "" {
|
||||
if l, err := strconv.Atoi(limitStr); err == nil && l > 0 && l <= 100 {
|
||||
limit = l
|
||||
}
|
||||
}
|
||||
|
||||
// Search in subjects and headers
|
||||
sqlQuery := `
|
||||
SELECT e.id, e.subject, e.user_id, e.created, e.gmail_message_id, e.email_attachment_count, e.is_downloaded
|
||||
FROM emails e
|
||||
WHERE e.subject LIKE ? OR e.raw_headers LIKE ?
|
||||
ORDER BY e.id DESC
|
||||
LIMIT ?`
|
||||
|
||||
searchTerm := "%" + query + "%"
|
||||
rows, err := h.db.Query(sqlQuery, searchTerm, searchTerm, limit)
|
||||
if err != nil {
|
||||
http.Error(w, fmt.Sprintf("Database error: %v", err), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var emails []EmailResponse
|
||||
for rows.Next() {
|
||||
var email EmailResponse
|
||||
var gmailMessageID sql.NullString
|
||||
var isDownloaded sql.NullBool
|
||||
|
||||
err := rows.Scan(
|
||||
&email.ID,
|
||||
&email.Subject,
|
||||
&email.UserID,
|
||||
&email.Created,
|
||||
&gmailMessageID,
|
||||
&email.AttachmentCount,
|
||||
&isDownloaded,
|
||||
)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
if gmailMessageID.Valid {
|
||||
email.GmailMessageID = &gmailMessageID.String
|
||||
}
|
||||
if isDownloaded.Valid {
|
||||
email.IsDownloaded = &isDownloaded.Bool
|
||||
}
|
||||
|
||||
emails = append(emails, email)
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(emails)
|
||||
}
|
||||
|
||||
// Stream email content from Gmail
|
||||
func (h *EmailHandler) StreamContent(w http.ResponseWriter, r *http.Request) {
|
||||
vars := mux.Vars(r)
|
||||
emailID, err := strconv.Atoi(vars["id"])
|
||||
if err != nil {
|
||||
http.Error(w, "Invalid email ID", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
// Get email details to check if it's a Gmail email
|
||||
query := `
|
||||
SELECT e.gmail_message_id, e.subject, e.created, e.user_id
|
||||
FROM emails e
|
||||
WHERE e.id = ?`
|
||||
|
||||
var gmailMessageID sql.NullString
|
||||
var subject string
|
||||
var created time.Time
|
||||
var userID int32
|
||||
|
||||
err = h.db.QueryRow(query, emailID).Scan(&gmailMessageID, &subject, &created, &userID)
|
||||
if err != nil {
|
||||
if err == sql.ErrNoRows {
|
||||
http.Error(w, "Email not found", http.StatusNotFound)
|
||||
} else {
|
||||
http.Error(w, fmt.Sprintf("Database error: %v", err), http.StatusInternalServerError)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
if !gmailMessageID.Valid {
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
html := `
|
||||
<div class="notification is-warning">
|
||||
<strong>Local Email</strong><br>
|
||||
This email is not from Gmail and does not have stored content available for display.
|
||||
</div>`
|
||||
w.Write([]byte(html))
|
||||
return
|
||||
}
|
||||
|
||||
// Check for stored message body content in attachments
|
||||
attachmentQuery := `
|
||||
SELECT id, name, type, size
|
||||
FROM email_attachments
|
||||
WHERE email_id = ? AND is_message_body = 1
|
||||
ORDER BY created ASC`
|
||||
|
||||
attachmentRows, err := h.db.Query(attachmentQuery, emailID)
|
||||
if err == nil {
|
||||
defer attachmentRows.Close()
|
||||
if attachmentRows.Next() {
|
||||
var attachmentID int32
|
||||
var name, attachmentType string
|
||||
var size int32
|
||||
|
||||
if attachmentRows.Scan(&attachmentID, &name, &attachmentType, &size) == nil {
|
||||
// Found stored message body - would normally read the content from file storage
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
html := fmt.Sprintf(`
|
||||
<div class="content">
|
||||
<div class="notification is-success is-light">
|
||||
<strong>Stored Email Content</strong><br>
|
||||
Message body is stored locally as attachment: %s (%s, %d bytes)
|
||||
</div>
|
||||
<div class="box">
|
||||
<p><em>Content would be loaded from local storage here.</em></p>
|
||||
<p>Attachment ID: %d</p>
|
||||
</div>
|
||||
</div>`, name, attachmentType, size, attachmentID)
|
||||
w.Write([]byte(html))
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Try to fetch from Gmail if service is available
|
||||
if h.gmailService != nil {
|
||||
// Fetch from Gmail
|
||||
message, err := h.gmailService.Users.Messages.Get("me", gmailMessageID.String).
|
||||
Format("RAW").Do()
|
||||
if err != nil {
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
html := fmt.Sprintf(`
|
||||
<div class="notification is-danger">
|
||||
<strong>Gmail API Error</strong><br>
|
||||
Failed to fetch email from Gmail: %v
|
||||
</div>`, err)
|
||||
w.Write([]byte(html))
|
||||
return
|
||||
}
|
||||
|
||||
// Decode message
|
||||
rawEmail, err := base64.URLEncoding.DecodeString(message.Raw)
|
||||
if err != nil {
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
html := fmt.Sprintf(`
|
||||
<div class="notification is-danger">
|
||||
<strong>Decode Error</strong><br>
|
||||
Failed to decode email: %v
|
||||
</div>`, err)
|
||||
w.Write([]byte(html))
|
||||
return
|
||||
}
|
||||
|
||||
// Parse with enmime
|
||||
env, err := enmime.ReadEnvelope(bytes.NewReader(rawEmail))
|
||||
if err != nil {
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
html := fmt.Sprintf(`
|
||||
<div class="notification is-danger">
|
||||
<strong>Parse Error</strong><br>
|
||||
Failed to parse email: %v
|
||||
</div>`, err)
|
||||
w.Write([]byte(html))
|
||||
return
|
||||
}
|
||||
|
||||
// Stream HTML or Text directly to client
|
||||
if env.HTML != "" {
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
w.Write([]byte(env.HTML))
|
||||
} else if env.Text != "" {
|
||||
// Convert plain text to HTML for better display
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
html := fmt.Sprintf(`
|
||||
<div class="content">
|
||||
<div class="notification is-info is-light">
|
||||
<strong>Plain Text Email</strong><br>
|
||||
This email contains only plain text content.
|
||||
</div>
|
||||
<div class="box">
|
||||
<pre style="white-space: pre-wrap; font-family: inherit;">%s</pre>
|
||||
</div>
|
||||
</div>`, env.Text)
|
||||
w.Write([]byte(html))
|
||||
} else {
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
html := `
|
||||
<div class="notification is-warning">
|
||||
<strong>No Content</strong><br>
|
||||
No HTML or text content found in this email.
|
||||
</div>`
|
||||
w.Write([]byte(html))
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// No Gmail service available - show error
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
html := fmt.Sprintf(`
|
||||
<div class="content">
|
||||
<div class="notification is-warning is-light">
|
||||
<strong>Gmail Service Unavailable</strong><br>
|
||||
<small>Subject: %s</small><br>
|
||||
<small>Date: %s</small><br>
|
||||
<small>Gmail Message ID: <code>%s</code></small>
|
||||
</div>
|
||||
|
||||
<div class="box">
|
||||
<div class="content">
|
||||
<h4>Integration Status</h4>
|
||||
<p>Gmail service is not available. To enable email content display:</p>
|
||||
<ol>
|
||||
<li>Ensure <code>credentials.json</code> and <code>token.json</code> files are present</li>
|
||||
<li>Configure Gmail API OAuth2 authentication</li>
|
||||
<li>Restart the application</li>
|
||||
</ol>
|
||||
<p class="has-text-grey-light is-size-7">
|
||||
<strong>Gmail Message ID:</strong> <code>%s</code>
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>`,
|
||||
subject,
|
||||
created.Format("2006-01-02 15:04:05"),
|
||||
gmailMessageID.String,
|
||||
gmailMessageID.String)
|
||||
|
||||
w.Write([]byte(html))
|
||||
}
|
||||
|
||||
// Stream attachment from Gmail
|
||||
func (h *EmailHandler) StreamAttachment(w http.ResponseWriter, r *http.Request) {
|
||||
vars := mux.Vars(r)
|
||||
emailID, err := strconv.Atoi(vars["id"])
|
||||
if err != nil {
|
||||
http.Error(w, "Invalid email ID", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
attachmentID := vars["attachmentId"]
|
||||
|
||||
// Get email's Gmail message ID
|
||||
var gmailMessageID sql.NullString
|
||||
err = h.db.QueryRow("SELECT gmail_message_id FROM emails WHERE id = ?", emailID).Scan(&gmailMessageID)
|
||||
if err != nil || !gmailMessageID.Valid {
|
||||
http.Error(w, "Email not found or not a Gmail email", http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
|
||||
if h.gmailService == nil {
|
||||
http.Error(w, "Gmail service not available", http.StatusServiceUnavailable)
|
||||
return
|
||||
}
|
||||
|
||||
// For dynamic attachments, we need to fetch the message and find the attachment
|
||||
message, err := h.gmailService.Users.Messages.Get("me", gmailMessageID.String).
|
||||
Format("FULL").Do()
|
||||
if err != nil {
|
||||
http.Error(w, "Failed to fetch email from Gmail", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
// Find the attachment by index
|
||||
var targetAttachment *gmail.MessagePart
|
||||
attachmentIndex := 1
|
||||
findAttachment(message.Payload, attachmentID, &attachmentIndex, &targetAttachment)
|
||||
|
||||
if targetAttachment == nil || targetAttachment.Body == nil || targetAttachment.Body.AttachmentId == "" {
|
||||
http.Error(w, "Attachment not found", http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
|
||||
// Fetch attachment data from Gmail
|
||||
attachment, err := h.gmailService.Users.Messages.Attachments.
|
||||
Get("me", gmailMessageID.String, targetAttachment.Body.AttachmentId).Do()
|
||||
if err != nil {
|
||||
http.Error(w, "Failed to fetch attachment from Gmail", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
// Decode base64
|
||||
data, err := base64.URLEncoding.DecodeString(attachment.Data)
|
||||
if err != nil {
|
||||
http.Error(w, "Failed to decode attachment", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
// Set headers and stream
|
||||
filename := targetAttachment.Filename
|
||||
if filename == "" {
|
||||
// Generate filename from content type (same logic as extractGmailAttachments)
|
||||
switch targetAttachment.MimeType {
|
||||
case "application/pdf":
|
||||
filename = "attachment.pdf"
|
||||
case "image/png":
|
||||
filename = "image.png"
|
||||
case "image/jpeg":
|
||||
filename = "image.jpg"
|
||||
case "text/plain":
|
||||
filename = "text.txt"
|
||||
default:
|
||||
filename = "attachment"
|
||||
}
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", targetAttachment.MimeType)
|
||||
w.Header().Set("Content-Disposition", fmt.Sprintf("attachment; filename=\"%s\"", filename))
|
||||
w.Header().Set("Content-Length", fmt.Sprintf("%d", len(data)))
|
||||
w.Write(data)
|
||||
}
|
||||
|
||||
// Helper function to find attachment by index
|
||||
func findAttachment(part *gmail.MessagePart, targetID string, currentIndex *int, result **gmail.MessagePart) {
|
||||
// Check if this part is an attachment (same logic as extractGmailAttachments)
|
||||
if part.Body != nil && part.Body.AttachmentId != "" {
|
||||
fmt.Printf("Checking attachment %d (looking for %s): %s\n", *currentIndex, targetID, part.Filename)
|
||||
if strconv.Itoa(*currentIndex) == targetID {
|
||||
fmt.Printf("Found matching attachment!\n")
|
||||
*result = part
|
||||
return
|
||||
}
|
||||
*currentIndex++
|
||||
}
|
||||
|
||||
for _, subPart := range part.Parts {
|
||||
findAttachment(subPart, targetID, currentIndex, result)
|
||||
if *result != nil {
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Helper function to join conditions
|
||||
func joinConditions(conditions []string, separator string) string {
|
||||
if len(conditions) == 0 {
|
||||
return ""
|
||||
}
|
||||
if len(conditions) == 1 {
|
||||
return conditions[0]
|
||||
}
|
||||
|
||||
result := conditions[0]
|
||||
for i := 1; i < len(conditions); i++ {
|
||||
result += separator + conditions[i]
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
// Gmail OAuth2 functions
|
||||
func getGmailService(credentialsFile, tokenFile string) (*gmail.Service, error) {
|
||||
ctx := context.Background()
|
||||
|
||||
b, err := ioutil.ReadFile(credentialsFile)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("unable to read client secret file: %v", err)
|
||||
}
|
||||
|
||||
config, err := google.ConfigFromJSON(b, gmail.GmailReadonlyScope)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("unable to parse client secret file to config: %v", err)
|
||||
}
|
||||
|
||||
client := getClient(config, tokenFile)
|
||||
srv, err := gmail.NewService(ctx, option.WithHTTPClient(client))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("unable to retrieve Gmail client: %v", err)
|
||||
}
|
||||
|
||||
return srv, nil
|
||||
}
|
||||
|
||||
func getClient(config *oauth2.Config, tokFile string) *http.Client {
|
||||
tok, err := tokenFromFile(tokFile)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
return config.Client(context.Background(), tok)
|
||||
}
|
||||
|
||||
func tokenFromFile(file string) (*oauth2.Token, error) {
|
||||
f, err := os.Open(file)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer f.Close()
|
||||
tok := &oauth2.Token{}
|
||||
err = json.NewDecoder(f).Decode(tok)
|
||||
return tok, err
|
||||
}
|
||||
|
|
@ -1,9 +1,11 @@
|
|||
package handlers
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"log"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"code.springupsoftware.com/cmc/cmc-sales/internal/cmc/db"
|
||||
"code.springupsoftware.com/cmc/cmc-sales/internal/cmc/templates"
|
||||
|
|
@ -13,12 +15,14 @@ import (
|
|||
type PageHandler struct {
|
||||
queries *db.Queries
|
||||
tmpl *templates.TemplateManager
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func NewPageHandler(queries *db.Queries, tmpl *templates.TemplateManager) *PageHandler {
|
||||
func NewPageHandler(queries *db.Queries, tmpl *templates.TemplateManager, database *sql.DB) *PageHandler {
|
||||
return &PageHandler{
|
||||
queries: queries,
|
||||
tmpl: tmpl,
|
||||
db: database,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -776,3 +780,404 @@ func (h *PageHandler) DocumentsView(w http.ResponseWriter, r *http.Request) {
|
|||
http.Error(w, err.Error(), http.StatusInternalServerError)
|
||||
}
|
||||
}
|
||||
|
||||
// Email page handlers
|
||||
func (h *PageHandler) EmailsIndex(w http.ResponseWriter, r *http.Request) {
|
||||
page := 1
|
||||
if p := r.URL.Query().Get("page"); p != "" {
|
||||
if val, err := strconv.Atoi(p); err == nil && val > 0 {
|
||||
page = val
|
||||
}
|
||||
}
|
||||
|
||||
limit := 30
|
||||
offset := (page - 1) * limit
|
||||
search := r.URL.Query().Get("search")
|
||||
filter := r.URL.Query().Get("filter")
|
||||
|
||||
// Build SQL query based on filters
|
||||
query := `
|
||||
SELECT e.id, e.subject, e.user_id, e.created, e.gmail_message_id,
|
||||
e.email_attachment_count, e.is_downloaded,
|
||||
u.email as user_email, u.first_name, u.last_name
|
||||
FROM emails e
|
||||
LEFT JOIN users u ON e.user_id = u.id`
|
||||
|
||||
var args []interface{}
|
||||
var conditions []string
|
||||
|
||||
// Apply search filter
|
||||
if search != "" {
|
||||
conditions = append(conditions, "(e.subject LIKE ? OR e.raw_headers LIKE ?)")
|
||||
searchTerm := "%" + search + "%"
|
||||
args = append(args, searchTerm, searchTerm)
|
||||
}
|
||||
|
||||
// Apply type filter
|
||||
switch filter {
|
||||
case "downloaded":
|
||||
conditions = append(conditions, "e.is_downloaded = 1")
|
||||
case "gmail":
|
||||
conditions = append(conditions, "e.gmail_message_id IS NOT NULL")
|
||||
case "unassociated":
|
||||
conditions = append(conditions, `NOT EXISTS (
|
||||
SELECT 1 FROM emails_enquiries WHERE email_id = e.id
|
||||
UNION SELECT 1 FROM emails_invoices WHERE email_id = e.id
|
||||
UNION SELECT 1 FROM emails_purchase_orders WHERE email_id = e.id
|
||||
UNION SELECT 1 FROM emails_jobs WHERE email_id = e.id
|
||||
)`)
|
||||
}
|
||||
|
||||
if len(conditions) > 0 {
|
||||
query += " WHERE " + joinConditions(conditions, " AND ")
|
||||
}
|
||||
|
||||
query += " ORDER BY e.id DESC LIMIT ? OFFSET ?"
|
||||
args = append(args, limit+1, offset) // Get one extra to check if there are more
|
||||
|
||||
// Execute the query to get emails
|
||||
rows, err := h.db.Query(query, args...)
|
||||
if err != nil {
|
||||
log.Printf("Error querying emails: %v", err)
|
||||
http.Error(w, "Database error", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
type EmailWithUser struct {
|
||||
ID int32 `json:"id"`
|
||||
Subject string `json:"subject"`
|
||||
UserID int32 `json:"user_id"`
|
||||
Created time.Time `json:"created"`
|
||||
GmailMessageID *string `json:"gmail_message_id"`
|
||||
AttachmentCount int32 `json:"attachment_count"`
|
||||
IsDownloaded *bool `json:"is_downloaded"`
|
||||
UserEmail *string `json:"user_email"`
|
||||
FirstName *string `json:"first_name"`
|
||||
LastName *string `json:"last_name"`
|
||||
}
|
||||
|
||||
var emails []EmailWithUser
|
||||
for rows.Next() {
|
||||
var email EmailWithUser
|
||||
var gmailMessageID, userEmail, firstName, lastName sql.NullString
|
||||
var isDownloaded sql.NullBool
|
||||
|
||||
err := rows.Scan(
|
||||
&email.ID,
|
||||
&email.Subject,
|
||||
&email.UserID,
|
||||
&email.Created,
|
||||
&gmailMessageID,
|
||||
&email.AttachmentCount,
|
||||
&isDownloaded,
|
||||
&userEmail,
|
||||
&firstName,
|
||||
&lastName,
|
||||
)
|
||||
if err != nil {
|
||||
log.Printf("Error scanning email row: %v", err)
|
||||
continue
|
||||
}
|
||||
|
||||
if gmailMessageID.Valid {
|
||||
email.GmailMessageID = &gmailMessageID.String
|
||||
}
|
||||
if isDownloaded.Valid {
|
||||
email.IsDownloaded = &isDownloaded.Bool
|
||||
}
|
||||
if userEmail.Valid {
|
||||
email.UserEmail = &userEmail.String
|
||||
}
|
||||
if firstName.Valid {
|
||||
email.FirstName = &firstName.String
|
||||
}
|
||||
if lastName.Valid {
|
||||
email.LastName = &lastName.String
|
||||
}
|
||||
|
||||
emails = append(emails, email)
|
||||
}
|
||||
|
||||
hasMore := len(emails) > limit
|
||||
if hasMore {
|
||||
emails = emails[:limit]
|
||||
}
|
||||
|
||||
data := map[string]interface{}{
|
||||
"Emails": emails,
|
||||
"Page": page,
|
||||
"PrevPage": page - 1,
|
||||
"NextPage": page + 1,
|
||||
"HasMore": hasMore,
|
||||
"TotalPages": ((len(emails) + limit - 1) / limit),
|
||||
}
|
||||
|
||||
// Check if this is an HTMX request
|
||||
if r.Header.Get("HX-Request") == "true" {
|
||||
if err := h.tmpl.RenderPartial(w, "emails/table.html", "email-table", data); err != nil {
|
||||
http.Error(w, err.Error(), http.StatusInternalServerError)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
if err := h.tmpl.Render(w, "emails/index.html", data); err != nil {
|
||||
http.Error(w, err.Error(), http.StatusInternalServerError)
|
||||
}
|
||||
}
|
||||
|
||||
func (h *PageHandler) EmailsShow(w http.ResponseWriter, r *http.Request) {
|
||||
vars := mux.Vars(r)
|
||||
id, err := strconv.Atoi(vars["id"])
|
||||
if err != nil {
|
||||
http.Error(w, "Invalid email ID", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
// Get email details from database
|
||||
emailQuery := `
|
||||
SELECT e.id, e.subject, e.user_id, e.created, e.gmail_message_id,
|
||||
e.gmail_thread_id, e.raw_headers, e.is_downloaded,
|
||||
u.email as user_email, u.first_name, u.last_name
|
||||
FROM emails e
|
||||
LEFT JOIN users u ON e.user_id = u.id
|
||||
WHERE e.id = ?`
|
||||
|
||||
var email struct {
|
||||
ID int32 `json:"id"`
|
||||
Subject string `json:"subject"`
|
||||
UserID int32 `json:"user_id"`
|
||||
Created time.Time `json:"created"`
|
||||
GmailMessageID *string `json:"gmail_message_id"`
|
||||
GmailThreadID *string `json:"gmail_thread_id"`
|
||||
RawHeaders *string `json:"raw_headers"`
|
||||
IsDownloaded *bool `json:"is_downloaded"`
|
||||
User *struct {
|
||||
Email string `json:"email"`
|
||||
FirstName string `json:"first_name"`
|
||||
LastName string `json:"last_name"`
|
||||
} `json:"user"`
|
||||
Enquiries []int32 `json:"enquiries"`
|
||||
Invoices []int32 `json:"invoices"`
|
||||
PurchaseOrders []int32 `json:"purchase_orders"`
|
||||
Jobs []int32 `json:"jobs"`
|
||||
}
|
||||
|
||||
var gmailMessageID, gmailThreadID, rawHeaders sql.NullString
|
||||
var isDownloaded sql.NullBool
|
||||
var userEmail, firstName, lastName sql.NullString
|
||||
|
||||
err = h.db.QueryRow(emailQuery, id).Scan(
|
||||
&email.ID,
|
||||
&email.Subject,
|
||||
&email.UserID,
|
||||
&email.Created,
|
||||
&gmailMessageID,
|
||||
&gmailThreadID,
|
||||
&rawHeaders,
|
||||
&isDownloaded,
|
||||
&userEmail,
|
||||
&firstName,
|
||||
&lastName,
|
||||
)
|
||||
|
||||
if err != nil {
|
||||
if err == sql.ErrNoRows {
|
||||
http.Error(w, "Email not found", http.StatusNotFound)
|
||||
} else {
|
||||
log.Printf("Error fetching email %d: %v", id, err)
|
||||
http.Error(w, "Database error", http.StatusInternalServerError)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// Set nullable fields
|
||||
if gmailMessageID.Valid {
|
||||
email.GmailMessageID = &gmailMessageID.String
|
||||
}
|
||||
if gmailThreadID.Valid {
|
||||
email.GmailThreadID = &gmailThreadID.String
|
||||
}
|
||||
if rawHeaders.Valid {
|
||||
email.RawHeaders = &rawHeaders.String
|
||||
}
|
||||
if isDownloaded.Valid {
|
||||
email.IsDownloaded = &isDownloaded.Bool
|
||||
}
|
||||
|
||||
// Set user info if available
|
||||
if userEmail.Valid {
|
||||
email.User = &struct {
|
||||
Email string `json:"email"`
|
||||
FirstName string `json:"first_name"`
|
||||
LastName string `json:"last_name"`
|
||||
}{
|
||||
Email: userEmail.String,
|
||||
FirstName: firstName.String,
|
||||
LastName: lastName.String,
|
||||
}
|
||||
}
|
||||
|
||||
// Get email attachments
|
||||
attachmentQuery := `
|
||||
SELECT id, name, type, size, filename, is_message_body, gmail_attachment_id, created
|
||||
FROM email_attachments
|
||||
WHERE email_id = ?
|
||||
ORDER BY is_message_body DESC, created ASC`
|
||||
|
||||
attachmentRows, err := h.db.Query(attachmentQuery, id)
|
||||
if err != nil {
|
||||
log.Printf("Error fetching attachments for email %d: %v", id, err)
|
||||
}
|
||||
|
||||
type EmailAttachment struct {
|
||||
ID int32 `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Type string `json:"type"`
|
||||
Size int32 `json:"size"`
|
||||
Filename string `json:"filename"`
|
||||
IsMessageBody bool `json:"is_message_body"`
|
||||
GmailAttachmentID *string `json:"gmail_attachment_id"`
|
||||
Created time.Time `json:"created"`
|
||||
}
|
||||
|
||||
var attachments []EmailAttachment
|
||||
hasStoredAttachments := false
|
||||
|
||||
if attachmentRows != nil {
|
||||
defer attachmentRows.Close()
|
||||
for attachmentRows.Next() {
|
||||
hasStoredAttachments = true
|
||||
var attachment EmailAttachment
|
||||
var gmailAttachmentID sql.NullString
|
||||
|
||||
err := attachmentRows.Scan(
|
||||
&attachment.ID,
|
||||
&attachment.Name,
|
||||
&attachment.Type,
|
||||
&attachment.Size,
|
||||
&attachment.Filename,
|
||||
&attachment.IsMessageBody,
|
||||
&gmailAttachmentID,
|
||||
&attachment.Created,
|
||||
)
|
||||
if err != nil {
|
||||
log.Printf("Error scanning attachment: %v", err)
|
||||
continue
|
||||
}
|
||||
|
||||
if gmailAttachmentID.Valid {
|
||||
attachment.GmailAttachmentID = &gmailAttachmentID.String
|
||||
}
|
||||
|
||||
attachments = append(attachments, attachment)
|
||||
}
|
||||
}
|
||||
|
||||
// If no stored attachments and this is a Gmail email, show a notice
|
||||
if !hasStoredAttachments && email.GmailMessageID != nil {
|
||||
// For the page view, we'll just show a notice that attachments can be fetched
|
||||
// The actual fetching will happen via the API endpoint when needed
|
||||
log.Printf("Email %d is a Gmail email without indexed attachments", id)
|
||||
}
|
||||
|
||||
// Get associated records (simplified queries for now)
|
||||
// Enquiries
|
||||
enquiryRows, err := h.db.Query("SELECT enquiry_id FROM emails_enquiries WHERE email_id = ?", id)
|
||||
if err == nil {
|
||||
defer enquiryRows.Close()
|
||||
for enquiryRows.Next() {
|
||||
var enquiryID int32
|
||||
if enquiryRows.Scan(&enquiryID) == nil {
|
||||
email.Enquiries = append(email.Enquiries, enquiryID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Invoices
|
||||
invoiceRows, err := h.db.Query("SELECT invoice_id FROM emails_invoices WHERE email_id = ?", id)
|
||||
if err == nil {
|
||||
defer invoiceRows.Close()
|
||||
for invoiceRows.Next() {
|
||||
var invoiceID int32
|
||||
if invoiceRows.Scan(&invoiceID) == nil {
|
||||
email.Invoices = append(email.Invoices, invoiceID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Purchase Orders
|
||||
poRows, err := h.db.Query("SELECT purchase_order_id FROM emails_purchase_orders WHERE email_id = ?", id)
|
||||
if err == nil {
|
||||
defer poRows.Close()
|
||||
for poRows.Next() {
|
||||
var poID int32
|
||||
if poRows.Scan(&poID) == nil {
|
||||
email.PurchaseOrders = append(email.PurchaseOrders, poID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Jobs
|
||||
jobRows, err := h.db.Query("SELECT job_id FROM emails_jobs WHERE email_id = ?", id)
|
||||
if err == nil {
|
||||
defer jobRows.Close()
|
||||
for jobRows.Next() {
|
||||
var jobID int32
|
||||
if jobRows.Scan(&jobID) == nil {
|
||||
email.Jobs = append(email.Jobs, jobID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
data := map[string]interface{}{
|
||||
"Email": email,
|
||||
"Attachments": attachments,
|
||||
}
|
||||
|
||||
if err := h.tmpl.Render(w, "emails/show.html", data); err != nil {
|
||||
http.Error(w, err.Error(), http.StatusInternalServerError)
|
||||
}
|
||||
}
|
||||
|
||||
func (h *PageHandler) EmailsSearch(w http.ResponseWriter, r *http.Request) {
|
||||
_ = r.URL.Query().Get("search") // TODO: Implement search functionality
|
||||
|
||||
// Empty result for now - would need proper implementation
|
||||
emails := []interface{}{}
|
||||
|
||||
data := map[string]interface{}{
|
||||
"Emails": emails,
|
||||
"Page": 1,
|
||||
"PrevPage": 0,
|
||||
"NextPage": 2,
|
||||
"HasMore": false,
|
||||
"TotalPages": 1,
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "text/html")
|
||||
if err := h.tmpl.RenderPartial(w, "emails/table.html", "email-table", data); err != nil {
|
||||
http.Error(w, err.Error(), http.StatusInternalServerError)
|
||||
}
|
||||
}
|
||||
|
||||
func (h *PageHandler) EmailsAttachments(w http.ResponseWriter, r *http.Request) {
|
||||
vars := mux.Vars(r)
|
||||
_, err := strconv.Atoi(vars["id"])
|
||||
if err != nil {
|
||||
http.Error(w, "Invalid email ID", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
// Empty attachments for now - would need proper implementation
|
||||
attachments := []interface{}{}
|
||||
|
||||
data := map[string]interface{}{
|
||||
"Attachments": attachments,
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "text/html")
|
||||
if err := h.tmpl.RenderPartial(w, "emails/attachments.html", "email-attachments", data); err != nil {
|
||||
http.Error(w, err.Error(), http.StatusInternalServerError)
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -54,6 +54,10 @@ func NewTemplateManager(templatesDir string) (*TemplateManager, error) {
|
|||
"enquiries/show.html",
|
||||
"enquiries/form.html",
|
||||
"enquiries/table.html",
|
||||
"emails/index.html",
|
||||
"emails/show.html",
|
||||
"emails/table.html",
|
||||
"emails/attachments.html",
|
||||
"documents/index.html",
|
||||
"documents/show.html",
|
||||
"documents/table.html",
|
||||
|
|
|
|||
52
go-app/sql/migrations/001_add_gmail_fields.sql
Normal file
52
go-app/sql/migrations/001_add_gmail_fields.sql
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
-- +goose Up
|
||||
-- Add Gmail-specific fields to emails table
|
||||
ALTER TABLE emails
|
||||
ADD COLUMN gmail_message_id VARCHAR(255) UNIQUE AFTER id,
|
||||
ADD COLUMN gmail_thread_id VARCHAR(255) AFTER gmail_message_id,
|
||||
ADD COLUMN is_downloaded BOOLEAN DEFAULT FALSE AFTER email_attachment_count,
|
||||
ADD COLUMN raw_headers TEXT AFTER subject;
|
||||
|
||||
-- +goose StatementBegin
|
||||
CREATE INDEX idx_gmail_message_id ON emails(gmail_message_id);
|
||||
-- +goose StatementEnd
|
||||
|
||||
-- +goose StatementBegin
|
||||
CREATE INDEX idx_gmail_thread_id ON emails(gmail_thread_id);
|
||||
-- +goose StatementEnd
|
||||
|
||||
-- Add Gmail-specific fields to email_attachments
|
||||
ALTER TABLE email_attachments
|
||||
ADD COLUMN gmail_attachment_id VARCHAR(255) AFTER email_id,
|
||||
ADD COLUMN gmail_message_id VARCHAR(255) AFTER gmail_attachment_id,
|
||||
ADD COLUMN content_id VARCHAR(255) AFTER filename;
|
||||
|
||||
-- +goose StatementBegin
|
||||
CREATE INDEX idx_gmail_attachment_id ON email_attachments(gmail_attachment_id);
|
||||
-- +goose StatementEnd
|
||||
|
||||
-- +goose Down
|
||||
-- Remove indexes
|
||||
-- +goose StatementBegin
|
||||
DROP INDEX idx_gmail_attachment_id ON email_attachments;
|
||||
-- +goose StatementEnd
|
||||
|
||||
-- +goose StatementBegin
|
||||
DROP INDEX idx_gmail_thread_id ON emails;
|
||||
-- +goose StatementEnd
|
||||
|
||||
-- +goose StatementBegin
|
||||
DROP INDEX idx_gmail_message_id ON emails;
|
||||
-- +goose StatementEnd
|
||||
|
||||
-- Remove columns from email_attachments
|
||||
ALTER TABLE email_attachments
|
||||
DROP COLUMN content_id,
|
||||
DROP COLUMN gmail_message_id,
|
||||
DROP COLUMN gmail_attachment_id;
|
||||
|
||||
-- Remove columns from emails
|
||||
ALTER TABLE emails
|
||||
DROP COLUMN raw_headers,
|
||||
DROP COLUMN is_downloaded,
|
||||
DROP COLUMN gmail_thread_id,
|
||||
DROP COLUMN gmail_message_id;
|
||||
|
|
@ -106,4 +106,20 @@ body {
|
|||
.input.is-danger:focus {
|
||||
border-color: #ff3860;
|
||||
box-shadow: 0 0 0 0.125em rgba(255,56,96,.25);
|
||||
}
|
||||
|
||||
/* Simple CSS loader */
|
||||
.loader {
|
||||
border: 2px solid #f3f3f3;
|
||||
border-top: 2px solid #3273dc;
|
||||
border-radius: 50%;
|
||||
width: 16px;
|
||||
height: 16px;
|
||||
animation: spin 1s linear infinite;
|
||||
display: inline-block;
|
||||
}
|
||||
|
||||
@keyframes spin {
|
||||
0% { transform: rotate(0deg); }
|
||||
100% { transform: rotate(360deg); }
|
||||
}
|
||||
67
go-app/templates/emails/attachments.html
Normal file
67
go-app/templates/emails/attachments.html
Normal file
|
|
@ -0,0 +1,67 @@
|
|||
{{define "email-attachments"}}
|
||||
<div class="table-container">
|
||||
<table class="table is-fullwidth">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
<th>Type</th>
|
||||
<th>Size</th>
|
||||
<th>Date</th>
|
||||
<th>Actions</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{{range .}}
|
||||
<tr>
|
||||
<td>
|
||||
{{if .IsMessageBody}}
|
||||
<span class="icon has-text-info">
|
||||
<i class="fas fa-envelope"></i>
|
||||
</span>
|
||||
<span class="has-text-weight-semibold">{{.Name}}</span>
|
||||
<span class="tag is-small is-info ml-2">Body</span>
|
||||
{{else}}
|
||||
<span class="icon has-text-grey">
|
||||
<i class="fas fa-paperclip"></i>
|
||||
</span>
|
||||
{{.Name}}
|
||||
{{end}}
|
||||
</td>
|
||||
<td>
|
||||
<span class="tag is-light">{{.Type}}</span>
|
||||
</td>
|
||||
<td>{{.Size}} bytes</td>
|
||||
<td class="is-size-7">
|
||||
{{.Created}}
|
||||
</td>
|
||||
<td>
|
||||
{{if .GmailAttachmentID}}
|
||||
<a href="/api/v1/emails/attachments/{{.ID}}/stream"
|
||||
target="_blank" class="button is-small is-info">
|
||||
<span class="icon">
|
||||
<i class="fas fa-cloud-download-alt"></i>
|
||||
</span>
|
||||
<span>Stream</span>
|
||||
</a>
|
||||
{{else}}
|
||||
<a href="/api/v1/emails/attachments/{{.ID}}"
|
||||
target="_blank" class="button is-small is-success">
|
||||
<span class="icon">
|
||||
<i class="fas fa-download"></i>
|
||||
</span>
|
||||
<span>Download</span>
|
||||
</a>
|
||||
{{end}}
|
||||
</td>
|
||||
</tr>
|
||||
{{else}}
|
||||
<tr>
|
||||
<td colspan="5" class="has-text-centered">
|
||||
<p class="has-text-grey">No attachments found</p>
|
||||
</td>
|
||||
</tr>
|
||||
{{end}}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
{{end}}
|
||||
82
go-app/templates/emails/index.html
Normal file
82
go-app/templates/emails/index.html
Normal file
|
|
@ -0,0 +1,82 @@
|
|||
{{define "title"}}Emails - CMC Sales{{end}}
|
||||
|
||||
{{define "content"}}
|
||||
<div class="level">
|
||||
<div class="level-left">
|
||||
<div class="level-item">
|
||||
<h1 class="title">Emails</h1>
|
||||
</div>
|
||||
</div>
|
||||
<div class="level-right">
|
||||
<div class="level-item">
|
||||
<div class="field has-addons">
|
||||
<div class="control">
|
||||
<div class="select">
|
||||
<select id="email-filter">
|
||||
<option value="">All Emails</option>
|
||||
<option value="downloaded">Downloaded</option>
|
||||
<option value="gmail">Gmail Only</option>
|
||||
<option value="unassociated">Unassociated</option>
|
||||
</select>
|
||||
</div>
|
||||
</div>
|
||||
<div class="control">
|
||||
<button class="button is-info" id="apply-filter">
|
||||
<span class="icon">
|
||||
<i class="fas fa-filter"></i>
|
||||
</span>
|
||||
<span>Filter</span>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Search Box -->
|
||||
<div class="box">
|
||||
<div class="field has-addons">
|
||||
<div class="control has-icons-left is-expanded">
|
||||
<input class="input" type="text" placeholder="Search emails by subject or content..."
|
||||
name="search" id="email-search"
|
||||
hx-get="/emails/search"
|
||||
hx-trigger="keyup changed delay:500ms"
|
||||
hx-target="#email-table-container">
|
||||
<span class="icon is-left">
|
||||
<i class="fas fa-search"></i>
|
||||
</span>
|
||||
</div>
|
||||
<div class="control">
|
||||
<button class="button is-info" onclick="document.getElementById('email-search').value=''; document.getElementById('email-search').dispatchEvent(new Event('keyup'));">
|
||||
Clear
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Email Table Container -->
|
||||
<div id="email-table-container">
|
||||
{{template "email-table" .}}
|
||||
</div>
|
||||
{{end}}
|
||||
|
||||
{{define "scripts"}}
|
||||
<script>
|
||||
document.getElementById('apply-filter').addEventListener('click', function() {
|
||||
const filter = document.getElementById('email-filter').value;
|
||||
const search = document.getElementById('email-search').value;
|
||||
|
||||
let url = '/emails';
|
||||
const params = new URLSearchParams();
|
||||
|
||||
if (filter) params.append('filter', filter);
|
||||
if (search) params.append('search', search);
|
||||
|
||||
if (params.toString()) {
|
||||
url += '?' + params.toString();
|
||||
}
|
||||
|
||||
htmx.ajax('GET', url, {target: '#email-table-container'});
|
||||
});
|
||||
</script>
|
||||
{{end}}
|
||||
293
go-app/templates/emails/show.html
Normal file
293
go-app/templates/emails/show.html
Normal file
|
|
@ -0,0 +1,293 @@
|
|||
{{define "title"}}Email {{.Email.ID}} - CMC Sales{{end}}
|
||||
|
||||
{{define "content"}}
|
||||
<div class="level">
|
||||
<div class="level-left">
|
||||
<div class="level-item">
|
||||
<nav class="breadcrumb" aria-label="breadcrumbs">
|
||||
<ul>
|
||||
<li><a href="/emails">Emails</a></li>
|
||||
<li class="is-active"><a href="#" aria-current="page">Email {{.Email.ID}}</a></li>
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
<div class="level-right">
|
||||
<div class="level-item">
|
||||
<div class="buttons">
|
||||
{{if .Email.GmailMessageID}}
|
||||
<a href="/api/v1/emails/{{.Email.ID}}/stream" target="_blank" class="button is-success">
|
||||
<span class="icon">
|
||||
<i class="fas fa-external-link-alt"></i>
|
||||
</span>
|
||||
<span>View in Gmail</span>
|
||||
</a>
|
||||
{{end}}
|
||||
<a href="/emails" class="button">
|
||||
<span class="icon">
|
||||
<i class="fas fa-arrow-left"></i>
|
||||
</span>
|
||||
<span>Back to Emails</span>
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="columns">
|
||||
<div class="column is-8">
|
||||
<!-- Email Details -->
|
||||
<div class="box">
|
||||
<h2 class="title is-4">
|
||||
{{if .Email.Subject}}{{.Email.Subject}}{{else}}<em>(No Subject)</em>{{end}}
|
||||
</h2>
|
||||
|
||||
<div class="content">
|
||||
<div class="columns is-multiline">
|
||||
<div class="column is-6">
|
||||
<strong>From:</strong>
|
||||
{{if .Email.User}}
|
||||
{{.Email.User.Email}} ({{.Email.User.FirstName}} {{.Email.User.LastName}})
|
||||
{{else}}
|
||||
<span class="has-text-grey">Unknown sender</span>
|
||||
{{end}}
|
||||
</div>
|
||||
<div class="column is-6">
|
||||
<strong>Date:</strong> {{.Email.Created.Format "2006-01-02 15:04:05"}}
|
||||
</div>
|
||||
<div class="column is-6">
|
||||
<strong>Type:</strong>
|
||||
{{if .Email.GmailMessageID}}
|
||||
<span class="tag is-link">
|
||||
<span class="icon is-small">
|
||||
<i class="fab fa-google"></i>
|
||||
</span>
|
||||
<span>Gmail</span>
|
||||
</span>
|
||||
{{if and .Email.IsDownloaded (not .Email.IsDownloaded)}}
|
||||
<span class="tag is-warning">Remote</span>
|
||||
{{else}}
|
||||
<span class="tag is-success">Downloaded</span>
|
||||
{{end}}
|
||||
{{else}}
|
||||
<span class="tag is-success">Local Email</span>
|
||||
{{end}}
|
||||
</div>
|
||||
<div class="column is-6">
|
||||
<strong>Attachments:</strong>
|
||||
{{if gt (len .Attachments) 0}}
|
||||
<span class="tag is-info">{{len .Attachments}} files</span>
|
||||
{{else}}
|
||||
<span class="has-text-grey">None</span>
|
||||
{{end}}
|
||||
</div>
|
||||
{{if .Email.GmailMessageID}}
|
||||
<div class="column is-12">
|
||||
<strong>Gmail Message ID:</strong>
|
||||
<code class="is-size-7">{{.Email.GmailMessageID}}</code>
|
||||
</div>
|
||||
{{end}}
|
||||
{{if .Email.GmailThreadID}}
|
||||
<div class="column is-12">
|
||||
<strong>Gmail Thread ID:</strong>
|
||||
<code class="is-size-7">{{.Email.GmailThreadID}}</code>
|
||||
</div>
|
||||
{{end}}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Email Content -->
|
||||
{{if .Email.GmailMessageID}}
|
||||
<div class="box">
|
||||
<h3 class="title is-5">Email Content</h3>
|
||||
<div id="email-content-container"
|
||||
hx-get="/api/v1/emails/{{.Email.ID}}/content"
|
||||
hx-trigger="load">
|
||||
<div class="notification is-info">
|
||||
<div class="is-flex is-align-items-center">
|
||||
<span class="loader mr-3"></span>
|
||||
<span>Loading email content...</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{{end}}
|
||||
|
||||
<!-- Attachments -->
|
||||
{{if gt (len .Attachments) 0}}
|
||||
<div class="box">
|
||||
<h3 class="title is-5">Attachments</h3>
|
||||
<div class="table-container">
|
||||
<table class="table is-fullwidth">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
<th>Type</th>
|
||||
<th>Size</th>
|
||||
<th>Date</th>
|
||||
<th>Actions</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{{range .Attachments}}
|
||||
<tr>
|
||||
<td>
|
||||
{{if .IsMessageBody}}
|
||||
<span class="icon has-text-info">
|
||||
<i class="fas fa-envelope"></i>
|
||||
</span>
|
||||
{{else}}
|
||||
<span class="icon has-text-grey">
|
||||
<i class="fas fa-paperclip"></i>
|
||||
</span>
|
||||
{{end}}
|
||||
{{.Name}}
|
||||
</td>
|
||||
<td>
|
||||
<span class="tag is-light">{{.Type}}</span>
|
||||
</td>
|
||||
<td>{{.Size}} bytes</td>
|
||||
<td class="is-size-7">
|
||||
{{.Created.Format "2006-01-02 15:04"}}
|
||||
</td>
|
||||
<td>
|
||||
{{if .GmailAttachmentID}}
|
||||
<a href="/api/v1/emails/{{$.Email.ID}}/attachments/{{.ID}}/stream"
|
||||
target="_blank" class="button is-small is-info">
|
||||
<span class="icon">
|
||||
<i class="fas fa-download"></i>
|
||||
</span>
|
||||
<span>Download</span>
|
||||
</a>
|
||||
{{else}}
|
||||
<a href="/api/v1/emails/{{$.Email.ID}}/attachments/{{.ID}}"
|
||||
target="_blank" class="button is-small is-success">
|
||||
<span class="icon">
|
||||
<i class="fas fa-download"></i>
|
||||
</span>
|
||||
<span>Download</span>
|
||||
</a>
|
||||
{{end}}
|
||||
</td>
|
||||
</tr>
|
||||
{{end}}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
{{else if .Email.GmailMessageID}}
|
||||
<!-- Gmail Attachments Check -->
|
||||
<div class="box">
|
||||
<h3 class="title is-5">Attachments</h3>
|
||||
<div id="gmail-attachments-container"
|
||||
hx-get="/api/v1/emails/{{.Email.ID}}/attachments"
|
||||
hx-trigger="load"
|
||||
hx-swap="outerHTML">
|
||||
<div class="notification is-info is-light">
|
||||
<div class="is-flex is-align-items-center">
|
||||
<span class="loader mr-3"></span>
|
||||
<span>Checking for Gmail attachments...</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{{end}}
|
||||
</div>
|
||||
|
||||
<div class="column is-4">
|
||||
<!-- Associated Records -->
|
||||
<div class="box">
|
||||
<h3 class="title is-5">Associated Records</h3>
|
||||
|
||||
{{if .Email.Enquiries}}
|
||||
<div class="field">
|
||||
<label class="label">Enquiries</label>
|
||||
<div class="tags">
|
||||
{{range .Email.Enquiries}}
|
||||
<a href="/enquiries/{{.}}" class="tag is-primary">
|
||||
ENQ-{{.}}
|
||||
</a>
|
||||
{{end}}
|
||||
</div>
|
||||
</div>
|
||||
{{end}}
|
||||
|
||||
{{if .Email.Invoices}}
|
||||
<div class="field">
|
||||
<label class="label">Invoices</label>
|
||||
<div class="tags">
|
||||
{{range .Email.Invoices}}
|
||||
<a href="/invoices/{{.}}" class="tag is-warning">
|
||||
INV-{{.}}
|
||||
</a>
|
||||
{{end}}
|
||||
</div>
|
||||
</div>
|
||||
{{end}}
|
||||
|
||||
{{if .Email.PurchaseOrders}}
|
||||
<div class="field">
|
||||
<label class="label">Purchase Orders</label>
|
||||
<div class="tags">
|
||||
{{range .Email.PurchaseOrders}}
|
||||
<a href="/purchase-orders/{{.}}" class="tag is-info">
|
||||
PO-{{.}}
|
||||
</a>
|
||||
{{end}}
|
||||
</div>
|
||||
</div>
|
||||
{{end}}
|
||||
|
||||
{{if .Email.Jobs}}
|
||||
<div class="field">
|
||||
<label class="label">Jobs</label>
|
||||
<div class="tags">
|
||||
{{range .Email.Jobs}}
|
||||
<a href="/jobs/{{.}}" class="tag is-success">
|
||||
JOB-{{.}}
|
||||
</a>
|
||||
{{end}}
|
||||
</div>
|
||||
</div>
|
||||
{{end}}
|
||||
|
||||
{{if and (not .Email.Enquiries) (not .Email.Invoices) (not .Email.PurchaseOrders) (not .Email.Jobs)}}
|
||||
<div class="notification is-warning is-light">
|
||||
<p><strong>No associations found</strong></p>
|
||||
<p class="is-size-7">This email is not associated with any enquiries, invoices, purchase orders, or jobs.</p>
|
||||
</div>
|
||||
{{end}}
|
||||
</div>
|
||||
|
||||
<!-- Quick Actions -->
|
||||
<div class="box">
|
||||
<h3 class="title is-5">Quick Actions</h3>
|
||||
<div class="buttons is-vertical is-fullwidth">
|
||||
<button class="button is-info is-outlined">
|
||||
<span class="icon">
|
||||
<i class="fas fa-link"></i>
|
||||
</span>
|
||||
<span>Associate with Record</span>
|
||||
</button>
|
||||
<button class="button is-warning is-outlined">
|
||||
<span class="icon">
|
||||
<i class="fas fa-flag"></i>
|
||||
</span>
|
||||
<span>Mark for Review</span>
|
||||
</button>
|
||||
{{if .Email.GmailMessageID}}
|
||||
<button class="button is-success is-outlined"
|
||||
hx-post="/api/v1/emails/{{.Email.ID}}/download"
|
||||
hx-confirm="Download this email and attachments locally?">
|
||||
<span class="icon">
|
||||
<i class="fas fa-download"></i>
|
||||
</span>
|
||||
<span>Download Locally</span>
|
||||
</button>
|
||||
{{end}}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{{end}}
|
||||
143
go-app/templates/emails/table.html
Normal file
143
go-app/templates/emails/table.html
Normal file
|
|
@ -0,0 +1,143 @@
|
|||
{{define "email-table"}}
|
||||
<div class="table-container">
|
||||
<table class="table is-fullwidth is-striped is-hoverable">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>ID</th>
|
||||
<th>Subject</th>
|
||||
<th>From</th>
|
||||
<th>Date</th>
|
||||
<th>Attachments</th>
|
||||
<th>Type</th>
|
||||
<th>Associated</th>
|
||||
<th>Actions</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{{range .Emails}}
|
||||
<tr>
|
||||
<td>{{.ID}}</td>
|
||||
<td>
|
||||
<a href="/emails/{{.ID}}" class="has-text-weight-semibold">
|
||||
{{if .Subject}}{{.Subject}}{{else}}<em>(No Subject)</em>{{end}}
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
{{if .UserEmail}}
|
||||
{{.UserEmail}}
|
||||
{{else}}
|
||||
<span class="has-text-grey">Unknown</span>
|
||||
{{end}}
|
||||
</td>
|
||||
<td>
|
||||
<span class="is-size-7">
|
||||
{{.Created.Format "2006-01-02 15:04"}}
|
||||
</span>
|
||||
</td>
|
||||
<td>
|
||||
{{if gt .AttachmentCount 0}}
|
||||
<span class="tag is-info is-light">
|
||||
<span class="icon is-small">
|
||||
<i class="fas fa-paperclip"></i>
|
||||
</span>
|
||||
<span>{{.AttachmentCount}}</span>
|
||||
</span>
|
||||
{{else}}
|
||||
<span class="has-text-grey">None</span>
|
||||
{{end}}
|
||||
</td>
|
||||
<td>
|
||||
{{if .GmailMessageID}}
|
||||
<span class="tag is-link is-light">
|
||||
<span class="icon is-small">
|
||||
<i class="fab fa-google"></i>
|
||||
</span>
|
||||
<span>Gmail</span>
|
||||
</span>
|
||||
{{if and .IsDownloaded (not .IsDownloaded)}}
|
||||
<span class="tag is-warning is-light">
|
||||
<span class="icon is-small">
|
||||
<i class="fas fa-cloud"></i>
|
||||
</span>
|
||||
<span>Remote</span>
|
||||
</span>
|
||||
{{end}}
|
||||
{{else}}
|
||||
<span class="tag is-success is-light">
|
||||
<span class="icon is-small">
|
||||
<i class="fas fa-envelope"></i>
|
||||
</span>
|
||||
<span>Local</span>
|
||||
</span>
|
||||
{{end}}
|
||||
</td>
|
||||
<td>
|
||||
<span class="tag is-light">Associations TBD</span>
|
||||
</td>
|
||||
<td>
|
||||
<div class="buttons are-small">
|
||||
<a href="/emails/{{.ID}}" class="button is-info is-outlined">
|
||||
<span class="icon">
|
||||
<i class="fas fa-eye"></i>
|
||||
</span>
|
||||
</a>
|
||||
{{if gt .AttachmentCount 0}}
|
||||
<button class="button is-link is-outlined"
|
||||
hx-get="/emails/{{.ID}}/attachments"
|
||||
hx-target="#attachment-modal-content"
|
||||
onclick="document.getElementById('attachment-modal').classList.add('is-active')">
|
||||
<span class="icon">
|
||||
<i class="fas fa-paperclip"></i>
|
||||
</span>
|
||||
</button>
|
||||
{{end}}
|
||||
{{if .GmailMessageID}}
|
||||
<a href="/api/v1/emails/{{.ID}}/stream" target="_blank" class="button is-success is-outlined">
|
||||
<span class="icon">
|
||||
<i class="fas fa-external-link-alt"></i>
|
||||
</span>
|
||||
</a>
|
||||
{{end}}
|
||||
</div>
|
||||
</td>
|
||||
</tr>
|
||||
{{else}}
|
||||
<tr>
|
||||
<td colspan="8" class="has-text-centered">
|
||||
<p class="has-text-grey">No emails found</p>
|
||||
</td>
|
||||
</tr>
|
||||
{{end}}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<!-- Pagination -->
|
||||
{{if .Emails}}
|
||||
<nav class="pagination is-centered" role="navigation" aria-label="pagination">
|
||||
<a class="pagination-previous" {{if eq .Page 1}}disabled{{end}}
|
||||
hx-get="/emails?page={{.PrevPage}}"
|
||||
hx-target="#email-table-container">Previous</a>
|
||||
<a class="pagination-next" {{if not .HasMore}}disabled{{end}}
|
||||
hx-get="/emails?page={{.NextPage}}"
|
||||
hx-target="#email-table-container">Next</a>
|
||||
<ul class="pagination-list">
|
||||
<li><span class="pagination-ellipsis">Page {{.Page}} of {{.TotalPages}}</span></li>
|
||||
</ul>
|
||||
</nav>
|
||||
{{end}}
|
||||
|
||||
<!-- Attachment Modal -->
|
||||
<div class="modal" id="attachment-modal">
|
||||
<div class="modal-background" onclick="document.getElementById('attachment-modal').classList.remove('is-active')"></div>
|
||||
<div class="modal-card">
|
||||
<header class="modal-card-head">
|
||||
<p class="modal-card-title">Email Attachments</p>
|
||||
<button class="delete" aria-label="close" onclick="document.getElementById('attachment-modal').classList.remove('is-active')"></button>
|
||||
</header>
|
||||
<section class="modal-card-body" id="attachment-modal-content">
|
||||
<!-- Attachment list will be loaded here -->
|
||||
</section>
|
||||
</div>
|
||||
</div>
|
||||
{{end}}
|
||||
|
|
@ -61,6 +61,11 @@
|
|||
<span class="icon"><i class="fas fa-envelope"></i></span>
|
||||
<span>Enquiries</span>
|
||||
</a>
|
||||
|
||||
<a class="navbar-item" href="/emails">
|
||||
<span class="icon"><i class="fas fa-mail-bulk"></i></span>
|
||||
<span>Emails</span>
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
</nav>
|
||||
|
|
|
|||
Loading…
Reference in a new issue