cmc-sales/TESTING_DOCKER.md

# Running CMC Django Tests in Docker

This guide explains how to run the comprehensive CMC Django test suite using Docker for consistent, isolated testing.

## Quick Start

```bash
# 1. Setup test environment (one-time)
./run-tests-docker.sh setup

# 2. Run all tests
./run-tests-docker.sh run

# 3. Run tests with coverage
./run-tests-docker.sh coverage
```

## Test Environment Overview

The Docker test environment includes:
- **Isolated test database** (MariaDB on port 3307)
- **Django test container** with all dependencies
- **Coverage reporting** with HTML and XML output
- **PDF generation testing** with WeasyPrint/ReportLab
- **Parallel test execution** support

## Available Commands

### Setup and Management

```bash
# Build containers and setup test database
./run-tests-docker.sh setup

# Clean up all test containers and data
./run-tests-docker.sh clean

# View test container logs
./run-tests-docker.sh logs

# Open shell in test container
./run-tests-docker.sh shell
```

### Running Tests

```bash
# Run all tests
./run-tests-docker.sh run

# Run specific test suites
./run-tests-docker.sh run models      # Model tests only
./run-tests-docker.sh run services    # Service layer tests
./run-tests-docker.sh run auth        # Authentication tests
./run-tests-docker.sh run views       # View and URL tests
./run-tests-docker.sh run pdf         # PDF generation tests
./run-tests-docker.sh run integration # Integration tests

# Run quick tests (models + services)
./run-tests-docker.sh quick

# Run tests with coverage reporting
./run-tests-docker.sh coverage
```

## Advanced Test Options

### Using Docker Compose Directly

```bash
# Run specific test with custom options
docker-compose -f docker-compose.test.yml run --rm cmc-django-test \
    python cmcsales/manage.py test cmc.tests.test_models --verbosity=2 --keepdb

# Run tests with coverage
docker-compose -f docker-compose.test.yml run --rm cmc-django-test \
    coverage run --source='.' cmcsales/manage.py test cmc.tests

# Generate coverage report
docker-compose -f docker-compose.test.yml run --rm cmc-django-test \
    coverage report --show-missing
```

### Using the Test Script Directly

```bash
# Inside the container, you can use the test script with advanced options
docker-compose -f docker-compose.test.yml run --rm cmc-django-test \
    /app/scripts/run-tests.sh --coverage --keepdb --failfast models

# Script options:
# -c, --coverage         Enable coverage reporting
# -k, --keepdb          Keep test database between runs
# -p, --parallel NUM    Run tests in parallel
# -f, --failfast        Stop on first failure
# -v, --verbosity NUM   Verbosity level 0-3
```

## Test Suite Structure

### 1. Model Tests (`test_models.py`)
Tests all Django models including:
- Customer, Enquiry, Job, Document models
- Model validation and constraints
- Relationships and cascade behavior
- Financial calculations

```bash
./run-tests-docker.sh run models
```

### 2. Service Tests (`test_services.py`)
Tests business logic layer:
- Number generation service
- Financial calculation service
- Document service workflows
- Validation service

```bash
./run-tests-docker.sh run services
```

### 3. Authentication Tests (`test_authentication.py`)
Tests authentication system:
- Multiple authentication backends
- Permission decorators and middleware
- User management workflows
- Security features

```bash
./run-tests-docker.sh run auth
```

### 4. View Tests (`test_views.py`)
Tests web interface:
- CRUD operations for all entities
- AJAX endpoints
- Permission enforcement
- URL routing

```bash
./run-tests-docker.sh run views
```

### 5. PDF Tests (`test_pdf.py`)
Tests PDF generation:
- WeasyPrint and ReportLab engines
- Template rendering
- Document formatting
- Security and performance

```bash
./run-tests-docker.sh run pdf
```

### 6. Integration Tests (`test_integration.py`)
Tests complete workflows:
- End-to-end business processes
- Multi-user collaboration
- System integration scenarios
- Performance and security

```bash
./run-tests-docker.sh run integration
```

## Test Reports and Coverage

### Coverage Reports

After running tests with coverage, reports are available in:
- **HTML Report**: `./coverage-reports/html/index.html`
- **XML Report**: `./coverage-reports/coverage.xml`
- **Console**: Displayed after test run

```bash
# Run tests with coverage
./run-tests-docker.sh coverage

# View HTML report
open coverage-reports/html/index.html
```

### Test Artifacts

Test outputs are saved to:
- **Test Reports**: `./test-reports/`
- **Coverage Reports**: `./coverage-reports/`
- **Logs**: `./logs/`
- **PDF Test Files**: `./test-reports/pdf/`

## Configuration

### Environment Variables

The test environment uses these key variables:

```yaml
# Database configuration
DATABASE_HOST: test-db
DATABASE_NAME: test_cmc
DATABASE_USER: test_cmc
DATABASE_PASSWORD: testPassword123

# Django settings
DJANGO_SETTINGS_MODULE: cmcsales.settings
TESTING: 1
DEBUG: 0

# PDF generation
PDF_GENERATION_ENGINE: weasyprint
PDF_SAVE_DIRECTORY: /app/test-reports/pdf
```

### Test Database

- **Isolated database** separate from development/production
- **Runs on port 3307** to avoid conflicts
- **Optimized for testing** with reduced buffer sizes
- **Automatically reset** between test runs (unless `--keepdb` used)

## Performance Optimization

### Parallel Test Execution

```bash
# Run tests in parallel (faster execution)
docker-compose -f docker-compose.test.yml run --rm cmc-django-test \
    /app/scripts/run-tests.sh --parallel=4 all
```

### Keeping Test Database

```bash
# Keep database between runs for faster subsequent tests
./run-tests-docker.sh run models --keepdb
```

### Quick Test Suite

```bash
# Run only essential tests for rapid feedback
./run-tests-docker.sh quick
```

## Continuous Integration

### GitHub Actions Example

```yaml
name: Test CMC Django
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Setup test environment
        run: ./run-tests-docker.sh setup
        
      - name: Run tests with coverage
        run: ./run-tests-docker.sh coverage
        
      - name: Upload coverage reports
        uses: codecov/codecov-action@v3
        with:
          file: ./coverage-reports/coverage.xml
```

## Troubleshooting

### Database Connection Issues

```bash
# Check database status
docker-compose -f docker-compose.test.yml ps

# View database logs
docker-compose -f docker-compose.test.yml logs test-db

# Restart database
docker-compose -f docker-compose.test.yml restart test-db
```

### Test Failures

```bash
# Run with maximum verbosity for debugging
docker-compose -f docker-compose.test.yml run --rm cmc-django-test \
    python cmcsales/manage.py test cmc.tests.test_models --verbosity=3

# Use failfast to stop on first error
./run-tests-docker.sh run models --failfast

# Open shell to investigate
./run-tests-docker.sh shell
```

### Permission Issues

```bash
# Fix file permissions
sudo chown -R $USER:$USER test-reports coverage-reports logs

# Check Docker permissions
docker-compose -f docker-compose.test.yml run --rm cmc-django-test whoami
```

### Memory Issues

```bash
# Run tests with reduced parallel workers
docker-compose -f docker-compose.test.yml run --rm cmc-django-test \
    /app/scripts/run-tests.sh --parallel=1 all

# Monitor resource usage
docker stats
```

## Development Workflow

### Recommended Testing Workflow

1. **Initial Setup** (one-time):
   ```bash
   ./run-tests-docker.sh setup
   ```

2. **During Development** (fast feedback):
   ```bash
   ./run-tests-docker.sh quick --keepdb
   ```

3. **Before Commit** (comprehensive):
   ```bash
   ./run-tests-docker.sh coverage
   ```

4. **Debugging Issues**:
   ```bash
   ./run-tests-docker.sh shell
   # Inside container:
   python cmcsales/manage.py test cmc.tests.test_models.CustomerModelTest.test_customer_creation --verbosity=3
   ```

### Adding New Tests

1. Create test file in appropriate module
2. Follow existing test patterns and base classes
3. Test locally:
   ```bash
   ./run-tests-docker.sh run models --keepdb
   ```
4. Run full suite before committing:
   ```bash
   ./run-tests-docker.sh coverage
   ```

## Integration with IDE

### PyCharm/IntelliJ

Configure remote interpreter using Docker:
1. Go to Settings → Project → Python Interpreter
2. Add Docker Compose interpreter
3. Use `docker-compose.test.yml` configuration
4. Set service to `cmc-django-test`

### VS Code

Use Dev Containers extension:
1. Create `.devcontainer/devcontainer.json`
2. Configure to use test Docker environment
3. Run tests directly in integrated terminal

## Best Practices

1. **Always run tests in Docker** for consistency
2. **Use `--keepdb` during development** for speed
3. **Run coverage reports before commits**
4. **Clean up regularly** to free disk space
5. **Monitor test performance** and optimize slow tests
6. **Use parallel execution** for large test suites
7. **Keep test data realistic** but minimal
8. **Test error conditions** as well as happy paths

## Resources

- **Django Testing Documentation**: https://docs.djangoproject.com/en/5.1/topics/testing/
- **Coverage.py Documentation**: https://coverage.readthedocs.io/
- **Docker Compose Reference**: https://docs.docker.com/compose/
- **CMC Test Suite Documentation**: See individual test modules for detailed information