xpeditis2.0/DEPLOYMENT.md

# Xpeditis 2.0 - Deployment Guide

## 📋 Table of Contents

1. [Prerequisites](#prerequisites)
2. [Environment Variables](#environment-variables)
3. [Local Development](#local-development)
4. [Database Migrations](#database-migrations)
5. [Docker Deployment](#docker-deployment)
6. [Production Deployment](#production-deployment)
7. [CI/CD Pipeline](#cicd-pipeline)
8. [Monitoring Setup](#monitoring-setup)
9. [Backup & Recovery](#backup--recovery)
10. [Troubleshooting](#troubleshooting)

---

## Prerequisites

### System Requirements

- **Node.js**: 20.x LTS
- **npm**: 10.x or higher
- **PostgreSQL**: 15.x or higher
- **Redis**: 7.x or higher
- **Docker**: 24.x (optional, for containerized deployment)
- **Docker Compose**: 2.x (optional)

### Development Tools

```bash
# Install Node.js (via nvm recommended)
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
nvm install 20
nvm use 20

# Verify installation
node --version  # Should be 20.x
npm --version   # Should be 10.x
```

---

## Environment Variables

### Backend (.env)

Create `apps/backend/.env`:

```bash
# Environment
NODE_ENV=production  # development | production | test

# Server
PORT=4000
API_PREFIX=api/v1

# Frontend URL
FRONTEND_URL=https://app.xpeditis.com

# Database
DATABASE_HOST=your-postgres-host.rds.amazonaws.com
DATABASE_PORT=5432
DATABASE_USER=xpeditis_user
DATABASE_PASSWORD=your-secure-password
DATABASE_NAME=xpeditis_prod
DATABASE_SYNC=false  # NEVER true in production
DATABASE_LOGGING=false

# Redis Cache
REDIS_HOST=your-redis-host.elasticache.amazonaws.com
REDIS_PORT=6379
REDIS_PASSWORD=your-redis-password
REDIS_TLS=true

# JWT Authentication
JWT_SECRET=your-jwt-secret-min-32-characters-long
JWT_ACCESS_EXPIRATION=15m
JWT_REFRESH_SECRET=your-refresh-secret-min-32-characters
JWT_REFRESH_EXPIRATION=7d

# Session
SESSION_SECRET=your-session-secret-min-32-characters

# Email (SMTP)
SMTP_HOST=smtp.sendgrid.net
SMTP_PORT=587
SMTP_USER=apikey
SMTP_PASSWORD=your-sendgrid-api-key
SMTP_FROM=noreply@xpeditis.com

# S3 Storage (AWS)
AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
S3_BUCKET=xpeditis-documents-prod
S3_ENDPOINT=  # Optional, for MinIO

# Sentry Monitoring
SENTRY_DSN=https://your-sentry-dsn@sentry.io/project-id
SENTRY_ENVIRONMENT=production
SENTRY_TRACES_SAMPLE_RATE=0.1
SENTRY_PROFILES_SAMPLE_RATE=0.05

# Rate Limiting
RATE_LIMIT_GLOBAL_TTL=60
RATE_LIMIT_GLOBAL_LIMIT=100

# Carrier API Keys (examples)
MAERSK_API_KEY=your-maersk-api-key
MSC_API_KEY=your-msc-api-key
CMA_CGM_API_KEY=your-cma-api-key

# Logging
LOG_LEVEL=info  # debug | info | warn | error
```

### Frontend (.env.local)

Create `apps/frontend/.env.local`:

```bash
# API Configuration
NEXT_PUBLIC_API_URL=https://api.xpeditis.com/api/v1
NEXT_PUBLIC_WS_URL=wss://api.xpeditis.com

# Sentry (Frontend)
NEXT_PUBLIC_SENTRY_DSN=https://your-frontend-sentry-dsn@sentry.io/project-id
NEXT_PUBLIC_SENTRY_ENVIRONMENT=production

# Feature Flags (optional)
NEXT_PUBLIC_ENABLE_ANALYTICS=true
NEXT_PUBLIC_ENABLE_CHAT=false

# Google Analytics (optional)
NEXT_PUBLIC_GA_ID=G-XXXXXXXXXX
```

### Security Best Practices

1. **Never commit .env files**: Add to `.gitignore`
2. **Use secrets management**: AWS Secrets Manager, HashiCorp Vault
3. **Rotate secrets regularly**: Every 90 days minimum
4. **Use strong passwords**: Min 32 characters, random
5. **Encrypt at rest**: Use AWS KMS, GCP KMS

---

## Local Development

### 1. Clone Repository

```bash
git clone https://github.com/your-org/xpeditis2.0.git
cd xpeditis2.0
```

### 2. Install Dependencies

```bash
# Install root dependencies
npm install

# Install backend dependencies
cd apps/backend
npm install

# Install frontend dependencies
cd ../frontend
npm install

cd ../..
```

### 3. Setup Local Database

```bash
# Using Docker
docker run --name xpeditis-postgres \
  -e POSTGRES_USER=xpeditis_user \
  -e POSTGRES_PASSWORD=dev_password \
  -e POSTGRES_DB=xpeditis_dev \
  -p 5432:5432 \
  -d postgres:15-alpine

# Or install PostgreSQL locally
# macOS: brew install postgresql@15
# Ubuntu: sudo apt install postgresql-15

# Create database
psql -U postgres
CREATE DATABASE xpeditis_dev;
CREATE USER xpeditis_user WITH ENCRYPTED PASSWORD 'dev_password';
GRANT ALL PRIVILEGES ON DATABASE xpeditis_dev TO xpeditis_user;
```

### 4. Setup Local Redis

```bash
# Using Docker
docker run --name xpeditis-redis \
  -p 6379:6379 \
  -d redis:7-alpine

# Or install Redis locally
# macOS: brew install redis
# Ubuntu: sudo apt install redis-server
```

### 5. Run Database Migrations

```bash
cd apps/backend

# Run all migrations
npm run migration:run

# Generate new migration (if needed)
npm run migration:generate -- -n MigrationName

# Revert last migration
npm run migration:revert
```

### 6. Start Development Servers

```bash
# Terminal 1: Backend
cd apps/backend
npm run start:dev

# Terminal 2: Frontend
cd apps/frontend
npm run dev
```

### 7. Access Application

- **Frontend**: http://localhost:3000
- **Backend API**: http://localhost:4000/api/v1
- **API Docs**: http://localhost:4000/api/docs

---

## Database Migrations

### Migration Files Location

```
apps/backend/src/infrastructure/persistence/typeorm/migrations/
```

### Running Migrations

```bash
# Production
npm run migration:run

# Check migration status
npm run migration:show

# Revert last migration (use with caution!)
npm run migration:revert
```

### Creating Migrations

```bash
# Generate from entity changes
npm run migration:generate -- -n AddUserProfileFields

# Create empty migration
npm run migration:create -- -n CustomMigration
```

### Migration Best Practices

1. **Always test locally first**
2. **Backup database before production migrations**
3. **Never edit existing migrations** (create new ones)
4. **Keep migrations idempotent** (safe to run multiple times)
5. **Add rollback logic** in `down()` method

---

## Docker Deployment

### Build Docker Images

```bash
# Backend
cd apps/backend
docker build -t xpeditis-backend:latest .

# Frontend
cd ../frontend
docker build -t xpeditis-frontend:latest .
```

### Docker Compose (Full Stack)

Create `docker-compose.yml`:

```yaml
version: '3.8'

services:
  postgres:
    image: postgres:15-alpine
    environment:
      POSTGRES_USER: xpeditis_user
      POSTGRES_PASSWORD: dev_password
      POSTGRES_DB: xpeditis_dev
    volumes:
      - postgres_data:/var/lib/postgresql/data
    ports:
      - '5432:5432'

  redis:
    image: redis:7-alpine
    ports:
      - '6379:6379'

  backend:
    image: xpeditis-backend:latest
    depends_on:
      - postgres
      - redis
    env_file:
      - apps/backend/.env
    ports:
      - '4000:4000'

  frontend:
    image: xpeditis-frontend:latest
    depends_on:
      - backend
    env_file:
      - apps/frontend/.env.local
    ports:
      - '3000:3000'

volumes:
  postgres_data:
```

### Run with Docker Compose

```bash
# Start all services
docker-compose up -d

# View logs
docker-compose logs -f

# Stop all services
docker-compose down

# Rebuild and restart
docker-compose up -d --build
```

---

## Production Deployment

### AWS Deployment (Recommended)

#### 1. Infrastructure Setup (Terraform)

```hcl
# main.tf (example)
provider "aws" {
  region = "us-east-1"
}

module "vpc" {
  source = "terraform-aws-modules/vpc/aws"
  # ... VPC configuration
}

module "rds" {
  source = "terraform-aws-modules/rds/aws"
  engine = "postgres"
  engine_version = "15.3"
  instance_class = "db.t3.medium"
  allocated_storage = 100
  # ... RDS configuration
}

module "elasticache" {
  source = "terraform-aws-modules/elasticache/aws"
  cluster_id = "xpeditis-redis"
  engine = "redis"
  node_type = "cache.t3.micro"
  # ... ElastiCache configuration
}

module "ecs" {
  source = "terraform-aws-modules/ecs/aws"
  cluster_name = "xpeditis-cluster"
  # ... ECS configuration
}
```

#### 2. Deploy Backend to ECS

```bash
# 1. Build and push Docker image to ECR
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin your-account-id.dkr.ecr.us-east-1.amazonaws.com

docker tag xpeditis-backend:latest your-account-id.dkr.ecr.us-east-1.amazonaws.com/xpeditis-backend:latest
docker push your-account-id.dkr.ecr.us-east-1.amazonaws.com/xpeditis-backend:latest

# 2. Update ECS task definition
aws ecs register-task-definition --cli-input-json file://task-definition.json

# 3. Update ECS service
aws ecs update-service --cluster xpeditis-cluster --service xpeditis-backend --task-definition xpeditis-backend:latest
```

#### 3. Deploy Frontend to Vercel/Netlify

```bash
# Vercel (recommended for Next.js)
npm install -g vercel
cd apps/frontend
vercel --prod

# Or Netlify
npm install -g netlify-cli
cd apps/frontend
npm run build
netlify deploy --prod --dir=out
```

#### 4. Configure Load Balancer

```bash
# Create Application Load Balancer
aws elbv2 create-load-balancer \
  --name xpeditis-alb \
  --subnets subnet-xxx subnet-yyy \
  --security-groups sg-xxx

# Create target group
aws elbv2 create-target-group \
  --name xpeditis-backend-tg \
  --protocol HTTP \
  --port 4000 \
  --vpc-id vpc-xxx

# Register targets
aws elbv2 register-targets \
  --target-group-arn arn:aws:elasticloadbalancing:... \
  --targets Id=i-xxx Id=i-yyy
```

#### 5. Setup SSL Certificate

```bash
# Request certificate from ACM
aws acm request-certificate \
  --domain-name api.xpeditis.com \
  --validation-method DNS

# Add HTTPS listener to ALB
aws elbv2 create-listener \
  --load-balancer-arn arn:aws:elasticloadbalancing:... \
  --protocol HTTPS \
  --port 443 \
  --certificates CertificateArn=arn:aws:acm:... \
  --default-actions Type=forward,TargetGroupArn=arn:...
```

---

## CI/CD Pipeline

### GitHub Actions Workflow

Create `.github/workflows/deploy.yml`:

```yaml
name: Deploy to Production

on:
  push:
    branches:
      - main

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '20'

      - name: Install dependencies
        run: |
          cd apps/backend
          npm ci          

      - name: Run tests
        run: |
          cd apps/backend
          npm test          

      - name: Run E2E tests
        run: |
          cd apps/frontend
          npm ci
          npx playwright install
          npm run test:e2e          

  deploy-backend:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v1

      - name: Build and push Docker image
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          ECR_REPOSITORY: xpeditis-backend
          IMAGE_TAG: ${{ github.sha }}
        run: |
          cd apps/backend
          docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG          

      - name: Update ECS service
        run: |
          aws ecs update-service \
            --cluster xpeditis-cluster \
            --service xpeditis-backend \
            --force-new-deployment          

  deploy-frontend:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '20'

      - name: Install Vercel CLI
        run: npm install -g vercel

      - name: Deploy to Vercel
        env:
          VERCEL_TOKEN: ${{ secrets.VERCEL_TOKEN }}
          VERCEL_ORG_ID: ${{ secrets.VERCEL_ORG_ID }}
          VERCEL_PROJECT_ID: ${{ secrets.VERCEL_PROJECT_ID }}
        run: |
          cd apps/frontend
          vercel --prod --token=$VERCEL_TOKEN          
```

---

## Monitoring Setup

### 1. Configure Sentry

```typescript
// apps/backend/src/main.ts
import { initializeSentry } from './infrastructure/monitoring/sentry.config';

initializeSentry({
  dsn: process.env.SENTRY_DSN,
  environment: process.env.NODE_ENV,
  tracesSampleRate: parseFloat(process.env.SENTRY_TRACES_SAMPLE_RATE || '0.1'),
  profilesSampleRate: parseFloat(process.env.SENTRY_PROFILES_SAMPLE_RATE || '0.05'),
  enabled: process.env.NODE_ENV === 'production',
});
```

### 2. Setup CloudWatch (AWS)

```bash
# Create log group
aws logs create-log-group --log-group-name /ecs/xpeditis-backend

# Create metric filter
aws logs put-metric-filter \
  --log-group-name /ecs/xpeditis-backend \
  --filter-name ErrorCount \
  --filter-pattern "ERROR" \
  --metric-transformations \
    metricName=ErrorCount,metricNamespace=Xpeditis,metricValue=1
```

### 3. Create Alarms

```bash
# High error rate alarm
aws cloudwatch put-metric-alarm \
  --alarm-name xpeditis-high-error-rate \
  --alarm-description "Alert when error rate exceeds 5%" \
  --metric-name ErrorCount \
  --namespace Xpeditis \
  --statistic Sum \
  --period 300 \
  --evaluation-periods 2 \
  --threshold 50 \
  --comparison-operator GreaterThanThreshold \
  --alarm-actions arn:aws:sns:us-east-1:xxx:ops-alerts
```

---

## Backup & Recovery

### Database Backups

```bash
# Automated backups (AWS RDS)
aws rds modify-db-instance \
  --db-instance-identifier xpeditis-prod \
  --backup-retention-period 30 \
  --preferred-backup-window "03:00-04:00"

# Manual snapshot
aws rds create-db-snapshot \
  --db-instance-identifier xpeditis-prod \
  --db-snapshot-identifier xpeditis-manual-snapshot-$(date +%Y%m%d)

# Restore from snapshot
aws rds restore-db-instance-from-db-snapshot \
  --db-instance-identifier xpeditis-restored \
  --db-snapshot-identifier xpeditis-manual-snapshot-20251014
```

### S3 Backups

```bash
# Enable versioning
aws s3api put-bucket-versioning \
  --bucket xpeditis-documents-prod \
  --versioning-configuration Status=Enabled

# Enable lifecycle policy (delete old versions after 90 days)
aws s3api put-bucket-lifecycle-configuration \
  --bucket xpeditis-documents-prod \
  --lifecycle-configuration file://lifecycle.json
```

---

## Troubleshooting

### Common Issues

#### 1. Database Connection Errors

```bash
# Check database status
aws rds describe-db-instances --db-instance-identifier xpeditis-prod

# Check security group rules
aws ec2 describe-security-groups --group-ids sg-xxx

# Test connection from ECS task
aws ecs execute-command \
  --cluster xpeditis-cluster \
  --task task-id \
  --container backend \
  --interactive \
  --command "/bin/sh"

# Inside container:
psql -h your-rds-endpoint -U xpeditis_user -d xpeditis_prod
```

#### 2. High Memory Usage

```bash
# Check ECS task metrics
aws cloudwatch get-metric-statistics \
  --namespace AWS/ECS \
  --metric-name MemoryUtilization \
  --dimensions Name=ServiceName,Value=xpeditis-backend \
  --start-time 2025-10-14T00:00:00Z \
  --end-time 2025-10-14T23:59:59Z \
  --period 3600 \
  --statistics Average

# Increase task memory
aws ecs register-task-definition --cli-input-json file://task-definition.json
# (edit memory from 512 to 1024)
```

#### 3. Rate Limiting Issues

```bash
# Check throttled requests in logs
aws logs filter-log-events \
  --log-group-name /ecs/xpeditis-backend \
  --filter-pattern "ThrottlerException"

# Adjust rate limits in .env
RATE_LIMIT_GLOBAL_LIMIT=200  # Increase from 100
```

---

## Health Checks

### Backend Health Endpoint

```typescript
// apps/backend/src/application/controllers/health.controller.ts
@Get('/health')
async healthCheck() {
  return {
    status: 'ok',
    timestamp: new Date().toISOString(),
    uptime: process.uptime(),
    database: await this.checkDatabase(),
    redis: await this.checkRedis(),
  };
}
```

### ALB Health Check Configuration

```bash
aws elbv2 modify-target-group \
  --target-group-arn arn:aws:elasticloadbalancing:... \
  --health-check-path /api/v1/health \
  --health-check-interval-seconds 30 \
  --health-check-timeout-seconds 5 \
  --healthy-threshold-count 2 \
  --unhealthy-threshold-count 3
```

---

## Pre-Launch Checklist

- [ ] All environment variables set
- [ ] Database migrations run
- [ ] SSL certificate configured
- [ ] DNS records updated
- [ ] Load balancer configured
- [ ] Health checks passing
- [ ] Monitoring and alerts setup
- [ ] Backup strategy tested
- [ ] Load testing completed
- [ ] Security audit passed
- [ ] Documentation complete
- [ ] Disaster recovery plan documented
- [ ] On-call rotation scheduled

---

*Document Version*: 1.0.0
*Last Updated*: October 14, 2025
*Author*: Xpeditis DevOps Team