xpeditis2.0/apps/backend/MINIO_SETUP_SUMMARY.md

# MinIO Document Storage Setup Summary

## Problem
Documents uploaded to MinIO were returning `AccessDenied` errors when users tried to download them from the admin documents page.

## Root Cause
The `xpeditis-documents` bucket did not have a public read policy configured, which prevented direct URL access to uploaded documents.

## Solution Implemented

### 1. Fixed Dummy URLs in Database
**Script**: `fix-dummy-urls.js`
- Updated 2 bookings that had dummy URLs (`https://dummy-storage.com/...`)
- Changed to proper MinIO URLs: `http://localhost:9000/xpeditis-documents/csv-bookings/{bookingId}/{documentId}-{fileName}`

### 2. Uploaded Test Documents
**Script**: `upload-test-documents.js`
- Created 54 test PDF documents
- Uploaded to MinIO with proper paths matching database records
- Files are minimal valid PDFs for testing purposes

### 3. Set Bucket Policy for Public Read Access
**Script**: `set-bucket-policy.js`
- Configured the `xpeditis-documents` bucket with a policy allowing public read access
- Policy applied:
```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": ["s3:GetObject"],
      "Resource": ["arn:aws:s3:::xpeditis-documents/*"]
    }
  ]
}
```

## Verification

### Test Document Download
```bash
# Test with curl (should return HTTP 200 OK)
curl -I http://localhost:9000/xpeditis-documents/csv-bookings/70f6802a-f789-4f61-ab35-5e0ebf0e29d5/eba1c60f-c749-4b39-8e26-dcc617964237-Document_Export.pdf

# Download actual file
curl -o test.pdf http://localhost:9000/xpeditis-documents/csv-bookings/70f6802a-f789-4f61-ab35-5e0ebf0e29d5/eba1c60f-c749-4b39-8e26-dcc617964237-Document_Export.pdf
```

### Frontend Verification
1. Navigate to: http://localhost:3000/dashboard/admin/documents
2. Click the "Download" button on any document
3. Document should download successfully without errors

## MinIO Console Access
- **URL**: http://localhost:9001
- **Username**: minioadmin
- **Password**: minioadmin

You can view the bucket policy and uploaded files directly in the MinIO console.

## Files Created
- `apps/backend/fix-dummy-urls.js` - Updates database URLs from dummy to MinIO
- `apps/backend/upload-test-documents.js` - Uploads test PDFs to MinIO
- `apps/backend/set-bucket-policy.js` - Configures bucket policy for public read

## Running the Scripts
```bash
cd apps/backend

# 1. Fix database URLs (run once)
node fix-dummy-urls.js

# 2. Upload test documents (run once)
node upload-test-documents.js

# 3. Set bucket policy (run once)
node set-bucket-policy.js
```

## Important Notes

### Development vs Production
- **Current Setup**: Public read access (suitable for development)
- **Production**: Consider using signed URLs for better security

### Signed URLs (Production Recommendation)
Instead of public bucket access, generate temporary signed URLs via the backend:

```typescript
// Backend endpoint to generate signed URL
@Get('documents/:id/download-url')
async getDownloadUrl(@Param('id') documentId: string) {
  const document = await this.documentsService.findOne(documentId);
  const signedUrl = await this.storageService.getSignedUrl(document.filePath);
  return { url: signedUrl };
}
```

This approach:
- ✅ More secure (temporary URLs that expire)
- ✅ Allows access control (check user permissions before generating URL)
- ✅ Audit trail (log who accessed what)
- ❌ Requires backend API call for each download

### Current Architecture
The `S3StorageAdapter` already has a `getSignedUrl()` method implemented (line 148-162 in `s3-storage.adapter.ts`), so migrating to signed URLs in the future is straightforward.

## Troubleshooting

### AccessDenied Error Returns
If you get AccessDenied errors again:
1. Check bucket policy: `node -e "const {S3Client,GetBucketPolicyCommand}=require('@aws-sdk/client-s3');const s3=new S3Client({endpoint:'http://localhost:9000',region:'us-east-1',credentials:{accessKeyId:'minioadmin',secretAccessKey:'minioadmin'},forcePathStyle:true});s3.send(new GetBucketPolicyCommand({Bucket:'xpeditis-documents'})).then(r=>console.log(r.Policy))"`
2. Re-run: `node set-bucket-policy.js`

### Document Not Found
If document URLs return 404:
1. Check MinIO console (http://localhost:9001)
2. Verify file exists in bucket
3. Check database URL matches MinIO path exactly

### Documents Not Showing in Admin Page
1. Verify bookings exist: `SELECT id, documents FROM csv_bookings WHERE documents IS NOT NULL`
2. Check frontend console for errors
3. Verify API endpoint returns data: http://localhost:4000/api/v1/admin/bookings

## Database Query Examples

### Check Document URLs
```sql
SELECT
  id,
  booking_id as "bookingId",
  documents::jsonb->0->>'filePath' as "firstDocumentUrl"
FROM csv_bookings
WHERE documents IS NOT NULL
LIMIT 5;
```

### Count Documents by Booking
```sql
SELECT
  id,
  jsonb_array_length(documents::jsonb) as "documentCount"
FROM csv_bookings
WHERE documents IS NOT NULL;
```

## Next Steps (Optional Production Enhancements)

1. **Implement Signed URLs**
   - Create backend endpoint for signed URL generation
   - Update frontend to fetch signed URL before download
   - Remove public bucket policy

2. **Add Document Permissions**
   - Check user permissions before generating download URL
   - Restrict access based on organization membership

3. **Implement Audit Trail**
   - Log document access events
   - Track who downloaded what and when

4. **Add Document Scanning**
   - Virus scanning on upload (ClamAV)
   - Content validation
   - File size limits enforcement

## Status
✅ **FIXED** - Documents can now be downloaded from the admin documents page without AccessDenied errors.