xpeditis2.0/apps/backend/MINIO_SETUP_SUMMARY.md

MinIO Document Storage Setup Summary

Problem

Documents uploaded to MinIO were returning AccessDenied errors when users tried to download them from the admin documents page.

Root Cause

The xpeditis-documents bucket did not have a public read policy configured, which prevented direct URL access to uploaded documents.

Solution Implemented

1. Fixed Dummy URLs in Database

Script: fix-dummy-urls.js

• Updated 2 bookings that had dummy URLs (https://dummy-storage.com/...)
• Changed to proper MinIO URLs: http://localhost:9000/xpeditis-documents/csv-bookings/{bookingId}/{documentId}-{fileName}

2. Uploaded Test Documents

Script: upload-test-documents.js

• Created 54 test PDF documents
• Uploaded to MinIO with proper paths matching database records
• Files are minimal valid PDFs for testing purposes

3. Set Bucket Policy for Public Read Access

Script: set-bucket-policy.js

• Configured the xpeditis-documents bucket with a policy allowing public read access
• Policy applied:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": ["s3:GetObject"],
      "Resource": ["arn:aws:s3:::xpeditis-documents/*"]
    }
  ]
}

Verification

Test Document Download

# Test with curl (should return HTTP 200 OK)
curl -I http://localhost:9000/xpeditis-documents/csv-bookings/70f6802a-f789-4f61-ab35-5e0ebf0e29d5/eba1c60f-c749-4b39-8e26-dcc617964237-Document_Export.pdf

# Download actual file
curl -o test.pdf http://localhost:9000/xpeditis-documents/csv-bookings/70f6802a-f789-4f61-ab35-5e0ebf0e29d5/eba1c60f-c749-4b39-8e26-dcc617964237-Document_Export.pdf

Frontend Verification

1. Navigate to: http://localhost:3000/dashboard/admin/documents
2. Click the "Download" button on any document
3. Document should download successfully without errors

MinIO Console Access

• URL: http://localhost:9001
• Username: minioadmin
• Password: minioadmin

You can view the bucket policy and uploaded files directly in the MinIO console.

Files Created

• apps/backend/fix-dummy-urls.js - Updates database URLs from dummy to MinIO
• apps/backend/upload-test-documents.js - Uploads test PDFs to MinIO
• apps/backend/set-bucket-policy.js - Configures bucket policy for public read

Running the Scripts

cd apps/backend

# 1. Fix database URLs (run once)
node fix-dummy-urls.js

# 2. Upload test documents (run once)
node upload-test-documents.js

# 3. Set bucket policy (run once)
node set-bucket-policy.js

Important Notes

Development vs Production

• Current Setup: Public read access (suitable for development)
• Production: Consider using signed URLs for better security

Signed URLs (Production Recommendation)

Instead of public bucket access, generate temporary signed URLs via the backend:

// Backend endpoint to generate signed URL
@Get('documents/:id/download-url')
async getDownloadUrl(@Param('id') documentId: string) {
  const document = await this.documentsService.findOne(documentId);
  const signedUrl = await this.storageService.getSignedUrl(document.filePath);
  return { url: signedUrl };
}

This approach:

• ✅ More secure (temporary URLs that expire)
• ✅ Allows access control (check user permissions before generating URL)
• ✅ Audit trail (log who accessed what)
• ❌ Requires backend API call for each download

Current Architecture

The S3StorageAdapter already has a getSignedUrl() method implemented (line 148-162 in s3-storage.adapter.ts), so migrating to signed URLs in the future is straightforward.

Troubleshooting

AccessDenied Error Returns

If you get AccessDenied errors again:

1. Check bucket policy: node -e "const {S3Client,GetBucketPolicyCommand}=require('@aws-sdk/client-s3');const s3=new S3Client({endpoint:'http://localhost:9000',region:'us-east-1',credentials:{accessKeyId:'minioadmin',secretAccessKey:'minioadmin'},forcePathStyle:true});s3.send(new GetBucketPolicyCommand({Bucket:'xpeditis-documents'})).then(r=>console.log(r.Policy))"
2. Re-run: node set-bucket-policy.js

Document Not Found

If document URLs return 404:

1. Check MinIO console (http://localhost:9001)
2. Verify file exists in bucket
3. Check database URL matches MinIO path exactly

Documents Not Showing in Admin Page

1. Verify bookings exist: SELECT id, documents FROM csv_bookings WHERE documents IS NOT NULL
2. Check frontend console for errors
3. Verify API endpoint returns data: http://localhost:4000/api/v1/admin/bookings

Database Query Examples

Check Document URLs

SELECT
  id,
  booking_id as "bookingId",
  documents::jsonb->0->>'filePath' as "firstDocumentUrl"
FROM csv_bookings
WHERE documents IS NOT NULL
LIMIT 5;

Count Documents by Booking

SELECT
  id,
  jsonb_array_length(documents::jsonb) as "documentCount"
FROM csv_bookings
WHERE documents IS NOT NULL;

Next Steps (Optional Production Enhancements)

1. Implement Signed URLs

   • Create backend endpoint for signed URL generation
   • Update frontend to fetch signed URL before download
   • Remove public bucket policy

2. Add Document Permissions

   • Check user permissions before generating download URL
   • Restrict access based on organization membership

3. Implement Audit Trail

   • Log document access events
   • Track who downloaded what and when

4. Add Document Scanning

   • Virus scanning on upload (ClamAV)
   • Content validation
   • File size limits enforcement

Status

✅ FIXED - Documents can now be downloaded from the admin documents page without AccessDenied errors.