import { Injectable, Logger } from '@nestjs/common'; import * as fs from 'fs/promises'; import * as path from 'path'; /** * CSV Converter Service * * Détecte automatiquement le format du CSV et convertit au format attendu * Supporte: * - Format standard Xpeditis * - Format "Frais FOB FRET" */ @Injectable() export class CsvConverterService { private readonly logger = new Logger(CsvConverterService.name); // Headers du format standard attendu private readonly STANDARD_HEADERS = [ 'companyName', 'origin', 'destination', 'containerType', 'minVolumeCBM', 'maxVolumeCBM', 'minWeightKG', 'maxWeightKG', 'palletCount', 'pricePerCBM', 'pricePerKG', 'basePriceUSD', 'basePriceEUR', 'currency', 'hasSurcharges', 'surchargeBAF', 'surchargeCAF', 'surchargeDetails', 'transitDays', 'validFrom', 'validUntil', ]; // Headers du format "Frais FOB FRET" private readonly FOB_FRET_HEADERS = [ 'Origine UN code', 'Destination UN code', 'Devise FRET', 'Taux de FRET (UP)', 'Minimum FRET (LS)', 'Transit time', ]; /** * Parse une ligne CSV en gérant les champs entre guillemets */ private parseCSVLine(line: string): string[] { const result: string[] = []; let current = ''; let inQuotes = false; for (let i = 0; i < line.length; i++) { const char = line[i]; if (char === '"') { inQuotes = !inQuotes; } else if (char === ',' && !inQuotes) { result.push(current.trim()); current = ''; } else { current += char; } } result.push(current.trim()); return result; } /** * Détecte le format du CSV */ async detectFormat(filePath: string): Promise<'STANDARD' | 'FOB_FRET' | 'UNKNOWN'> { try { const content = await fs.readFile(filePath, 'utf-8'); const lines = content.split('\n').filter(line => line.trim()); if (lines.length === 0) { return 'UNKNOWN'; } // Vérifier les 2 premières lignes (parfois la vraie ligne d'en-tête est la ligne 2) for (let i = 0; i < Math.min(2, lines.length); i++) { const headers = this.parseCSVLine(lines[i]); // Vérifier format standard const hasStandardHeaders = this.STANDARD_HEADERS.some(h => headers.includes(h)); if (hasStandardHeaders) { return 'STANDARD'; } // Vérifier format FOB FRET const hasFobFretHeaders = this.FOB_FRET_HEADERS.some(h => headers.includes(h)); if (hasFobFretHeaders) { return 'FOB_FRET'; } } return 'UNKNOWN'; } catch (error) { const errorMessage = error instanceof Error ? error.message : 'Unknown error'; this.logger.error(`Error detecting CSV format: ${errorMessage}`); return 'UNKNOWN'; } } /** * Calcule les surcharges à partir des colonnes FOB */ private calculateSurcharges(row: Record): string { const surcharges: string[] = []; const surchargeFields = [ { key: 'Documentation (LS et Minimum)', prefix: 'DOC' }, { key: 'ISPS (LS et Minimum)', prefix: 'ISPS' }, { key: 'Manutention', prefix: 'HANDLING' }, { key: 'Solas (LS et Minimum)', prefix: 'SOLAS' }, { key: 'Douane (LS et Minimum)', prefix: 'CUSTOMS' }, { key: 'AMS/ACI (LS et Minimum)', prefix: 'AMS_ACI' }, { key: 'ISF5 (LS et Minimum)', prefix: 'ISF5' }, { key: 'Frais admin de dangereux (LS et Minimum)', prefix: 'DG_FEE' }, ]; surchargeFields.forEach(({ key, prefix }) => { if (row[key]) { const unit = key === 'Manutention' ? row['Unité de manutention (UP;Tonne)'] || 'UP' : ''; surcharges.push(`${prefix}:${row[key]}${unit ? ' ' + unit : ''}`); } }); return surcharges.join(' | '); } /** * Convertit une ligne FOB FRET vers le format standard */ private convertFobFretRow(row: Record, companyName: string): Record { const currency = row['Devise FRET'] || 'USD'; const freightRate = parseFloat(row['Taux de FRET (UP)']) || 0; const minFreight = parseFloat(row['Minimum FRET (LS)']) || 0; const transitDays = parseInt(row['Transit time']) || 0; // Calcul des surcharges const surchargeDetails = this.calculateSurcharges(row); const hasSurcharges = surchargeDetails.length > 0; // Dates de validité (90 jours par défaut) const validFrom = new Date().toISOString().split('T')[0]; const validUntil = new Date(Date.now() + 90 * 24 * 60 * 60 * 1000).toISOString().split('T')[0]; // Volumes et poids standards pour LCL const minVolumeCBM = 1; const maxVolumeCBM = 20; const minWeightKG = 100; const maxWeightKG = 20000; // Prix par CBM const pricePerCBM = freightRate > 0 ? freightRate : minFreight; // Prix par KG (estimation: prix CBM / 200 kg/m³) const pricePerKG = pricePerCBM > 0 ? (pricePerCBM / 200).toFixed(2) : '0'; return { companyName, origin: row['Origine UN code'] || '', destination: row['Destination UN code'] || '', containerType: 'LCL', minVolumeCBM, maxVolumeCBM, minWeightKG, maxWeightKG, palletCount: 0, pricePerCBM, pricePerKG, basePriceUSD: currency === 'USD' ? pricePerCBM : 0, basePriceEUR: currency === 'EUR' ? pricePerCBM : 0, currency, hasSurcharges, surchargeBAF: '', surchargeCAF: '', surchargeDetails, transitDays, validFrom, validUntil, }; } /** * Convertit un CSV FOB FRET vers le format standard */ async convertFobFretToStandard( inputPath: string, companyName: string ): Promise<{ outputPath: string; rowsConverted: number }> { this.logger.log(`Converting FOB FRET CSV: ${inputPath}`); try { // Lire le fichier const content = await fs.readFile(inputPath, 'utf-8'); const lines = content.split('\n').filter(line => line.trim()); if (lines.length < 2) { throw new Error('CSV file is empty or has no data rows'); } // Trouver la ligne d'en-tête réelle (chercher celle avec "Devise FRET") let headerLineIndex = 0; for (let i = 0; i < Math.min(2, lines.length); i++) { const headers = this.parseCSVLine(lines[i]); if (this.FOB_FRET_HEADERS.some(h => headers.includes(h))) { headerLineIndex = i; break; } } // Parse headers const headers = this.parseCSVLine(lines[headerLineIndex]); this.logger.log(`Found FOB FRET headers at line ${headerLineIndex + 1}`); // Parse data rows (commencer après la ligne d'en-tête) const dataRows: Record[] = []; for (let i = headerLineIndex + 1; i < lines.length; i++) { const values = this.parseCSVLine(lines[i]); const row: Record = {}; headers.forEach((header, index) => { row[header] = values[index] || ''; }); // Vérifier que la ligne a des données valides if (row['Origine UN code'] && row['Destination UN code']) { dataRows.push(row); } } this.logger.log(`Found ${dataRows.length} valid data rows`); // Convertir les lignes const convertedRows = dataRows.map(row => this.convertFobFretRow(row, companyName)); // Générer le CSV de sortie const outputLines: string[] = [this.STANDARD_HEADERS.join(',')]; convertedRows.forEach(row => { const values = this.STANDARD_HEADERS.map(header => { const value = row[header]; // Échapper les virgules et quotes if ( typeof value === 'string' && (value.includes(',') || value.includes('"') || value.includes('\n')) ) { return `"${value.replace(/"/g, '""')}"`; } return value; }); outputLines.push(values.join(',')); }); // Écrire le fichier converti (garder le chemin absolu) const outputPath = inputPath.replace('.csv', '-converted.csv'); const absoluteOutputPath = path.isAbsolute(outputPath) ? outputPath : path.resolve(process.cwd(), outputPath); await fs.writeFile(absoluteOutputPath, outputLines.join('\n'), 'utf-8'); this.logger.log(`Conversion completed: ${absoluteOutputPath} (${convertedRows.length} rows)`); return { outputPath: absoluteOutputPath, rowsConverted: convertedRows.length, }; } catch (error) { const errorMessage = error instanceof Error ? error.message : 'Unknown error'; const errorStack = error instanceof Error ? error.stack : undefined; this.logger.error(`Error converting CSV: ${errorMessage}`, errorStack); throw new Error(`CSV conversion failed: ${errorMessage}`); } } /** * Convertit automatiquement un CSV si nécessaire */ async autoConvert( inputPath: string, companyName: string ): Promise<{ convertedPath: string; wasConverted: boolean; rowsConverted?: number }> { const format = await this.detectFormat(inputPath); this.logger.log(`Detected CSV format: ${format}`); if (format === 'STANDARD') { return { convertedPath: inputPath, wasConverted: false, }; } if (format === 'FOB_FRET') { const result = await this.convertFobFretToStandard(inputPath, companyName); return { convertedPath: result.outputPath, wasConverted: true, rowsConverted: result.rowsConverted, }; } throw new Error(`Unknown CSV format. Please provide a valid CSV file.`); } }