# LocalStorageService Documentation

## Overview

`LocalStorageService` is a unified service for local filesystem operations, designed to replace scattered `File::` and `Storage::` facade calls throughout the application. It provides safe, configurable methods for scanning, filtering, and managing files on the local filesystem.

## Purpose

- **Centralize** filesystem operations in one testable service
- **Replace** direct `File::` and `Storage::disk('local')` calls
- **Provide** safe file deletion with retention policies
- **Enable** easy mocking in tests
- **Support** temporary file cleanup and maintenance operations

---

## Location

```
src/App/Services/LocalStorageService.php
```

---

## Key Features

### 1. **Safe File Scanning**
- Recursive directory traversal
- Pattern-based filtering (extensions, exclusions)
- Age-based filtering (retention + safety window)
- Lock file detection
- Symlink handling

### 2. **Batch Deletion**
- Process files in configurable batches
- Track bytes reclaimed
- Dry-run mode support
- Detailed error reporting

### 3. **Directory Management**
- Size calculation
- File counting
- Empty directory cleanup
- Ensure directory exists

### 4. **Telemetry**
- Deleted file count
- Failed deletion tracking
- Bytes reclaimed calculation
- Formatted size output

---

## Methods

### `scanDirectory(string $path, array $options = []): array`

Scan a directory and return files matching criteria.

**Parameters:**
- `$path` - Base directory path to scan
- `$options` - Array of scan options:
  - `recursive` (bool) - Scan subdirectories (default: `true`)
  - `older_than_hours` (int|null) - Only include files older than X hours (default: `null`)
  - `safety_window_minutes` (int) - Exclude files modified within X minutes (default: `10`)
  - `extensions` (array) - Filter by extensions, e.g., `['csv', 'pdf']` (default: `[]`)
  - `exclusions` (array) - Glob patterns to exclude, e.g., `['*.lock', '.gitignore']` (default: `['*.lock', '.gitignore', '.placeholder']`)
  - `skip_symlinks` (bool) - Skip symbolic links (default: `true`)
  - `skip_locked` (bool) - Skip files with lock files (default: `true`)
  - `max_files` (int|null) - Maximum files to scan (default: `null`)

**Returns:**
Array of file information:
```php
[
    [
        'path' => '/full/path/to/file.csv',
        'name' => 'file.csv',
        'size' => 1024,
        'modified_at' => Carbon instance,
        'extension' => 'csv',
    ],
    // ...
]
```

**Example:**
```php
use App\Services\LocalStorageService;

$service = new LocalStorageService();

// Scan temp directory for old CSV files
$files = $service->scanDirectory(storage_path('app/temp'), [
    'older_than_hours' => 24,
    'safety_window_minutes' => 10,
    'extensions' => ['csv'],
    'exclusions' => ['*.lock', '.gitignore'],
]);

echo "Found " . count($files) . " files to clean up";
```

---

### `deleteFiles(array $files, int $batchSize = 500, bool $dryRun = false): array`

Delete files in batches with telemetry.

**Parameters:**
- `$files` - Array of file information from `scanDirectory()`
- `$batchSize` - Number of files to delete per batch (default: `500`)
- `$dryRun` - If true, don't actually delete files (default: `false`)

**Returns:**
Deletion statistics:
```php
[
    'attempted' => 100,
    'deleted' => 98,
    'failed' => 2,
    'bytes_reclaimed' => 1048576,
    'errors' => [
        [
            'path' => '/path/to/file.csv',
            'error' => 'Permission denied',
        ],
    ],
]
```

**Example:**
```php
// Delete files found by scan
$stats = $service->deleteFiles($files, 500, false);

echo "Deleted: {$stats['deleted']} files\n";
echo "Failed: {$stats['failed']} files\n";
echo "Reclaimed: " . $service->formatBytes($stats['bytes_reclaimed']);
```

---

### `getDirectorySize(string $path): int`

Get directory size recursively in bytes.

**Example:**
```php
$size = $service->getDirectorySize(storage_path('app/temp'));
echo "Directory size: " . $service->formatBytes($size);
```

---

### `countFiles(string $path, bool $recursive = true): int`

Count files in directory.

**Example:**
```php
$count = $service->countFiles(storage_path('app/temp'));
echo "Files in temp: {$count}";
```

---

### `ensureDirectoryExists(string $path, int $mode = 0755): bool`

Ensure a directory exists, creating it if necessary.

**Example:**
```php
$service->ensureDirectoryExists(storage_path('app/temp/downloads'));
```

---

### `cleanEmptyDirectories(string $path): int`

Remove empty directories recursively.

**Example:**
```php
$removed = $service->cleanEmptyDirectories(storage_path('app/temp'));
echo "Removed {$removed} empty directories";
```

---

### `formatBytes(int $bytes, int $precision = 2): string`

Format bytes to human-readable format.

**Example:**
```php
echo $service->formatBytes(1048576); // "1.00 MB"
echo $service->formatBytes(1536, 1); // "1.5 KB"
```

---

### `getPathStatistics(string $path): array`

Get comprehensive statistics for a path.

**Returns:**
```php
[
    'exists' => true,
    'readable' => true,
    'writable' => true,
    'size' => 1048576,
    'size_formatted' => '1.00 MB',
    'file_count' => 42,
]
```

**Example:**
```php
$stats = $service->getPathStatistics(storage_path('app/temp'));
print_r($stats);
```

---

## Safety Features

### 1. **Lock File Detection**

The service automatically detects and skips locked files by checking for:

- **Sibling lock files:** `file.csv` → `file.csv.lock`
- **Basename lock files:** `file.csv` → `file.lock`

**Example:**
```php
// These files will be skipped:
// - /temp/data.csv (if /temp/data.csv.lock exists)
// - /temp/report.pdf (if /temp/report.lock exists)

$files = $service->scanDirectory($path, ['skip_locked' => true]);
```

### 2. **Safety Window**

Files modified within the safety window are automatically excluded to prevent deleting files currently being written.

**Example:**
```php
// Skip files modified in the last 10 minutes
$files = $service->scanDirectory($path, [
    'safety_window_minutes' => 10,
]);
```

### 3. **Modification Time Check**

Before deletion, the service verifies that the file hasn't been modified since scanning to prevent race conditions.

### 4. **Exclusion Patterns**

Use glob patterns to protect specific files:

**Example:**
```php
$files = $service->scanDirectory($path, [
    'exclusions' => [
        '*.lock',        // All lock files
        '.gitignore',    // Git ignore file
        '.placeholder',  // Placeholder files
        'important_*',   // Files starting with "important_"
    ],
]);
```

---

## Usage Examples

### Example 1: Clean Old Temporary Files

```php
use App\Services\LocalStorageService;

$service = new LocalStorageService();

// Find files older than 24 hours
$files = $service->scanDirectory(storage_path('app/temp'), [
    'older_than_hours' => 24,
    'safety_window_minutes' => 10,
    'exclusions' => ['*.lock', '.gitignore'],
]);

// Delete in batches of 500
$stats = $service->deleteFiles($files, 500);

Log::info('Temp cleanup completed', [
    'deleted' => $stats['deleted'],
    'bytes_reclaimed' => $stats['bytes_reclaimed'],
]);
```

---

### Example 2: Clean iPaaS CSV Files

```php
$service = new LocalStorageService();

// Find old CSV files from iPaaS
$files = $service->scanDirectory(storage_path('app/temp'), [
    'older_than_hours' => 24,
    'extensions' => ['csv'],
    'skip_locked' => true,
]);

$stats = $service->deleteFiles($files, 500);

echo "Cleaned {$stats['deleted']} CSV files\n";
echo "Reclaimed " . $service->formatBytes($stats['bytes_reclaimed']);
```

---

### Example 3: Dry-Run Preview

```php
$service = new LocalStorageService();

// Preview what would be deleted
$files = $service->scanDirectory(storage_path('app/temp'), [
    'older_than_hours' => 48,
]);

$stats = $service->deleteFiles($files, 500, true); // dry-run = true

echo "Would delete {$stats['deleted']} files\n";
echo "Would reclaim " . $service->formatBytes($stats['bytes_reclaimed']);
```

---

### Example 4: Monitor Directory Health

```php
$service = new LocalStorageService();

$stats = $service->getPathStatistics(storage_path('app/temp'));

if ($stats['size'] > 1073741824) { // 1GB
    Log::warning('Temp directory exceeds 1GB', $stats);
}

echo "Temp directory:\n";
echo "  Size: {$stats['size_formatted']}\n";
echo "  Files: {$stats['file_count']}\n";
echo "  Writable: " . ($stats['writable'] ? 'Yes' : 'No') . "\n";
```

---

### Example 5: Replace Direct File Operations

**Before (scattered File:: calls):**
```php
// Old way - scattered throughout codebase
$tempDir = storage_path('app/temp');
File::ensureDirectoryExists($tempDir);

$files = File::allFiles($tempDir);
foreach ($files as $file) {
    if ($file->getMTime() < time() - (24 * 3600)) {
        File::delete($file->getPathname());
    }
}
```

**After (using LocalStorageService):**
```php
// New way - centralized and testable
$service = new LocalStorageService();
$service->ensureDirectoryExists(storage_path('app/temp'));

$files = $service->scanDirectory(storage_path('app/temp'), [
    'older_than_hours' => 24,
]);

$service->deleteFiles($files);
```

---

## Integration with Existing Code

### Replace in `IpaasHelper.php`

**Before:**
```php
$tempFileName = uniqid('download_', true);
$tempPath = 'temp/' . $tempFileName;
Storage::disk('local')->put($tempPath, '');
$tempFile = Storage::disk('local')->path($tempPath);
```

**After:**
```php
$service = app(LocalStorageService::class);
$tempDir = storage_path('app/temp');
$service->ensureDirectoryExists($tempDir);

$tempFileName = uniqid('download_', true);
$tempFile = $tempDir . '/' . $tempFileName;
file_put_contents($tempFile, '');
```

### Replace in `ApiNode.php`

**Before:**
```php
$this->tempDirectory = storage_path("app/temp");
File::ensureDirectoryExists($this->tempDirectory);
```

**After:**
```php
$service = app(LocalStorageService::class);
$this->tempDirectory = storage_path("app/temp");
$service->ensureDirectoryExists($this->tempDirectory);
```

---

## Testing

### Mocking in Tests

```php
use App\Services\LocalStorageService;
use Mockery;

// Mock the service
$mockService = Mockery::mock(LocalStorageService::class);

// Mock scanDirectory
$mockService->shouldReceive('scanDirectory')
    ->with('/path/to/temp', Mockery::any())
    ->andReturn([
        [
            'path' => '/path/to/temp/file1.csv',
            'name' => 'file1.csv',
            'size' => 1024,
            'modified_at' => now()->subHours(25),
            'extension' => 'csv',
        ],
    ]);

// Mock deleteFiles
$mockService->shouldReceive('deleteFiles')
    ->andReturn([
        'attempted' => 1,
        'deleted' => 1,
        'failed' => 0,
        'bytes_reclaimed' => 1024,
        'errors' => [],
    ]);

// Bind mock to container
app()->instance(LocalStorageService::class, $mockService);
```

---

## Configuration

The service uses sensible defaults but can be configured via method parameters:

```php
$defaultOptions = [
    'recursive' => true,
    'older_than_hours' => null,
    'safety_window_minutes' => 10,
    'extensions' => [],
    'exclusions' => ['*.lock', '.gitignore', '.placeholder'],
    'skip_symlinks' => true,
    'skip_locked' => true,
    'max_files' => null,
];
```

---

## Performance Considerations

1. **Batch Processing**: Deletes files in batches with delays to reduce filesystem pressure
2. **Memory Efficient**: Uses iterators instead of loading all files into memory
3. **Early Returns**: Filters files during scan to avoid processing unnecessary data
4. **Configurable Limits**: `max_files` option prevents scanning huge directories

---

## Error Handling

The service handles errors gracefully:

- **Logs all errors** with context
- **Continues processing** after individual file failures
- **Returns detailed error information** in results
- **Never throws exceptions** to caller (logs instead)

---

## Future Enhancements

Potential additions for Phase B/C:

- [ ] Archive files before deletion
- [ ] Compression of old files
- [ ] Move to cloud storage integration
- [ ] Prometheus metrics export
- [ ] Webhook notifications on cleanup

---

## Related Documentation

- **Command Usage**: `docs/maintenance/temp-cleanup-guide.md`
- **Testing Guide**: `docs/testing/temp-cleanup-tests.md`
- **Configuration**: `config/maintenance.php`

---

**Last Updated:** October 2, 2025  
**Version:** 1.0.0