# Import Queue Configuration

## Overview

All import-related jobs now run on a dedicated 'import' queue for better separation and management. This allows for independent scaling and monitoring of import operations.

## Modified Jobs

### Core Import Jobs
- **ImportJobCoordinator**: Main coordinator job for chunked imports
- **BatchJobProcessor**: Abstract base class for batch processing jobs
- **ImportNetSuiteRecordsBatch**: Individual batch processing jobs
- **ImportNetSuiteRecords**: Legacy monolithic import job

### Configuration Changes
All import jobs are configured to use the import queue via dispatch calls:
```php
// Jobs use Laravel's default Queueable trait without conflicts
class ImportJobCoordinator implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
    // No explicit $queue property needed
}
```

### Dispatch Changes
All dispatch calls explicitly specify the queue to avoid trait conflicts:
```php
ImportJobCoordinator::dispatch($params)->onQueue('import');
ImportNetSuiteRecords::dispatch($params)->onQueue('import');
ImportNetSuiteRecordsBatch::dispatch($params)->onQueue('import');
```

**Important Note**: We use `->onQueue('import')` instead of `public $queue = 'import'` to avoid trait conflicts with Laravel's `Queueable` trait, which already defines a `$queue` property.

## Running Queue Workers

### Start Import Queue Worker
```bash
# Run a dedicated worker for import jobs
php artisan queue:work --queue=import

# Run with specific configuration
php artisan queue:work --queue=import --sleep=3 --tries=3 --max-time=3600

# Run multiple workers for scalability
php artisan queue:work --queue=import --sleep=3 --tries=3 --max-time=3600 &
php artisan queue:work --queue=import --sleep=3 --tries=3 --max-time=3600 &
php artisan queue:work --queue=import --sleep=3 --tries=3 --max-time=3600 &
```

### Priority Queue Processing
If you need to process both import and default queues:
```bash
# Process import queue first, then default
php artisan queue:work --queue=import,default
```

### Production Deployment with Supervisor

Create a supervisor configuration file `/etc/supervisor/conf.d/laravel-import-worker.conf`:

```ini
[program:laravel-import-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /path/to/your/project/artisan queue:work --queue=import --sleep=3 --tries=3 --max-time=3600
directory=/path/to/your/project
autostart=true
autorestart=true
numprocs=3
redirect_stderr=true
stdout_logfile=/path/to/your/project/storage/logs/import-worker.log
stopwaitsecs=3600
```

Then start the workers:
```bash
sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl start laravel-import-worker:*
```

## Laravel Horizon Configuration

### Horizon Dashboard
Horizon provides a beautiful dashboard for monitoring your import queues at `/horizon` (or your configured `HORIZON_PATH`).

### Configuration Summary
The import queue has been configured with:
- **Queue Name**: `import` (consistent across all environments)
- **Wait Threshold**: 300 seconds (5 minutes)
- **Memory Allocation**:
  - Local: 256MB
  - Staging: 512MB
  - Production: 1GB
- **Process Scaling**:
  - Local: 2 processes
  - Staging: 5 processes
  - Production: 8 processes (with auto-scaling)
- **Timeouts**:
  - Local: 10 minutes
  - Staging: 30 minutes
  - Production: 1 hour

### Environment Variables
Add to your `.env` file:
```bash
# Optional: Override local import process count
IMPORT_LOCAL_MAX_PROCESS=2
```

### Horizon Commands
```bash
# Start Horizon (will process import queue automatically)
php artisan horizon

# Restart Horizon after configuration changes
php artisan horizon:terminate
php artisan horizon

# Check Horizon status
php artisan horizon:status
```

## Monitoring

### Horizon Dashboard
Access the Horizon dashboard at: `http://your-app.com/horizon`

Features available:
- Real-time job processing metrics
- Import queue-specific monitoring
- Failed job management
- Job retry capabilities
- Performance metrics and graphs

### Command Line Monitoring
```bash
# Check import queue status
php artisan queue:monitor import

# List all jobs in import queue
php artisan queue:status --queue=import
```

### Failed Jobs
```bash
# Retry failed import jobs
php artisan queue:retry --queue=import

# Clear failed import jobs
php artisan queue:flush --queue=import
```

## Benefits

1. **Isolation**: Import jobs run separately from other application jobs
2. **Scalability**: Can scale import workers independently
3. **Monitoring**: Better visibility into import job performance
4. **Resource Management**: Dedicated resources for import operations
5. **Priority Control**: Can prioritize import jobs or other operations as needed

## Troubleshooting

### If Jobs Don't Process
1. Ensure queue workers are running with the `--queue=import` flag
2. Check that the queue configuration in `config/queue.php` supports your queue driver
3. Verify that jobs are being dispatched with `->onQueue('import')`

### Queue Property Conflicts
If you encounter this error:
```
App\Jobs\ImportNetSuiteRecords and Illuminate\Bus\Queueable define the same property ($queue) in the composition
```

**Solution**: Remove any explicit `public $queue` properties from job classes that use the `Queueable` trait. Use `->onQueue('import')` in dispatch calls instead:

```php
// ❌ WRONG - Causes trait conflict
class ImportJob implements ShouldQueue {
    use Queueable;
    public $queue = 'import'; // Conflicts with Queueable trait
}

// ✅ CORRECT - No conflicts
class ImportJob implements ShouldQueue {
    use Queueable;
    // No explicit $queue property
}

// Dispatch with queue specification
ImportJob::dispatch($data)->onQueue('import');
```

### Performance Tuning
- Adjust `--sleep` parameter based on job frequency
- Increase `numprocs` in supervisor for more parallel workers
- Monitor memory usage and adjust `--max-time` accordingly

## Migration Notes

All existing import functionality remains unchanged. Jobs will now:
- Process on the dedicated 'import' queue
- Have better isolation from other system operations
- Allow for independent scaling and monitoring

No changes are required to existing import workflows or API calls.

### Choosing Between Queue Workers and Horizon

**Use Laravel Horizon if:**
- You're using Redis as your queue driver
- You want a beautiful dashboard for monitoring
- You need advanced features like auto-scaling
- You want detailed metrics and job insights

**Use Basic Queue Workers if:**
- You're using database or other queue drivers
- You prefer simple command-line management
- You have specific supervisor configurations
- You need fine-grained control over worker processes

**Note**: The configuration supports both approaches. Horizon will automatically handle the import queue based on the `config/horizon.php` settings, while manual queue workers can target the import queue specifically with `--queue=import`.