platform/doc/queue_gap_analysis.md

Queue Package Gap Analysis

Overview

This document analyzes the gaps between our Queue package's actual implementation and Laravel's queue functionality, identifying areas that need implementation or documentation updates.

Related Documentation

• See Queue Package Specification for current implementation
• See Laravel Compatibility Roadmap for overall status
• See Foundation Integration Guide for integration patterns
• See Testing Guide for testing approaches
• See Getting Started Guide for development setup
• See Events Package Specification for event integration
• See Bus Package Specification for command bus integration

Implementation Gaps

1. Missing Laravel Features

// Documented but not implemented:

// 1. Job Middleware
class JobMiddleware {
  // Need to implement:
  Future<void> handle(Job job, Function next);
  Future<void> withoutOverlapping(Job job, Function next);
  Future<void> rateLimit(Job job, Function next, int maxAttempts);
  Future<void> throttle(Job job, Function next, Duration duration);
}

// 2. Job Events
class JobEvents {
  // Need to implement:
  void beforeJob(Job job);
  void afterJob(Job job);
  void failingJob(Job job, Exception exception);
  void failedJob(Job job, Exception exception);
  void retryingJob(Job job);
  void retriedJob(Job job);
}

// 3. Job Chaining
class JobChain {
  // Need to implement:
  void chain(List<Job> jobs);
  void onConnection(String connection);
  void onQueue(String queue);
  void catch(Function(Exception) handler);
  void finally(Function handler);
}

2. Missing Queue Features

// Need to implement:

// 1. Queue Monitoring
class QueueMonitor {
  // Need to implement:
  Future<Map<String, int>> queueSizes();
  Future<List<Map<String, dynamic>>> failedJobs();
  Future<void> retryFailedJob(String id);
  Future<void> forgetFailedJob(String id);
  Future<void> pruneFailedJobs([Duration? older]);
}

// 2. Queue Rate Limiting
class QueueRateLimiter {
  // Need to implement:
  Future<bool> tooManyAttempts(String key, int maxAttempts);
  Future<void> hit(String key, Duration decay);
  Future<void> clear(String key);
  Future<int> attempts(String key);
  Future<int> remaining(String key, int maxAttempts);
}

// 3. Queue Batching
class QueueBatch {
  // Need to implement:
  Future<void> then(Function handler);
  Future<void> catch(Function(Exception) handler);
  Future<void> finally(Function handler);
  Future<void> allowFailures();
  Future<void> onConnection(String connection);
  Future<void> onQueue(String queue);
}

3. Missing Worker Features

// Need to implement:

// 1. Worker Management
class WorkerManager {
  // Need to implement:
  Future<void> scale(int processes);
  Future<void> pause();
  Future<void> resume();
  Future<void> restart();
  Future<List<WorkerStatus>> status();
}

// 2. Worker Events
class WorkerEvents {
  // Need to implement:
  void workerStarting();
  void workerStopping();
  void workerStopped();
  void queueEmpty(String queue);
  void looping();
}

// 3. Worker Options
class WorkerOptions {
  // Need to implement:
  Duration sleep;
  Duration timeout;
  int maxTries;
  int maxJobs;
  bool force;
  bool stopWhenEmpty;
  bool rest;
}

Documentation Gaps

1. Missing API Documentation

// Need to document:

/// Handles job middleware.
/// 
/// Example:
/// ```dart
/// class RateLimitedJob extends Job with JobMiddleware {
///   @override
///   Future<void> middleware(Function next) async {
///     return await rateLimit(5, Duration(minutes: 1), next);
///   }
/// }
/// ```
Future<void> middleware(Function next);

/// Handles job events.
///
/// Example:
/// ```dart
/// queue.beforeJob((job) {
///   print('Processing ${job.id}');
/// });
/// ```
void beforeJob(Function(Job) callback);

2. Missing Integration Examples

// Need examples for:

// 1. Job Chaining
var chain = queue.chain([
  ProcessPodcast(podcast),
  OptimizeAudio(podcast),
  NotifySubscribers(podcast)
])
.onQueue('podcasts')
.catch((e) => handleError(e))
.finally(() => cleanup());

// 2. Queue Monitoring
var monitor = QueueMonitor(queue);
var sizes = await monitor.queueSizes();
print('Default queue size: ${sizes["default"]}');

// 3. Worker Management
var manager = WorkerManager(queue);
await manager.scale(4); // Scale to 4 processes
var status = await manager.status();
print('Active workers: ${status.length}');

3. Missing Test Coverage

// Need tests for:

void main() {
  group('Job Middleware', () {
    test('applies rate limiting', () async {
      var job = RateLimitedJob();
      var limiter = MockRateLimiter();
      
      await job.handle();
      
      verify(() => limiter.tooManyAttempts(any, any)).called(1);
    });
  });
  
  group('Queue Monitoring', () {
    test('monitors queue sizes', () async {
      var monitor = QueueMonitor(queue);
      var sizes = await monitor.queueSizes();
      
      expect(sizes, containsPair('default', greaterThan(0)));
    });
  });
}

Implementation Priority

1. High Priority

   • Job middleware (Laravel compatibility)
   • Job events (Laravel compatibility)
   • Queue monitoring

2. Medium Priority

   • Job chaining
   • Queue rate limiting
   • Worker management

3. Low Priority

   • Additional worker features
   • Additional monitoring features
   • Performance optimizations

Next Steps

1. Implementation Tasks

   • Add job middleware
   • Add job events
   • Add queue monitoring
   • Add worker management

2. Documentation Tasks

   • Document job middleware
   • Document job events
   • Document monitoring
   • Add integration examples

3. Testing Tasks

   • Add middleware tests
   • Add event tests
   • Add monitoring tests
   • Add worker tests

Development Guidelines

1. Getting Started

Before implementing queue features:

1. Review Getting Started Guide
2. Check Laravel Compatibility Roadmap
3. Follow Testing Guide
4. Use Foundation Integration Guide
5. Review Queue Package Specification
6. Review Events Package Specification
7. Review Bus Package Specification

2. Implementation Process

For each queue feature:

1. Write tests following Testing Guide
2. Implement following Laravel patterns
3. Document following Getting Started Guide
4. Integrate following Foundation Integration Guide

3. Quality Requirements

All implementations must:

1. Pass all tests (see Testing Guide)
2. Meet Laravel compatibility requirements
3. Follow integration patterns (see Foundation Integration Guide)
4. Match specifications in Queue Package Specification

4. Integration Considerations

When implementing queue features:

1. Follow patterns in Foundation Integration Guide
2. Ensure Laravel compatibility per Laravel Compatibility Roadmap
3. Use testing approaches from Testing Guide
4. Follow development setup in Getting Started Guide

5. Performance Guidelines

Queue system must:

1. Handle high job throughput
2. Process chains efficiently
3. Support concurrent workers
4. Scale horizontally
5. Meet performance targets in Laravel Compatibility Roadmap

6. Testing Requirements

Queue tests must:

1. Cover all queue operations
2. Test middleware behavior
3. Verify event handling
4. Check worker management
5. Follow patterns in Testing Guide

7. Documentation Requirements

Queue documentation must:

1. Explain queue patterns
2. Show middleware examples
3. Cover error handling
4. Include performance tips
5. Follow standards in Getting Started Guide