301 lines
8.2 KiB
Markdown
301 lines
8.2 KiB
Markdown
# Queue Package Gap Analysis
|
|
|
|
## Overview
|
|
|
|
This document analyzes the gaps between our Queue package's actual implementation and Laravel's queue functionality, identifying areas that need implementation or documentation updates.
|
|
|
|
> **Related Documentation**
|
|
> - See [Queue Package Specification](queue_package_specification.md) for current implementation
|
|
> - See [Laravel Compatibility Roadmap](laravel_compatibility_roadmap.md) for overall status
|
|
> - See [Foundation Integration Guide](foundation_integration_guide.md) for integration patterns
|
|
> - See [Testing Guide](testing_guide.md) for testing approaches
|
|
> - See [Getting Started Guide](getting_started.md) for development setup
|
|
> - See [Events Package Specification](events_package_specification.md) for event integration
|
|
> - See [Bus Package Specification](bus_package_specification.md) for command bus integration
|
|
|
|
## Implementation Gaps
|
|
|
|
### 1. Missing Laravel Features
|
|
```dart
|
|
// Documented but not implemented:
|
|
|
|
// 1. Job Middleware
|
|
class JobMiddleware {
|
|
// Need to implement:
|
|
Future<void> handle(Job job, Function next);
|
|
Future<void> withoutOverlapping(Job job, Function next);
|
|
Future<void> rateLimit(Job job, Function next, int maxAttempts);
|
|
Future<void> throttle(Job job, Function next, Duration duration);
|
|
}
|
|
|
|
// 2. Job Events
|
|
class JobEvents {
|
|
// Need to implement:
|
|
void beforeJob(Job job);
|
|
void afterJob(Job job);
|
|
void failingJob(Job job, Exception exception);
|
|
void failedJob(Job job, Exception exception);
|
|
void retryingJob(Job job);
|
|
void retriedJob(Job job);
|
|
}
|
|
|
|
// 3. Job Chaining
|
|
class JobChain {
|
|
// Need to implement:
|
|
void chain(List<Job> jobs);
|
|
void onConnection(String connection);
|
|
void onQueue(String queue);
|
|
void catch(Function(Exception) handler);
|
|
void finally(Function handler);
|
|
}
|
|
```
|
|
|
|
### 2. Missing Queue Features
|
|
```dart
|
|
// Need to implement:
|
|
|
|
// 1. Queue Monitoring
|
|
class QueueMonitor {
|
|
// Need to implement:
|
|
Future<Map<String, int>> queueSizes();
|
|
Future<List<Map<String, dynamic>>> failedJobs();
|
|
Future<void> retryFailedJob(String id);
|
|
Future<void> forgetFailedJob(String id);
|
|
Future<void> pruneFailedJobs([Duration? older]);
|
|
}
|
|
|
|
// 2. Queue Rate Limiting
|
|
class QueueRateLimiter {
|
|
// Need to implement:
|
|
Future<bool> tooManyAttempts(String key, int maxAttempts);
|
|
Future<void> hit(String key, Duration decay);
|
|
Future<void> clear(String key);
|
|
Future<int> attempts(String key);
|
|
Future<int> remaining(String key, int maxAttempts);
|
|
}
|
|
|
|
// 3. Queue Batching
|
|
class QueueBatch {
|
|
// Need to implement:
|
|
Future<void> then(Function handler);
|
|
Future<void> catch(Function(Exception) handler);
|
|
Future<void> finally(Function handler);
|
|
Future<void> allowFailures();
|
|
Future<void> onConnection(String connection);
|
|
Future<void> onQueue(String queue);
|
|
}
|
|
```
|
|
|
|
### 3. Missing Worker Features
|
|
```dart
|
|
// Need to implement:
|
|
|
|
// 1. Worker Management
|
|
class WorkerManager {
|
|
// Need to implement:
|
|
Future<void> scale(int processes);
|
|
Future<void> pause();
|
|
Future<void> resume();
|
|
Future<void> restart();
|
|
Future<List<WorkerStatus>> status();
|
|
}
|
|
|
|
// 2. Worker Events
|
|
class WorkerEvents {
|
|
// Need to implement:
|
|
void workerStarting();
|
|
void workerStopping();
|
|
void workerStopped();
|
|
void queueEmpty(String queue);
|
|
void looping();
|
|
}
|
|
|
|
// 3. Worker Options
|
|
class WorkerOptions {
|
|
// Need to implement:
|
|
Duration sleep;
|
|
Duration timeout;
|
|
int maxTries;
|
|
int maxJobs;
|
|
bool force;
|
|
bool stopWhenEmpty;
|
|
bool rest;
|
|
}
|
|
```
|
|
|
|
## Documentation Gaps
|
|
|
|
### 1. Missing API Documentation
|
|
```dart
|
|
// Need to document:
|
|
|
|
/// Handles job middleware.
|
|
///
|
|
/// Example:
|
|
/// ```dart
|
|
/// class RateLimitedJob extends Job with JobMiddleware {
|
|
/// @override
|
|
/// Future<void> middleware(Function next) async {
|
|
/// return await rateLimit(5, Duration(minutes: 1), next);
|
|
/// }
|
|
/// }
|
|
/// ```
|
|
Future<void> middleware(Function next);
|
|
|
|
/// Handles job events.
|
|
///
|
|
/// Example:
|
|
/// ```dart
|
|
/// queue.beforeJob((job) {
|
|
/// print('Processing ${job.id}');
|
|
/// });
|
|
/// ```
|
|
void beforeJob(Function(Job) callback);
|
|
```
|
|
|
|
### 2. Missing Integration Examples
|
|
```dart
|
|
// Need examples for:
|
|
|
|
// 1. Job Chaining
|
|
var chain = queue.chain([
|
|
ProcessPodcast(podcast),
|
|
OptimizeAudio(podcast),
|
|
NotifySubscribers(podcast)
|
|
])
|
|
.onQueue('podcasts')
|
|
.catch((e) => handleError(e))
|
|
.finally(() => cleanup());
|
|
|
|
// 2. Queue Monitoring
|
|
var monitor = QueueMonitor(queue);
|
|
var sizes = await monitor.queueSizes();
|
|
print('Default queue size: ${sizes["default"]}');
|
|
|
|
// 3. Worker Management
|
|
var manager = WorkerManager(queue);
|
|
await manager.scale(4); // Scale to 4 processes
|
|
var status = await manager.status();
|
|
print('Active workers: ${status.length}');
|
|
```
|
|
|
|
### 3. Missing Test Coverage
|
|
```dart
|
|
// Need tests for:
|
|
|
|
void main() {
|
|
group('Job Middleware', () {
|
|
test('applies rate limiting', () async {
|
|
var job = RateLimitedJob();
|
|
var limiter = MockRateLimiter();
|
|
|
|
await job.handle();
|
|
|
|
verify(() => limiter.tooManyAttempts(any, any)).called(1);
|
|
});
|
|
});
|
|
|
|
group('Queue Monitoring', () {
|
|
test('monitors queue sizes', () async {
|
|
var monitor = QueueMonitor(queue);
|
|
var sizes = await monitor.queueSizes();
|
|
|
|
expect(sizes, containsPair('default', greaterThan(0)));
|
|
});
|
|
});
|
|
}
|
|
```
|
|
|
|
## Implementation Priority
|
|
|
|
1. **High Priority**
|
|
- Job middleware (Laravel compatibility)
|
|
- Job events (Laravel compatibility)
|
|
- Queue monitoring
|
|
|
|
2. **Medium Priority**
|
|
- Job chaining
|
|
- Queue rate limiting
|
|
- Worker management
|
|
|
|
3. **Low Priority**
|
|
- Additional worker features
|
|
- Additional monitoring features
|
|
- Performance optimizations
|
|
|
|
## Next Steps
|
|
|
|
1. **Implementation Tasks**
|
|
- Add job middleware
|
|
- Add job events
|
|
- Add queue monitoring
|
|
- Add worker management
|
|
|
|
2. **Documentation Tasks**
|
|
- Document job middleware
|
|
- Document job events
|
|
- Document monitoring
|
|
- Add integration examples
|
|
|
|
3. **Testing Tasks**
|
|
- Add middleware tests
|
|
- Add event tests
|
|
- Add monitoring tests
|
|
- Add worker tests
|
|
|
|
## Development Guidelines
|
|
|
|
### 1. Getting Started
|
|
Before implementing queue features:
|
|
1. Review [Getting Started Guide](getting_started.md)
|
|
2. Check [Laravel Compatibility Roadmap](laravel_compatibility_roadmap.md)
|
|
3. Follow [Testing Guide](testing_guide.md)
|
|
4. Use [Foundation Integration Guide](foundation_integration_guide.md)
|
|
5. Review [Queue Package Specification](queue_package_specification.md)
|
|
6. Review [Events Package Specification](events_package_specification.md)
|
|
7. Review [Bus Package Specification](bus_package_specification.md)
|
|
|
|
### 2. Implementation Process
|
|
For each queue feature:
|
|
1. Write tests following [Testing Guide](testing_guide.md)
|
|
2. Implement following Laravel patterns
|
|
3. Document following [Getting Started Guide](getting_started.md#documentation)
|
|
4. Integrate following [Foundation Integration Guide](foundation_integration_guide.md)
|
|
|
|
### 3. Quality Requirements
|
|
All implementations must:
|
|
1. Pass all tests (see [Testing Guide](testing_guide.md))
|
|
2. Meet Laravel compatibility requirements
|
|
3. Follow integration patterns (see [Foundation Integration Guide](foundation_integration_guide.md))
|
|
4. Match specifications in [Queue Package Specification](queue_package_specification.md)
|
|
|
|
### 4. Integration Considerations
|
|
When implementing queue features:
|
|
1. Follow patterns in [Foundation Integration Guide](foundation_integration_guide.md)
|
|
2. Ensure Laravel compatibility per [Laravel Compatibility Roadmap](laravel_compatibility_roadmap.md)
|
|
3. Use testing approaches from [Testing Guide](testing_guide.md)
|
|
4. Follow development setup in [Getting Started Guide](getting_started.md)
|
|
|
|
### 5. Performance Guidelines
|
|
Queue system must:
|
|
1. Handle high job throughput
|
|
2. Process chains efficiently
|
|
3. Support concurrent workers
|
|
4. Scale horizontally
|
|
5. Meet performance targets in [Laravel Compatibility Roadmap](laravel_compatibility_roadmap.md#performance-benchmarks)
|
|
|
|
### 6. Testing Requirements
|
|
Queue tests must:
|
|
1. Cover all queue operations
|
|
2. Test middleware behavior
|
|
3. Verify event handling
|
|
4. Check worker management
|
|
5. Follow patterns in [Testing Guide](testing_guide.md)
|
|
|
|
### 7. Documentation Requirements
|
|
Queue documentation must:
|
|
1. Explain queue patterns
|
|
2. Show middleware examples
|
|
3. Cover error handling
|
|
4. Include performance tips
|
|
5. Follow standards in [Getting Started Guide](getting_started.md#documentation)
|