laravel-quarantine maintained by vortech
Why
When Stripe, your mail provider or your CRM goes down, every job that talks to it starts failing at once, and Laravel dutifully retries each one. The real problem is not the job: the dependency is unavailable.
Quarantine groups failures by dependency. After a threshold is reached, jobs that depend on it are released back to the queue instead of retrying (and instead of landing in failed_jobs). After a cooldown, a single probe job checks whether the service is back, and the queue resumes by itself.
Stripe starts returning 503
↓
5 failures in 60 seconds
↓
dependency quarantined
↓
affected jobs delayed
↓
5 minute cooldown
↓
one probe job
↓
Stripe recovered
↓
queue resumes automatically
It is cache-backed, needs no dashboard and works with any queue driver.
Requirements
- PHP 8.4+
- Laravel 13
Installation
composer require vortech/laravel-quarantine
The service provider is registered automatically. Run the install command to publish the config file:
php artisan quarantine:install
Use --force to overwrite an existing config file. php artisan vendor:publish --tag=quarantine-config does the same.
Usage
Add the middleware to every job that talks to the dependency:
use Vortech\Quarantine\Middleware\Quarantine;
final class ChargeCustomer implements ShouldQueue
{
public function middleware(): array
{
return [
new Quarantine('stripe'),
];
}
public function handle(): void
{
// Charge the customer through Stripe.
}
}
stripe is the dependency identifier. Every job using it contributes to, and is protected by, the same state.
Attribute
use Vortech\Quarantine\Attributes\DependsOn;
use Vortech\Quarantine\Middleware\Quarantine;
#[DependsOn('stripe')]
#[DependsOn('mailgun')]
final class SendReceipt implements ShouldQueue
{
public function middleware(): array
{
return Quarantine::for($this);
}
}
Laravel only calls middleware(), so the attribute is resolved into the same middleware through Quarantine::for($this). Merge it into your own list when the job needs other middleware.
With several dependencies the job is blocked as soon as one of them is, and a dependency failure is recorded against all of them.
How it works
HEALTHY ──failure──► DEGRADED ──threshold reached──► QUARANTINED
▲ │
│ cooldown elapsed
│ ▼
└──────────────── success ◄──────────────────────── PROBING
│
failure ──► QUARANTINED
- Healthy / degraded. Jobs run. Dependency failures are counted in a moving window; there is no state change until the threshold is reached.
- Quarantined. Jobs are released with
release_afterseconds of delay (plus jitter) and their handler does not run. - Probing. Once the cooldown has elapsed,
probe_jobsjobs (default 1) are let through. A success recovers the dependency and clears its failures, a dependency failure quarantines it again for another cooldown. Every other job stays released.
Only failures the detector attributes to the dependency count. An application error (a validation exception, a 422) never affects the state, and a probe that ends that way simply frees its slot. A probe that never reports back, because the worker died, frees its slot after probe_timeout seconds.
A blocked job that cannot be released (the sync driver, for example) throws DependencyUnavailable.
Failure detection
The default HttpFailureDetector counts connection errors and the statuses in quarantine.http_statuses (429, 500, 502, 503, 504), for both the Laravel HTTP client and Guzzle, including wrapped exceptions. A 400 or 422 is never counted.
Write your own detector for anything else:
use Vortech\Quarantine\Contracts\FailureDetector;
final class StripeFailureDetector implements FailureDetector
{
public function causedByDependency(Throwable $exception): bool
{
return $exception instanceof StripeUnavailable;
}
}
and reference it by class name, which keeps the middleware safe to serialize:
new Quarantine('stripe', detector: StripeFailureDetector::class);
or per dependency in config/quarantine.php ('detector' => StripeFailureDetector::class), or in a service provider.
Registering dependencies
use Vortech\Quarantine\Facades\Quarantine;
public function boot(): void
{
Quarantine::dependency('stripe')
->threshold(5)
->within(seconds: 60)
->cooldown(minutes: 5)
->releaseAfter(seconds: 60)
->probeJobs(1)
->detectUsing(StripeFailureDetector::class);
// Closures are fine here: definitions are never serialized with a job.
Quarantine::dependency('crm')
->when(fn (Throwable $e): bool => $e instanceof CrmTimeout);
}
Fluent calls override the config file, which stays fully supported.
Configuration
| Key | Default | Description |
|---|---|---|
store |
null (QUARANTINE_STORE) |
Cache store holding the state. null is the default store. |
prefix |
quarantine |
Cache key prefix. |
strict |
false |
Throw UnknownDependency for dependencies that are neither configured nor registered. |
detector |
HttpFailureDetector |
Default failure detector. |
http_statuses |
429, 500, 502, 503, 504 |
Statuses HttpFailureDetector counts. |
default.threshold |
5 |
Failures inside the window that trigger quarantine. |
default.window |
60 |
Moving window, in seconds. |
default.cooldown |
300 |
Seconds in quarantine before probing. |
default.release_after |
60 |
Delay for blocked jobs, in seconds. |
default.release_jitter |
15 |
± seconds of random jitter on the delay. |
default.probe_jobs |
1 |
Jobs allowed through while probing. |
default.probe_timeout |
120 |
Seconds after which an unfinished probe slot is freed. |
dependencies.{name} |
[] |
Per-dependency overrides of the default keys (and detector). |
State lives in the cache under quarantine:{dependency}:state, :failures, :last-failure, :last-success, :quarantined-at, :probe-at and :probe-lock.
Production notes
- Use an atomic cache store. Every state transition runs under a cache lock (
quarantine:{dependency}:transition), so a hundred workers failing together record each failure once and quarantine the dependency exactly once. Redis is recommended:QUARANTINE_STORE=redis. A store without lock support still works, but transitions are then no longer atomic. - Multi-server. State is never held in process memory; all workers and servers share the cache store.
- Octane. The manager holds only definitions registered at boot, so it works under Octane, RoadRunner, Swoole and FrankenPHP.
- Horizon. Nothing to configure; Quarantine is plain job middleware.
- Retries. A released job uses up an attempt in Laravel's
triescounter, so give jobs a generoustriesor useretryUntil()rather than a small fixed number.
Runtime API
use Vortech\Quarantine\Facades\Quarantine;
Quarantine::isHealthy('stripe');
Quarantine::isDegraded('stripe');
Quarantine::isQuarantined('stripe');
Quarantine::isProbing('stripe');
$status = Quarantine::status('stripe'); // DependencyStatus
Quarantine::quarantine('stripe'); // block until released
Quarantine::quarantine('stripe', 600); // block for 10 minutes, then probe
Quarantine::release('stripe'); // back to healthy
Quarantine::reset('stripe'); // forget everything
DependencyStatus exposes name, state, failures, lastFailureAt, lastSuccessAt, quarantinedAt, nextProbeAt and lastError.
Artisan commands
php artisan quarantine:install [--force]
php artisan quarantine:list
php artisan quarantine:status stripe
php artisan quarantine:release stripe
php artisan quarantine:block stripe --for=10m # 90, 30s, 10m, 2h, 1d; omit to block until released
php artisan quarantine:reset stripe
php artisan quarantine:reset --all
Dependency Status Failures Since
stripe QUARANTINED 18 22:41
mailgun HEALTHY 0 -
crm DEGRADED 2 22:51
Events
| Event | When |
|---|---|
DependencyDegraded |
The first failure inside the window. |
DependencyQuarantined |
The threshold was reached. |
DependencyProbeStarted |
The cooldown elapsed and the first probe was let through. |
DependencyRecovered |
A probe succeeded. |
DependencyProbeFailed |
A probe failed. |
DependencyManuallyQuarantined |
Quarantine::quarantine() or quarantine:block. |
DependencyManuallyReleased |
Quarantine::release() or quarantine:release. |
Notifications are intentionally left out. Listen to the events:
Event::listen(DependencyQuarantined::class, SendQuarantineSlackNotification::class);
Testing
composer test
composer analyse
composer format
The suite uses Pest with Orchestra Testbench and requires PHP 8.4+.
Changelog
See CHANGELOG for what has changed recently.
Security
If you discover a security issue, please email mate@vortech.hu instead of using the issue tracker.
Credits
- Mate Papp, Developer @ Vortech
License
The MIT License (MIT). See the License File for more information.