Laravel Queues and Jobs: Background Processing Explained

Khimananda Oli 7 min read DevOps
Laravel Queues and Jobs: Background Processing Explained

By Khimananda Oli | Last reviewed: August 2026

Synchronous request cycles are the primary bottleneck preventing PHP applications from scaling effectively under load. When your application sends emails, generates PDFs, or processes webhooks during a user request, latency increases and timeout risks multiply. Laravel Queues and Jobs: Background Processing Explained addresses this by decoupling heavy tasks from the HTTP lifecycle, allowing your web servers to respond instantly while workers handle intensive operations asynchronously. For teams deploying on infrastructure like Ubuntu VPS with Nginx, mastering this pattern is non-negotiable for maintaining sub-second response times.

Web Server(Nginx + PHP-FPM)Queue Driver(Redis / SQS / DB)Queue Worker(php artisan queue:work)Dispatch JobFetch & Execute
High-level architecture of Laravel Queues and Jobs background processing in a production environment

How do you configure Laravel Queues and Jobs for background processing?

Setting up background processing requires three distinct components: a Job class defining the task, a queue connection configured in config/queue.php, and a persistent worker process. Never use the sync driver in production; it executes jobs immediately within the request cycle, negating every benefit of asynchronous processing.

Create and Dispatch a Job

Generate a job using Artisan. This creates a class implementing the ShouldQueue interface, which signals Laravel to push it to the configured driver instead of executing it inline.

php artisan make:job ProcessUserReport

Inside the generated job, inject dependencies via the constructor and place logic in the handle() method. Laravel’s service container automatically resolves dependencies when the worker executes the job.

<?php

namespace App\Jobs;

use App\Models\User;
use App\Services\ReportGenerator;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;

class ProcessUserReport implements ShouldQueue
{
    use Queueable;

    public function __construct(
        private User $user,
        private string $reportType
    ) {}

    public function handle(ReportGenerator $generator): void
    {
        $generator->generate($this->user, $this->reportType);
    }
}

Dispatch the job from a controller, event listener, or command. The HTTP response returns immediately while the report generation happens asynchronously.

ProcessUserReport::dispatch($user, 'monthly');

Select and Configure a Queue Driver

Your driver choice dictates reliability, throughput, and operational complexity. Set the default connection in .env:

QUEUE_CONNECTION=redis

For Redis, ensure the connection block in config/queue.php specifies retry limits and timeouts appropriate for your workload. In production environments I manage, we always set explicit retry_after values higher than the maximum expected job duration to prevent duplicate executions.

Which Laravel queue driver should you choose for production workloads?

Driver selection depends on traffic volume, infrastructure constraints, and compliance requirements. Each option carries trade-offs between simplicity, performance, and operational overhead.

DriverBest ForThroughputReliabilityOperational Complexity
RedisMost production apps, real-time workloadsHigh (10k+ jobs/min)Excellent (atomic operations)Moderate (requires Redis instance)
SQSAWS-native apps, massive scaleVirtually unlimitedExcellent (managed service)Low (fully managed)
DatabaseLow-volume apps, simple deploymentsLow (<100 jobs/min)Poor (table locks, race conditions)Very Low
BeanstalkdLegacy systems, dedicated queue needsMedium-HighGoodModerate (single-purpose daemon)

In my experience supporting teams across Nepal and globally, Redis serves as the default recommendation for 90% of Laravel applications. It provides atomic operations, supports multiple queues with priority, and integrates natively with Laravel Horizon for monitoring. If you’re already running on AWS infrastructure as described in hosting Laravel on AWS EC2, SQS eliminates Redis management overhead entirely and scales without intervention.

Avoid the database driver beyond prototyping. Under concurrent load, row-level locking causes worker contention and silent failures. I’ve audited systems where database queues caused cascading slowdowns during peak traffic because workers spent more time waiting for locks than processing jobs.

Worker StartsFetch JobExecute handle()AcknowledgeFailed?Retry / FailException thrownSuccess path
Queue worker lifecycle: job fetch, execution, acknowledgment, and failure retry flow in Laravel background processing

How do you handle failed jobs and retries reliably?

Failures are inevitable. Network timeouts, third-party API outages, and transient database errors will occur. Your queue configuration must account for this explicitly rather than hoping jobs succeed on the first attempt.

Configure Retry Limits and Backoff

Define retry behavior directly on the job class. Exponential backoff prevents overwhelming recovering services:

public int $tries = 5;

public function backoff(): array
{
    return [10, 60, 300, 900, 3600];
}

This retries after 10 seconds, 1 minute, 5 minutes, 15 minutes, and 1 hour. After five failures, Laravel moves the job to the failed_jobs table.

Set Up Failed Job Storage

Always create the failed jobs table. Without it, failed jobs vanish silently with no audit trail — a critical gap for SOC 2 or ISO 27001 compliance where you must demonstrate error handling and recovery procedures.

php artisan queue:failed-table
php artisan migrate

Monitor failed jobs proactively. In production, I configure alerts when the failed job count exceeds a threshold within a rolling window. Tools like Laravel Horizon provide dashboards for this, but even basic CloudWatch or Prometheus metrics on the failed_jobs table catch issues before users report them. See monitoring with Prometheus and Grafana for setting up observable queue health.

Handle Poison Pills

Some jobs fail permanently due to bad data or logic errors. Retrying these wastes resources. Implement circuit-breaker logic:

public function failed(\Throwable $exception): void
{
    if ($exception instanceof InvalidReportDataException) {
        // Log and notify; do not retry
        logger()->error('Permanent failure', ['job' => $this->job->getJobId()]);
        return;
    }
    // Allow normal retry behavior for transient errors
}

What are the best practices for deploying Laravel queue workers?

Workers are long-lived processes. Deploying them incorrectly causes memory leaks, stale code execution, and duplicate job processing. Treat workers as stateful services, not stateless request handlers.

  • Use Supervisor or systemd: Workers must auto-restart on crash. Never run queue:work manually in production.
  • Deploy with zero-downtime restarts: Use php artisan queue:restart after deployments. Workers finish their current job then gracefully exit, allowing Supervisor to spawn fresh processes with updated code.
  • Set memory limits: Add --memory=256 to prevent unbounded growth. PHP’s garbage collector doesn’t always reclaim memory in long-running processes.
  • Separate queues by priority: Critical tasks (password resets, payment confirmations) should never wait behind bulk operations (report generation, data imports). Configure multiple queues and assign workers accordingly.
  • Match concurrency to resources: A 2GB VPS typically handles 2–4 workers safely. Over-provisioning causes OOM kills; under-provisioning leaves jobs queued unnecessarily.

For teams using containerized deployments, each worker should run in its own container replica. This isolates failures and allows independent scaling. Refer to containerizing Laravel with Docker for proper worker containerization patterns.

Synchronous RequestResponse: 3.2s | Timeout risk: HIGHWith Queue WorkersResponse: 120ms | Timeout risk: LOWBackground WorkerProcesses async (no user wait)vs
Synchronous vs Laravel Queues and Jobs background processing: response time and resource utilization comparison

Conclusion

Laravel Queues and Jobs transform background processing from an afterthought into a foundational architectural pattern. Start with Redis as your driver, implement explicit retry and failure handling, deploy workers with proper process management, and monitor queue health as rigorously as you monitor HTTP endpoints. The difference between an app that degrades under load and one that scales gracefully almost always comes down to disciplined queue implementation. If your team needs help designing compliant, observable queue infrastructure or auditing existing setups for production readiness, reach out to discuss your architecture.

Frequently Asked Questions

Queues offload time-consuming tasks like email sending or image resizing from the HTTP request cycle. This keeps user response times fast and prevents timeouts during heavy operations in your Laravel application.

Redis is the standard recommendation for production due to its speed, atomic operations, and support for advanced features like job tagging and rate limiting. Database drivers are acceptable only for low-traffic apps or development environments where infrastructure complexity must be minimized.

Set the tries property on your job class or pass --tries to the queue worker command. Laravel 12 defaults to one attempt, so explicitly defining three to five retries with exponential backoff prevents infinite loops on transient failures.

Check if the queue worker daemon is running via php artisan queue:work. Jobs sit in storage until a worker consumes them. Also verify your QUEUE_CONNECTION environment variable matches the intended driver configuration in config/queue.php.

Yes, use separate queues by passing --queue=high,default,low to the worker command. High-priority jobs process first when available. You can also assign priority dynamically within the job constructor using the onQueue method for granular control.

Implement rate limiting using Laravel's built-in RateLimiter facade within the job's middleware method. Configure global or per-key limits in a service provider to throttle outgoing requests and prevent third-party bans during bulk background processing tasks.

Closures cannot be serialized safely for queue storage. Always dispatch dedicated job classes instead of anonymous functions. If you must pass dynamic logic, use invokable classes or store parameters as public properties that serialize correctly across worker processes.

Run php artisan queue:failed to list all failed jobs with timestamps and exception messages. Use queue:retry to reprocess specific IDs after fixing underlying issues. Regularly prune old failures with queue:flush to keep the failed_jobs table manageable.

Asynchronous dispatch prevents blocking the web request during long transactions. Synchronous execution suits admin panels or CLI commands where immediate feedback matters. For user-facing endpoints, always queue database writes exceeding two hundred milliseconds to maintain responsiveness.

Never pass raw secrets or tokens as job properties since they persist in queue storage. Retrieve credentials inside the handle method using encrypted config values or vault integrations. Encrypt any PII before serialization and decrypt only during execution.

Graceful restarts with php artisan queue:restart signal workers to finish current jobs before exiting. New deployments pick up pending work automatically. Without this signal, workers may run stale code indefinitely, causing inconsistencies between deployed versions and executing job logic.

Use Queue::fake() in tests to assert jobs were dispatched without execution. Mock external dependencies injected via constructor or method binding. Test the handle method separately with fake implementations to validate business logic independently of queue transport mechanics.

Horizon provides essential visibility into throughput, wait times, and failure rates for Redis-backed queues. Small projects can rely on basic logging and monitoring, but teams scaling beyond ten workers need Horizon’s dashboard and auto-scaling configuration to maintain reliability.

Set --max-jobs=1000 to recycle workers periodically and prevent memory leaks. Enable OPcache preloading and avoid loading unnecessary service providers in console context. Monitor RSS usage and adjust batch sizes to keep individual job memory footprints under fifty megabytes.

Use Bus::chain() to execute jobs sequentially where each depends on the previous result. For parallel dependencies, dispatch multiple jobs then use a final aggregation job triggered by events or database flags when all prerequisites complete successfully.