Log Rotation and Disk Space Management on Linux

By Khimananda Oli | Last reviewed: August 2026

Uncontrolled log growth is one of the most common causes of avoidable production outages I see across both global and Nepali infrastructure teams. Effective log rotation and disk space management on Linux prevents "No space left on device" errors that crash databases and web servers at peak traffic. This guide provides the exact configuration patterns, cleanup routines, and monitoring hooks you need to keep systems stable without losing critical audit data.

How Do You Configure Logrotate for Production Applications?

The logrotate utility is the standard mechanism for log rotation and disk space management on Linux. While default OS logs are handled automatically, application logs from Nginx, Laravel, Docker, or custom services require explicit configuration. A common mistake is relying solely on time-based rotation; in high-traffic environments, size-based triggers are essential to prevent partition overflow between scheduled runs.

Create an Application-Specific Rotation Policy

Place custom configurations in /etc/logrotate.d/ rather than editing the main logrotate.conf. This keeps your rules modular and version-controllable — a practice I enforce when setting up Laravel deployments on Ubuntu VPS.

# /etc/logrotate.d/myapp
/var/log/myapp/*.log {
    daily
    rotate 30
    size 100M
    missingok
    notifempty
    compress
    delaycompress
    create 0640 www-data adm
    sharedscripts
    postrotate
        systemctl reload myapp > /dev/null 2>&1 || true
    endscript
}

• size + daily: Rotates if either threshold is met first, covering both burst traffic and quiet periods.
• delaycompress: Keeps the most recent rotated file uncompressed so debugging tools can read it without decompression.
• create: Ensures the new log file has correct ownership immediately; without this, applications may fail to write after rotation.
• postrotate: Signals the application to reopen file handles. Omitting this is the #1 cause of "logs not rotating" complaints.

Validate Configuration Before Deploying

Always test with debug mode to catch syntax errors or permission issues without modifying actual files:

sudo logrotate -d /etc/logrotate.d/myapp

Force a manual rotation during maintenance windows to verify the full cycle works under real conditions:

sudo logrotate -f /etc/logrotate.d/myapp

What Commands Diagnose Disk Space Issues Accurately?

When alerts fire, you need precise diagnostics fast. df -h shows filesystem-level usage but hides inode exhaustion and per-directory bloat. Pair it with these targeted commands for complete visibility during incident response.

# Find directories consuming most space under /var/log
sudo du -ahx /var/log | sort -rh | head -20

# Check inode usage (critical for mail queues, cache dirs)
df -i /var

# Identify deleted-but-open files holding space
sudo lsof +L1 | grep deleted

# List files larger than 500MB modified in last 7 days
find /var/log -type f -size +500M -mtime -7 -exec ls -lh {} \;

The lsof +L1 command deserves special attention. When a process deletes a log file but keeps it open, the space remains allocated until the process closes the handle or restarts. This phantom usage confuses many engineers who see du reporting low usage while df shows 98% full. Truncating the file descriptor (: > /proc/PID/fd/FD) reclaims space without restarting the service.

Automate Diagnostics with Cron

Schedule lightweight checks that report anomalies before they become emergencies. I use this pattern on every managed server:

# /etc/cron.d/disk-health-check
*/15 * * * * root /usr/local/bin/check-disk.sh | logger -t disk-monitor

Pair automated checks with structured observability. If you're running Prometheus and Grafana for monitoring, expose disk metrics via node_exporter and set alerts at 80% warning / 90% critical thresholds.

How Do You Balance Retention Requirements Against Storage Costs?

Retention isn't purely technical — it's governed by compliance, forensics, and cost. SOC 2 and ISO 27001 audits typically require 12-month log availability, but keeping everything on primary storage is wasteful. Tier your approach based on access patterns.

Implement Automated Offloading

Use a simple script triggered by postrotate or a dedicated cron job to push compressed logs to object storage. Verify uploads before deleting local copies to prevent data loss:

#!/bin/bash
# /usr/local/bin/offload-logs.sh
BUCKET="s3://company-logs-archive"
LOCAL_DIR="/var/log/myapp/archive"

aws s3 sync "$LOCAL_DIR" "$BUCKET" --only-show-errors
if [ $? -eq 0 ]; then
    find "$LOCAL_DIR" -name "*.gz" -mtime +7 -delete
    echo "$(date): Offload complete" >> /var/log/offload.log
else
    echo "$(date): OFFLOAD FAILED - local files retained" >> /var/log/offload.log
    exit 1
fi

This tiered model directly supports cloud cost optimization by moving bulk storage to cheaper tiers while keeping recent data locally accessible.

When Should You Use Journald vs Traditional Log Files?

Modern systemd-based distributions default to journald, which offers structured metadata and binary indexing. However, traditional plaintext files remain preferable for many production workloads due to tooling compatibility and predictable rotation behavior.

In practice, I configure journald for system services but route application output to dedicated files. Edit /etc/systemd/journald.conf to enforce sane defaults:

[Journal]
SystemMaxUse=2G
SystemKeepFree=4G
MaxRetentionSec=30day
ForwardToSyslog=no

For applications where plaintext logs are mandatory (e.g., PCI-DSS environments requiring direct file integrity monitoring), disable journald forwarding entirely and rely on logrotate as the single source of truth for log rotation and disk space management on Linux.

How Do You Prevent Common Rotation Failures Silently?

Configuration drift and silent failures undermine even well-designed rotation policies. These three checks should be part of your baseline hardening, similar to the steps covered in initial Ubuntu server setup.

1. Verify cron execution: Confirm /etc/cron.daily/logrotate exists and is executable. On minimal containers, cron may not be installed — add it explicitly or use systemd timers.
2. Monitor rotation success: Parse /var/lib/logrotate/status to detect stale entries. Alert if any configured log hasn't rotated within expected intervals.
3. Test permissions post-rotation: Include an assertion in your deployment pipeline that validates new log files have correct ownership after forced rotation. Permission bugs often surface only after the first production rotation.

Conclusion

Reliable log rotation and disk space management on Linux combines disciplined configuration, tiered retention, and proactive monitoring. Start by auditing your current /etc/logrotate.d/ policies against the patterns above, implement size-based triggers alongside time schedules, and establish automated offloading to object storage within 30 days. Document your retention tiers explicitly — auditors and future engineers will thank you. If your team needs help designing compliant, cost-efficient logging infrastructure, reach out to discuss your specific requirements.