Application logs are essential when something breaks, but unmanaged log files can quietly consume disk space, slow backups, and make incidents harder to investigate. A scheduled web cron job can keep logs useful by rotating active files, compressing archives, and removing data only after a clear retention period.
What automated log rotation should accomplish
A dependable rotation workflow has four jobs: preserve the current log, start a fresh writable file, compress older archives, and enforce retention limits. The process should be safe to repeat and should never delete the only useful copy of recent events.
Choose a retention policy before writing code
Define how long each class of log must remain available. Security and audit records may need a different policy from debug output. A practical policy can combine age, such as 30 days, with a total storage ceiling, such as 5 GB. Document exceptions before automation begins.
Expose a narrow maintenance endpoint
Create a server-side endpoint dedicated to log maintenance. It should accept no arbitrary paths from the request. Keep an allowlist of known log directories, verify every resolved path stays inside those directories, and return a concise machine-readable result.
Protect the endpoint with the same principles described in secure web cron endpoints: HTTPS, a secret sent outside the URL where possible, restricted methods, and rate limits. Never include credentials in response output.
Prevent overlapping rotation runs
Two rotation processes touching the same file can produce partial archives or missing lines. Acquire a lock before scanning files, record when the lock was created, and release it in a finally block. Use a bounded stale-lock recovery rule rather than deleting locks blindly.
The patterns in preventing overlapping cron jobs apply directly to log maintenance.
Rotate the active file safely
Prefer an atomic rename on the same filesystem. Move the current file to a timestamped archive name, then create a new active file with the correct owner and permissions. If the application keeps an open file handle, signal it to reopen the log or use a logging library that supports rotation.
Use predictable archive names
A name such as app-2026-08-09T064500Z.log sorts chronologically and avoids locale ambiguity. Include the environment or service name when several applications share one archive directory. Never use untrusted request values in filenames.
Compress completed archives
Compress only files that are no longer being written. Gzip is widely supported and usually effective for text logs. Write to a temporary destination, verify that the compressed file can be opened, and replace the source only after validation succeeds.
Enforce age and storage limits separately
First remove archives older than the approved retention period. Then calculate the remaining directory size and, if necessary, remove the oldest eligible archives until the storage ceiling is met. Protect recent files and any archive held for an investigation.
Keep deletion auditable
Every run should report the files rotated, compressed, skipped, and deleted, along with total bytes reclaimed. Send counts and identifiers to a separate monitoring channel so the cleanup process does not depend on the files it is managing.
Choose a schedule based on volume
High-volume applications may rotate hourly; smaller sites may need one daily run. Schedule often enough that the active file remains manageable, but avoid unnecessary work. If file size is the primary trigger, let the endpoint exit cleanly when no file exceeds the threshold.
Use a standard schedule from these cron expression examples, and confirm the server time zone before relying on a daily boundary.
Make retries safe
A retry must recognize archives already created by the first attempt. Store progress per file, use deterministic names, and treat an existing verified archive as success. Apply bounded backoff for transient storage failures using the guidance in cron job retry strategies.
Monitor outcomes, not just HTTP status
A 200 response does not prove that logs were rotated. Return structured counts, duration, reclaimed bytes, and warning details. Alert when the job has not succeeded within its expected window, when free disk space keeps falling, or when an active file remains above the threshold.
Build these signals into your wider cron job monitoring setup.
Test with disposable files first
Create a sandbox directory with small fixtures representing active, recent, old, locked, and malformed files. Run the endpoint in dry-run mode, inspect the planned actions, and then test real rotation without production data. The checklist in testing a web cron job before production helps cover authorization, failures, and repeated execution.
Operational checklist
- Allowlist every managed directory and resolve paths safely.
- Use a lock with bounded stale-lock recovery.
- Rename active logs atomically and reopen file handles correctly.
- Compress to a temporary file and validate before replacement.
- Apply both age and total-size retention rules.
- Protect held archives and record every deletion.
- Return structured metrics and alert on missed runs.
Final takeaway
Automated log rotation is a small maintenance task with large reliability benefits. A narrowly scoped, idempotent endpoint scheduled through a web cron service keeps storage predictable while preserving the evidence operators need when problems occur.