A page can keep returning HTTP 200 while quietly becoming much heavier. A proxy rule may stop applying Brotli, a CDN migration may drop gzip for JSON, or a new content type may bypass compression entirely. Users then download more bytes, mobile pages slow down, and bandwidth costs rise even though ordinary uptime checks remain green.
This guide shows how to use a web cron job to detect HTTP compression regressions. The monitor records encoding, transfer size, content size, cache behavior, and relevant headers, then compares the result with a stable baseline for each representative URL.
Why compression needs its own check
Availability and latency checks do not prove that a response is compressed. A small test page might remain fast while large JavaScript, CSS, JSON, HTML, or SVG responses become several times larger. Synthetic page-speed tests can reveal the effect later, but a focused compression monitor pinpoints the header and size change immediately.
Treat compression as one layer of edge configuration. Pair it with TLS configuration monitoring and redirect-chain monitoring when proxies, CDNs, or load balancers are frequently changed.
Select representative resources
Do not scan only the home page. Build a small, versioned list that covers the response types most important to users and bandwidth:
- HTML for a public landing page and a dynamic application page;
- one large JavaScript bundle and one stylesheet;
- a typical JSON API response;
- SVG, XML, or text feeds when the site serves them;
- an authenticated response through a safe test account, if appropriate.
Avoid image, video, archive, and already-compressed font formats unless you are checking that the server correctly leaves them alone. Recompressing these formats wastes CPU and can even increase the transfer size.
Send realistic negotiation headers
The probe should advertise the encodings that supported browsers use, then inspect the server’s selected Content-Encoding. Run separate policy checks when you need to verify Brotli over HTTPS and gzip compatibility for older clients. Preserve a consistent user agent and request shape so baseline comparisons remain meaningful.
Check that responses varying by accepted encoding include an appropriate Vary: Accept-Encoding signal. Missing variation can allow caches to serve compressed bytes to a client that did not request them, or to reuse an uncompressed object for capable clients.
Measure both encoded and decoded sizes
Store the wire transfer size and the decoded content size. Their ratio shows how much compression was achieved. A missing encoding header is an obvious regression, but a sharp ratio change may reveal a minification failure, accidental whitespace, embedded data, or a different payload shape.
Keep response status, content type, content encoding, content length when available, ETag, cache status, resolved edge, and deployment version. Normalize volatile values before fingerprinting the header set. For dynamic APIs, also record an application-level item count so a larger legitimate payload is not mistaken for worse compression.
Establish per-resource policies
Different files compress differently, so avoid one universal percentage. Define expectations for each monitored URL or resource class:
- required encoding for compressible responses above a minimum size;
- maximum encoded size or an allowed increase from the recent median;
- minimum compression ratio for stable assets;
- expected
Varybehavior and content type; - exceptions for tiny responses where compression overhead is not worthwhile.
Versioned static assets can use strict byte thresholds. Dynamic responses should use wider ranges, a normalized fixture, or a ratio based on comparable payload volume.
Schedule checks without polluting caches
Run the monitor hourly or daily for stable sites and after every CDN, proxy, or build deployment. Use a bounded query parameter only if it is approved and will not create unlimited cache objects. Often it is better to request the normal public URL, record cache status, and occasionally perform a separate controlled origin check.
Keep the web cron job idempotent. Store one result per URL, encoding mode, and run time. Retries must update the same incident rather than opening duplicate alerts. Track monitor duration as well; scheduled job duration monitoring can reveal when an edge scan itself becomes slow.
Distinguish real regressions from normal changes
A new application release may legitimately change bundle size. Compare the result with deployment metadata and the asset’s versioned URL. For mutable HTML or JSON, require repeated threshold breaches or compare against the same fixture.
Test from a second location when only one CDN region fails. A single edge node may have stale configuration while the majority remains correct. Record the resolved address or edge identifier when the provider exposes a non-sensitive diagnostic header.
Write actionable alerts
The alert should name the URL, expected and observed encoding, previous and current transfer size, decoded size, ratio, cache state, first-seen time, and affected location. Include the recent deployment and a safe command or dashboard link for reproduction.
Classify missing compression separately from payload growth. The first usually points to server or CDN configuration; the second may be an application change. If API fields changed at the same time, correlate the result with API schema drift monitoring.
Investigate in a consistent order
- Repeat the request with and without each accepted encoding.
- Confirm the content type is eligible for compression.
- Check CDN, reverse-proxy, and origin headers separately.
- Compare cached and uncached responses.
- Review the build output for minification or embedded-data changes.
- Test multiple regions and every relevant hostname.
- Correct the configuration, purge only necessary cache objects, and recheck.
Do not enable compression blindly for secrets reflected in attacker-controlled responses. Review sensitive authenticated endpoints for compression side-channel risk and exclude them when the safer policy is no compression.
Track the long-term trend
A weekly view of encoded bytes, decoded bytes, ratio, and cache hit status shows gradual growth before it becomes an incident. It also demonstrates whether a CDN or build optimization delivered a real reduction for users.
HTTP compression monitoring turns a subtle performance dependency into a clear operational contract. With representative resources, realistic negotiation, resource-specific thresholds, and concise alerts, a web cron job can catch expensive regressions minutes after they appear.