Aggregate Structured Log Lines

easyPROObservability & SLOsSystems & InfrastructureML Systems & Infrastructure 1

Problem Statement

Structured logging exists so that log lines are data, not prose: one JSON object per line, parsed by the aggregator rather than by a regex someone wrote at 3am. Aggregate a batch of such lines into the numbers a dashboard actually shows.

Background

A well-behaved line looks like:

{"ts": "2026-08-17T10:00:01Z", "level": "INFO", "status": 200, "path": "/predict", "latency_ms": 12.5}

Real log streams are never that clean — a library writes a plain-text line, a crash truncates a record. Log hygiene means counting those separately instead of letting them silently vanish or crash the parser.

Rules:

A line is valid when it parses as JSON, is an object, and has a "status" key. Anything else is malformed and counted, not aggregated.
Bucket each valid status by its class: 200 -> "2xx", 404 -> "4xx", 503 -> "5xx".
error_rate is the fraction of valid lines whose status is a 5xx (server errors only — a 404 is the caller's problem), rounded to 4 decimals.
top_error_path is the path with the most 5xx responses; break ties alphabetically. None when there were no 5xx lines. Use "unknown" when a 5xx line has no path.

Dict key order in the output must be deterministic, so by_class is built with its keys sorted.

Your Task

Implement:

def analyze_logs(lines):

Return a dict with keys "total", "malformed", "by_class", "error_rate", "top_error_path", in that order.

Input Format

lines: list of strings, each intended to be one JSON log record.

Output Format

The dict described above. "total" counts only valid lines.

Sample

lines = ['{"status": 200, "path": "/predict"}',
         '{"status": 500, "path": "/predict"}',
         'oops not json',
         '{"status": 404, "path": "/v1/missing"}']
print(analyze_logs(lines))

Output:

{'total': 3, 'malformed': 1, 'by_class': {'2xx': 1, '4xx': 1, '5xx': 1}, 'error_rate': 0.3333, 'top_error_path': '/predict'}

Three lines parse; the plain-text line is counted as malformed rather than dropped. One of the three valid lines is a 5xx, so the error rate is 0.3333.

Example

Input

lines = ['{"status": 200, "path": "/predict"}', '{"status": 500, "path": "/predict"}', 'oops not json', '{"status": 404, "path": "/v1/missing"}']
print(analyze_logs(lines))

Output

{'total': 3, 'malformed': 1, 'by_class': {'2xx': 1, '4xx': 1, '5xx': 1}, 'error_rate': 0.3333, 'top_error_path': '/predict'}

Reasoning

oops not json fails to parse, so it is counted as malformed and excluded from total. The three valid lines bucket to 2xx, 5xx and 4xx by integer-dividing the status by 100. Only the 500 counts toward the error rate: 1 / 3 = 0.3333, and its path is the only 5xx path, so it is the top offender.

Constraints

0 <= len(lines) <= 100000
A line is valid only if it parses as a JSON object containing a "status" key; everything else counts as malformed
by_class keys are "1xx".."5xx", present only when observed, and must be inserted in sorted order
error_rate counts 5xx only, over valid lines, rounded to 4 decimals
top_error_path breaks ties alphabetically and is None when there are no 5xx lines
A 5xx line with no "path" is attributed to "unknown"

Learn this in

Monitoring & ObservabilitySystems & Networking Networking FundamentalsSystems & Networking

More from ML Systems & Infrastructure 1

Spot vs On-Demand Expected-Cost Decisionmedium Minimum-Cost Job Scheduling Across Priced Windowshard Deduplicate Retried Pipeline Runs by Idempotency Keymedium Retry with Capped Exponential Backoff and Jitter Capmedium Critical Path Length of a Pipeline DAGmedium

Concept primerPREVIEW

1. Background Knowledge

Structured Logging transforms application logs from unstructured text into machine-readable formats, typically JSON. This allows aggregators to parse fields like status, latency, and path without brittle regular expressions. In observability systems, log hygiene is critical: malformed lines must be counted separately to ensure dashboards reflect data quality issues rather than silently dropping them.

HTTP Status Code Classes group responses by their first digit: 2xx (Success), 4xx (Client Error), and 5xx (Server Error). For SLOs (Service Level Objectives), error rates usually focus on 5xx codes, as 4xx errors often indicate client misuse rather than system failure. The error rate is calculated as the fraction of valid requests that resulted in a server error.

Deterministic Output is essential for testing and debugging. When aggregating data, keys in dictionaries (like by_class) should be sorted to ensure consistent ordering across runs. Tie-breaking rules (e.g., alphabetical order for top_error_path) further ensure deterministic results when multiple paths have the same error count.

2. Algorithm Approach

The problem requires a single-pass aggregation algorithm. Iterate through each log line, attempt to parse it as JSON, and classify it as either valid or malformed. For valid lines, extract the status code, bucket it into its class (2xx, 4xx, 5xx), and track 5xx errors by path. Finally, compute the error rate and determine the top error path using the collected counts.

Key components:

Parsing & Validation: Use json.loads() with try-except blocks to handle malformed JSON.
Classification: Map status codes to classes using integer division or string slicing.
Aggregation: Use dictionaries to count occurrences of each class and error paths.
Post-Processing: Calculate the error rate and find the top error path with tie-breaking.

3. Step-by-Step Strategy

PRO

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.

PRO

Editor locked

The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.

solution.py

Python 3.13

Test results