API response increased from 300ms to 8 seconds. How do you debug?

A jump from 300ms to 8 seconds is usually a traffic jam somewhere in the request path, not a single slow line of Java.

Question: API response increased from 300ms to 8 seconds. How do you debug?

Answer: I debug this by first proving where the time is spent: client, network, application, database, or a downstream service. I compare p50, p95, and p99 latency, inspect traces and logs for one slow request, and check saturation signals like CPU, GC, thread pools, and connection pools. Once I find the slow hop, I compare before and after the change to see whether it is a recent deploy, a config change, a dependency slowdown, or queueing under load.

Interview-Ready Answer: I start by narrowing the problem: is the 8 seconds on every request, only some requests, or only under load? Then I use metrics, logs, and distributed tracing to break the request into steps and find the slowest hop, often a database lock, connection pool exhaustion, or a retry loop. If this started after a deploy, I compare before and after, check slow query logs and thread dumps, and rollback or fix the bottleneck once I have evidence.

What is actually happening

A latency spike means the request is waiting somewhere. In Java services, the most common waits are blocked threads, a slow database query, a full connection pool, garbage collection pauses, or a downstream API that is timing out and being retried. The key idea is simple: do not guess from one symptom; break the request into smaller pieces until one piece explains most of the 8 seconds.

Step-by-step debug flow

1. Confirm the scope. Check whether the slowdown affects all users, one region, one endpoint, one payload size, or only p95 and p99. Averages can lie; one very slow path can be hidden by many fast requests.
2. Compare against a known-good baseline. Did this start after a deploy, config change, schema migration, traffic spike, or dependency incident? If latency jumped right after a release, that is a strong clue for a regression.
3. Break the request into spans. Use distributed tracing with a correlation ID so one request can be followed across controller, service, database, and downstream calls. The slowest span usually tells you where the time went.
4. Check saturation signals. Look at CPU, memory, GC pauses, thread pool queue depth, DB connection pool usage, and HTTP client pool usage. High latency with low CPU often means waiting, not computing.
5. Inspect the usual suspects. In Java systems this is often a slow SQL query, lock contention, synchronized blocks, N+1 database calls, cache misses, DNS or TLS setup, or retries multiplying a small failure into a huge delay.
6. Prove the cause by isolation. Call the dependency directly, bypass the cache, disable retries temporarily in a test environment, or replay the request with a smaller payload. If the problem disappears when one piece is removed, you found the bottleneck.
7. Fix and prevent. Add the right timeout, limit retries, tune the query, enlarge or shrink a pool carefully, or rollback the bad change. Then add a dashboard and alert so the same issue is caught earlier next time.

How to read the signals

Under the hood of a good debugging approach

1. Measure with monotonic time. In Java, use System.nanoTime() for elapsed time because it is monotonic, meaning it is not affected by clock changes. currentTimeMillis() is fine for timestamps, but not for precise duration measurements.
2. Follow the request path. A request usually enters through a load balancer, lands on a servlet thread, executes business logic, calls a database or cache, and maybe calls another service. The 8 seconds can be in any hop, including waiting for a free thread or connection.
3. Separate compute from wait. A CPU profile tells you where the code burns time on the processor. A thread dump tells you where threads are blocked. If latency is high but CPU is low, the thread is probably waiting.
4. Watch for retries. A 2 second timeout with 3 retries can become 6 to 8 seconds very quickly. Retries are useful for resilience, but without limits they amplify latency and load.
5. Use the smallest useful blast radius. If you can reproduce the slowdown in one endpoint, one tenant, or one dependency call, debugging becomes much faster than staring at the whole service.

Comparison table: which tool answers which question

Real numbers interviewers like

• p50, p95, and p99 matter more than average for user experience.
• Common HTTP client connect timeouts are often around 1 to 3 seconds; read timeouts depend on the endpoint, but should be intentional, not infinite.
• DB connection pools are often sized in the tens per instance, not hundreds, and pool exhaustion can turn a fast query into a long wait.
• A small GC pause is normal, but repeated long pauses can add hundreds of milliseconds or seconds.

Memory hook

Think of the request as a highway: do not stare at one car, find the lane with the traffic jam. The job is to locate the blocked lane, not to guess which driver is late.

Real-World Story: In an e-commerce checkout service, a new pricing lookup was added so the API could show live discounts. The code worked in staging, but in production the pricing service occasionally slowed down to several seconds. Because the call had a long timeout and retries, checkout threads piled up, the Tomcat request queue grew, and users saw spinning loaders while trying to pay. The logs showed Read timed out messages, and the trace made it obvious that almost all of the 8 seconds sat inside one downstream span. The fix was to add a short timeout, a circuit breaker, and a fallback price path so checkout stayed fast even when pricing was unhealthy.

What goes wrong when this is misunderstood: Teams often blame the controller or add more servers too early. That can hide the real issue for a while, but the symptoms keep showing up as rising thread counts, connection pool exhaustion, and user complaints like slow pages, failed logins, or abandoned carts. The outage is not just slower code; it is queued work building up faster than the system can drain it.

Follow-up & Tricky Questions:

• How do you tell if the problem is client-side or server-side? I check end-to-end tracing and compare client timing with server timing. If the client sees 8 seconds but the server trace is only 200ms, the extra time is in network, DNS, TLS, retries, or client-side queuing.
• What metrics do you check first? I start with p50, p95, p99, error rate, CPU, GC pause time, thread pool queue length, DB pool usage, and downstream timeout counts. Those usually show whether the system is waiting, overloaded, or failing fast.
• What if only one instance is slow? That suggests node-specific trouble such as bad deployment, noisy neighbor, cache corruption, or a host-level issue. I compare that instance with healthy ones before assuming the application code is broken.
• When would you rollback? If the spike started right after a deploy and the trace points to the new path, rollback is often the safest immediate mitigation. I still keep investigating, but I do not wait to protect users.
• How do you confirm a database issue? I look for slow query logs, lock waits, increased connection pool wait time, and query plans that changed after a schema or data change. If a direct DB call is slow, the app is often just the messenger.
• Why can retries make latency worse? Each retry adds another wait, so a 2 second timeout with 3 attempts can become 6 to 8 seconds very easily. Retries should be capped, jittered, and used only for failures that are likely to succeed on another try.
• Could GC alone cause 8 seconds? Yes, repeated stop-the-world pauses or memory pressure can cause long stalls, especially if the heap is too small or allocation rate is high. I would check GC logs and pause metrics before blaming the database.
• Is restarting the service a valid fix? It is only a temporary mitigation if you still need to find the root cause. A restart may clear a stuck pool or bad state, but it does not explain why the latency jumped in the first place.

Tricky / gotcha questions:

• Could the average latency stay flat while users still complain? Yes. Tail latency can spike while the average looks fine, so p95 and p99 are much better signals for user pain.
• If CPU is low, does that mean the service is healthy? No. Low CPU with high latency often means the service is blocked on I/O, a lock, or a pool wait.
• Can a slow API be caused by the client? Yes. Bad client timeouts, DNS issues, TLS handshake delays, or client-side retries can all make the user think the server is slow even when server processing is fine.

Common Mistakes:

• Jumping straight into code. Correction: First check metrics and traces so you know which layer is slow.
• Using only average latency. Correction: Look at p95 and p99, because tail latency is what users feel.
• Restarting immediately. Correction: Restart only as a mitigation after you capture evidence, not as the diagnosis.
• Forgetting retries and timeouts. Correction: A few retries can turn one small delay into seconds of extra waiting.

Memory Hook: Traffic jam first, engine second. When latency explodes, look for the blocked lane in the request path before you inspect the code that drove into it.

Cheat Sheet:

• Check scope: all requests, some requests, or only under load.
• Use p95 and p99, not just average latency.
• Break the request into spans with tracing.
• Look for saturation: CPU, GC, threads, DB pools, HTTP pools.
• Watch for retries, locks, slow queries, and downstream timeouts.
• Rollback fast if the problem began after a deploy.

Practice Tasks:

• Wrap one Java method with System.nanoTime() and print its duration.
• Add timing around a fake DB call and a fake downstream call, then make one of them slow.
• Simulate a retry loop and see how a 2 second timeout becomes many seconds of user delay.