api_backoff¶
Observable backoff sleep helper for retrying throttled external API calls.
The problem¶
When a request is throttled, the API provider usually tells you how long to wait — for example, an HTTP 429 response carries a Retry-After header, or you back off for a known interval. You need to sleep that exact amount before retrying. A plain asyncio.sleep works, but operators cannot see that a service is being throttled, or for how long. When something goes wrong in production, you want the logs to tell you which service is throttled and how long the wait is.
How this bite helps¶
You determine the wait time — scraped from the provider's Retry-After header, or computed from an exponential backoff or the rate limiter's refill estimate — and pass it to async_backoff. It sleeps that exact time and logs a WARNING only when the wait exceeds a configurable threshold, so a long or throttled stall shows up in structured logs instead of an invisible silence. You also pass a short context string so the log line says which service and which retry.
from langshark_bites.api_backoff import async_backoff, retry_after_seconds
# Provider said to wait 30s (Retry-After header); fall back to 30s if absent:
wait = retry_after_seconds(response) or 30.0
await async_backoff(wait, context="NewsAPI retry 1/3")
async_backoff is a drop-in replacement for asyncio.sleep: anywhere you currently await asyncio.sleep(wait) before a retry, you can call await async_backoff(wait, context=...) instead and get the observability for free.
What topologies it supports¶
- Any node that retries a throttled API call.
- Works alongside
api_rate_limiterfor the residual cases where you still get throttled. - Standalone in any async retry loop.
Example¶
See examples/backoff.py for a runnable example of this bite. Run it with:
API reference¶
Observability utility for rate-limit backoff delays.
Provides async_backoff — a drop-in replacement for asyncio.sleep
that sleeps the wait time determined by the caller and logs a WARNING
only when that wait exceeds a configurable threshold. The wait time
typically comes from the provider: the Retry-After header on an HTTP
429 response, an exponential backoff schedule, or the rate limiter's
token-bucket refill estimate. The caller passes that value in, so the
helper makes an otherwise-silent sleep visible in structured logs and
records which service was throttled and for how long.
Usage::
from langshark_bites.api_backoff import async_backoff, retry_after_seconds
# Provider says to wait before retrying (Retry-After header);
# fall back to 30s if the header is missing:
wait = retry_after_seconds(response) or 30.0
await async_backoff(wait, context="NewsAPI retry 1/3")
The warning threshold is passed as a parameter (default 10s) so callers can adapt it to their own project's needs.
async
¶
async_backoff(
delay_sec, *, context="", warning_threshold_sec=10.0
)
Sleep delay_sec seconds, logging a WARNING for long delays.
Emits the WARNING when the delay exceeds warning_threshold_sec.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
|
float
|
The number of seconds to sleep (non-blocking async sleep). |
required |
|
str
|
A short description of what is being rate-limited (e.g.
|
''
|
|
float
|
Log a WARNING when delay_sec is >= this threshold. Set to 0 to log every retry; set to a large value to suppress completely. |
10.0
|
Return the wait time (seconds) from an HTTP response's Retry-After.
Accepts any object with a .headers mapping (for example
httpx.Response, requests.Response, or an ASGI response). Handles
both forms allowed by the HTTP spec (RFC 9110):
- an integer number of seconds:
Retry-After: 30->30.0 - an HTTP-date:
Retry-After: Wed, 21 Oct 2015 07:28:00 GMT-> seconds from now until that instant.
Returns None if the header is absent or cannot be parsed, so callers
can fall back to a fixed backoff.
Usage::
from langshark_bites.api_backoff import async_backoff, retry_after_seconds
wait = retry_after_seconds(response) or 30.0
await async_backoff(wait, context="NewsAPI retry 1/3")