Maaz Zindani

Loading the index…

↑↓ move↵ openesc close4072 pages
AI and LLM integration

The OpenAI API answers 429: out of credit, or just too fast?

Error429 Rate limit reached for requests

Short answer

A 429 from the OpenAI API means one of two unrelated things. If error.type is insufficient_quota, or the code names a credit, spend or usage limit, billing has stopped the key and retrying never helps. Otherwise a requests- or tokens-per-minute limit was hit: read the x-ratelimit headers, honour Retry-After, back off with jitter, and lower concurrency or token use.

  • Applies to openai Node.js SDK 7, Responses API, Node.js 22, p-limit 6
  • Updated
  • By Maaz Zindani, Senior Software Engineer

Fix it fast

  1. 1Find out which cause you have

    Make one small call with curl and print the status, the rate-limit headers and the error fields. A billing 429 shows up on the very first request and carries a billing type or code. A rate-limit 429 comes with x-ratelimit-remaining-* values at or near zero.

    curl -s -i https://api.openai.com/v1/responses \
      -H "Authorization: Bearer $OPENAI_API_KEY" \
      -H "Content-Type: application/json" \
      -d "{\"model\": \"$OPENAI_MODEL\", \"input\": \"ping\", \"max_output_tokens\": 16}" \
      | grep -iE '^HTTP|^x-ratelimit|^retry-after|"(type|code)"'
  2. 2Match what you see, then jump to its fix

    1. 1error.type is insufficient_quota, or error.code is credit_balance_exhausted, organization_spend_limit_exceeded, project_spend_limit_exceeded or organization_usage_limit_exceeded; it fails even at one request per minute.Billing has stopped the key: credit, spend or usage limitFix
    2. 2Errors arrive in bursts when a job fans out (Promise.all over a list), and x-ratelimit-remaining-requests reaches 0 while token headroom is left.Too many requests at once: the requests-per-minute limitFix
    3. 3x-ratelimit-remaining-tokens drops to zero while requests remain; long prompts, long chat histories or a large max_output_tokens are involved.Too many tokens per minute, often inflated by max tokensFix
    4. 4The 429 count keeps rising after the first few, and the number of HTTP calls is a multiple of the number of jobs.Retries multiply the trafficFix
    5. 5429s appear while this service's own traffic is low, often when a cron job, a backfill or a second environment runs at the same time.Another service or environment shares the same limitFix

What does the error look like?

  • 429 Rate limit reached for requests
  • "type": "insufficient_quota"
  • "code": "credit_balance_exhausted"
  • "code": "slow_down"

Cause 1 of 5

Billing has stopped the key: credit, spend or usage limit

1Check

Check the organization's prepaid credit and any spend limit set on the organization or on the project the key belongs to (Settings in the OpenAI dashboard). Note which project the failing key was created in; a project cap can stop one key while others work.

curl -s https://api.openai.com/v1/responses -H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" -d "{\"model\": \"$OPENAI_MODEL\", \"input\": \"ping\"}" | grep -E '"(type|code)"'

2Fix

Add credit, or raise or remove the spend limit that was reached. In code, treat these as a stop signal: no retries (they cannot succeed and only fill your logs), one alert to a person, and a clear message to users that the AI feature is paused. Anthropic's API has the same split: its monthly spend cap also answers 429, but without a retry-after header and with error.details.error_code set to enforced_spend_limit_reached.

import OpenAI from 'openai'

export function isBillingStop(err: unknown): boolean {
  return err instanceof OpenAI.APIError && err.status === 429 &&
    (err.type === 'insufficient_quota' || /credit|spend|usage_limit/.test(err.code ?? ''))
}

// where you call the model
try {
  return await client.responses.create(params)
} catch (err) {
  if (isBillingStop(err)) {
    await alertOnce('OpenAI billing limit reached', (err as OpenAI.APIError).requestID)
    throw new Error('AI features paused: billing limit', { cause: err })
  }
  throw err
}

3Verify

The same curl call returns HTTP 200 with x-ratelimit headers, and the alert fires once in a test where the key has no credit, not once per request.

curl -s -o /dev/null -w "%{http_code}\n" https://api.openai.com/v1/responses -H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" -d "{\"model\": \"$OPENAI_MODEL\", \"input\": \"ping\"}"
Did this fix it?

Cause 2 of 5

Too many requests at once: the requests-per-minute limit

1Check

Log the request headers on successful calls during a normal run. x-ratelimit-limit-requests is your ceiling, the remaining value shows how close you get, and x-ratelimit-reset-requests says when it refills. Limits apply per organization and per project, not per user or per server.

const { response } = await client.responses.create(params).withResponse()
console.log({
  limit: response.headers.get('x-ratelimit-limit-requests'),
  remaining: response.headers.get('x-ratelimit-remaining-requests'),
  reset: response.headers.get('x-ratelimit-reset-requests'),
})

2Fix

Cap how many calls are in flight, and put one retry wrapper around each call that waits for Retry-After when it is present and otherwise backs off exponentially with random jitter, so many workers do not retry in step. If requests are the binding limit but tokens are not, combine several small tasks into one request.

import OpenAI from 'openai'
import pLimit from 'p-limit'

const client = new OpenAI({ maxRetries: 0 })   // this wrapper is the only retry layer
const limit = pLimit(4)                        // at most 4 calls in flight

function waitMs(err: OpenAI.APIError, attempt: number): number {
  const seconds = Number(err.headers?.get('retry-after'))
  if (seconds > 0) return seconds * 1000 + Math.random() * 1000   // server's wait plus jitter
  const ceiling = Math.min(30_000, 500 * 2 ** attempt)            // 0.5s, 1s, 2s ... up to 30s
  return ceiling / 2 + Math.random() * (ceiling / 2)
}

export async function callModel(params: OpenAI.Responses.ResponseCreateParamsNonStreaming, maxAttempts = 5) {
  for (let attempt = 0; ; attempt++) {
    try {
      return await limit(() => client.responses.create(params))
    } catch (err) {
      const retryable = err instanceof OpenAI.APIError && err.status === 429 && !isBillingStop(err)
      if (!retryable || attempt + 1 >= maxAttempts) throw err
      await new Promise((r) => setTimeout(r, waitMs(err, attempt)))   // sleeps outside the limiter
    }
  }
}

3Verify

Run the same job and count outcomes: 429s should be rare, each followed by a success, and the job's total time should stay close to items divided by the allowed rate.

node scripts/run-job.mjs 2>&1 | grep -c ' 429 '
Did this fix it?

Cause 3 of 5

Too many tokens per minute, often inflated by max tokens

1Check

OpenAI counts a request against the token limit by the larger of its max tokens setting and the estimated size of the input, before the model writes anything. A generous max_output_tokens therefore uses up budget even when answers are short. Compare the token headers with the request headers.

const { response } = await client.responses.create(params).withResponse()
for (const h of ['x-ratelimit-remaining-tokens', 'x-ratelimit-reset-tokens', 'x-ratelimit-remaining-project-tokens']) {
  console.log(h, response.headers.get(h))
}

2Fix

Set max_output_tokens close to the real answer size, send only the context the answer needs, and move bulk work that can wait (backfills, nightly summaries) to the Batch API, which does not count against the per-minute limits. Limits also rise with usage tier, which goes up automatically as total credit purchases cross each threshold; the current per-model values are under Settings, Organization, Limits.

await callModel({
  model: process.env.OPENAI_MODEL!,
  instructions: SUPPORT_PROMPT,
  input: lastTurns(history, 6),   // the turns the answer needs, not the whole thread
  max_output_tokens: 400,         // near the real answer size; it is counted up front
})

3Verify

During a normal run x-ratelimit-remaining-tokens stays above zero, and the per-request token usage in your logs falls.

grep -o 'x-ratelimit-remaining-tokens [0-9]*' app.log | sort -t' ' -k2 -n | head -3
Did this fix it?

Cause 4 of 5

Retries multiply the traffic

1Check

Count the retry layers: the official SDKs retry 429s by themselves (twice by default in Node), your code may loop again, and a queue may re-run the whole job. Failed requests also count toward the per-minute limit, so stacked retries keep you over it.

grep -rnE "maxRetries|retry|backoff|attempts" src lib --include=*.ts --include=*.js 2>/dev/null

2Fix

Keep exactly one retry layer. Either keep the SDK's retries and remove your own loop, or set maxRetries: 0 and use a wrapper like the one above. Cap attempts and total time, and make queue-level retries start only after the call-level ones are used up.

// option A: let the SDK retry, no loop of your own
const client = new OpenAI({ maxRetries: 3 })

// option B: own wrapper, SDK retries off (as in callModel above)
const client = new OpenAI({ maxRetries: 0 })

// queue: no job-level retry for rate limits that the call already retried
worker.on('failed', (job, err) => { if (isBillingStop(err)) pauseQueue() })

3Verify

Log one line per HTTP attempt for a test job: attempts per job should never exceed the cap you set.

grep -c 'openai attempt' app.log
Did this fix it?

Cause 5 of 5

Another service or environment shares the same limit

1Check

Look at usage in the dashboard per project and per API key around the time of the errors. Staging, a batch worker and production on keys from one project draw from the same per-minute pool.

grep -h 'x-ratelimit-remaining-tokens' prod.log worker.log | awk '{print $1, $2, $NF}' | sort | tail -20

2Fix

Give each environment and each background workload its own project and key, schedule bulk jobs away from peak hours or move them to the Batch API, and throttle the background job harder than the live path.

// background worker: smaller concurrency than the live API path
const limit = pLimit(Number(process.env.OPENAI_CONCURRENCY ?? 1))

3Verify

Run the background job during a busy hour: the live service's 429 rate stays flat.

grep -c ' 429 ' prod.log
Did this fix it?

How do I stop it coming back?

  • Classify every 429 as billing or rate limit in one helper, and alert on billing ones instead of retrying.
  • Log x-ratelimit-remaining-requests and x-ratelimit-remaining-tokens on every call, and alert at 10 percent remaining.
  • Keep one retry layer, with jitter and a cap on attempts and total wait.
  • Set a monthly budget alert and a spend limit per project so a runaway job stops before it stops production.

Sources

How was this page made?

I drafted this page with AI assistance from the official documents listed under Sources. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.

Which stack is behind this page?

Projects of mine built on the same technologies:

Related fixes

AI

The model's reply breaks JSON.parse

SyntaxError: Unterminated string in JSON at position 1874

JSON.parse rejects model output for a few repeatable reasons: the JSON sits inside markdown fences or next to a sentence, the reply stopped at the token limit mid-object, a stream was parsed before it ended, or the model refused.

5 causes, each with a fix

Next step

Want this handled for good?

I look after servers, apps and releases end to end. Tell me what keeps breaking.