The model's reply breaks JSON.parse
SyntaxError: Unexpected token '`', "```json ..." is not valid JSONShort answer
JSON.parse rejects model output for a few repeatable reasons: the JSON sits inside markdown fences or next to a sentence, the reply stopped at the token limit mid-object, a stream was parsed before it ended, or the model refused. Ask for schema-constrained output (OpenAI Structured Outputs, Anthropic structured outputs), check why generation stopped before parsing, and validate the result with a schema.
Fix it fast
1Find out which cause you have
Before changing the prompt, log one failing call: why generation stopped, and the first and last characters of the raw text. Fences or a polite sentence show up at the start or the end; a reply that was cut off ends in the middle of a value and comes with a token-limit stop reason.
console.log(JSON.stringify({ stopped: res.status ?? res.stop_reason, // OpenAI Responses: status; Anthropic Messages: stop_reason why: res.incomplete_details?.reason, // OpenAI only, e.g. 'max_output_tokens' head: text.slice(0, 80), tail: text.slice(-80), }))2Match what you see, then jump to its fix
- 1
Unexpected token '`' (a fence), Unexpected token 'S' or 'H' (a preface such as Sure, here is...), or Unexpected non-whitespace character after JSON (a closing remark).The JSON comes wrapped in code fences or a sentenceFix - 2
Unterminated string in JSON, Unexpected end of JSON input, or Expected ',' or ']' after array element, mostly on long inputs or long lists.The reply hit the token limit and stopped mid-objectFix - 3
The errors appear only with stream: true, the position in the message is small, and JSON.parse sits inside the loop that reads stream events.A streamed reply is parsed before it has finishedFix - 4
Unexpected token 'I', "I can't he"... and similar; with Anthropic structured outputs the response has HTTP 200 and stop_reason refusal; with OpenAI the output contains a refusal item instead of text.The model refused, so the reply is proseFix - 5
No SyntaxError, but later code throws TypeError: Cannot read properties of undefined, or an enum value differs by capitalisation.The JSON parses, but its shape is wrongFix
- 1
What does the error look like?
SyntaxError: Unexpected token '`', "```json ..." is not valid JSONSyntaxError: Unterminated string in JSON at position 1874SyntaxError: Unexpected end of JSON inputSyntaxError: Unexpected non-whitespace character after JSON at position 212SyntaxError: Unexpected token 'I', "I can't he"... is not valid JSON
Cause 1 of 5
The JSON comes wrapped in code fences or a sentence
1Check
The head and tail in the log show the fence or the extra words. This happens when JSON is only requested in the prompt: nothing stops the model from formatting the answer the way chat replies usually look. OpenAI's older JSON mode guarantees valid JSON but not your schema, and it errors unless the word JSON appears in the input.
const looksFenced = /^\s*```/.test(text) || /```\s*$/.test(text)
const hasPreface = !/^\s*[[{]/.test(text)
console.log({ looksFenced, hasPreface })2Fix
Let the API enforce the shape. OpenAI Structured Outputs (a json_schema text format with strict mode) and Anthropic structured outputs (output_config.format) constrain generation to your JSON Schema, and both Node.js SDKs accept a Zod schema and return a parsed object. Under strict OpenAI schemas every field must be required; express optional values as nullable.
import { z } from 'zod'
import OpenAI from 'openai'
import { zodTextFormat } from 'openai/helpers/zod'
import Anthropic from '@anthropic-ai/sdk'
import { zodOutputFormat } from '@anthropic-ai/sdk/helpers/zod'
const Invoice = z.object({
vendor: z.string(),
total: z.number(),
currency: z.string(),
due_date: z.string().nullable(), // optional in practice, but present in the output
})
// OpenAI (Responses API)
const openai = new OpenAI()
const r = await openai.responses.parse({
model: process.env.OPENAI_MODEL!,
input: [{ role: 'user', content: invoiceText }],
text: { format: zodTextFormat(Invoice, 'invoice') },
})
const fromOpenAI = r.output_parsed
// Anthropic (Messages API)
const anthropic = new Anthropic()
const m = await anthropic.messages.parse({
model: process.env.ANTHROPIC_MODEL!,
max_tokens: 1024,
messages: [{ role: 'user', content: invoiceText }],
output_config: { format: zodOutputFormat(Invoice) },
})
const fromClaude = m.parsed_output3Verify
Run the same input many times and count parse failures; with schema-constrained output it should be zero.
let failures = 0
for (let i = 0; i < 50; i++) {
const r = await extractInvoice(sampleText)
if (!r) failures++
}
console.log({ failures })Cause 2 of 5
The reply hit the token limit and stopped mid-object
1Check
Look at why generation stopped. OpenAI's Responses API marks the response incomplete with the reason max_output_tokens; Anthropic sets stop_reason to max_tokens. Both providers warn that a cut-off reply may not match the schema, even in structured mode.
// OpenAI Responses API
if (res.status === 'incomplete') console.log('cut off:', res.incomplete_details?.reason)
// Anthropic Messages API
if (msg.stop_reason === 'max_tokens') console.log('cut off at', msg.usage.output_tokens, 'output tokens')2Fix
Treat a cut-off reply as a failed call, never as text to parse or repair. Raise the output limit with headroom, ask for fewer items per call and page through the rest, and keep free-text fields short in the schema description.
function assertComplete(res: { status?: string; stop_reason?: string | null }) {
if (res.status === 'incomplete' || res.stop_reason === 'max_tokens') {
throw new Error('model output truncated: raise the token limit or request fewer items')
}
}
// ask for 25 items per call instead of everything at once
for (let page = 0; page * 25 < rows.length; page++) {
const chunk = rows.slice(page * 25, (page + 1) * 25)
results.push(...(await classify(chunk)))
}3Verify
Log output tokens against the limit for a week of traffic; the largest reply should stay well under the limit, and no call should end with a token-limit stop.
grep -o '"output_tokens":[0-9]*' llm.log | cut -d: -f2 | sort -n | tail -1Cause 3 of 5
A streamed reply is parsed before it has finished
1Check
Each streamed event carries only a fragment of the text. Parsing a fragment fails, and parsing the text so far fails until the last delta arrives.
grep -n -A6 'for await' src/llm/*.ts | grep -n 'JSON.parse'2Fix
Append the text deltas to a buffer and parse once, after the stream reports completion. If the user needs to see progress, show the raw text while it streams and swap in the parsed result at the end.
const stream = await openai.responses.create({ ...params, stream: true })
let text = ''
for await (const event of stream) {
if (event.type === 'response.output_text.delta') text += event.delta
if (event.type === 'response.completed') break
}
const data = Invoice.parse(JSON.parse(text)) // parse once, after the last delta3Verify
Stream and non-stream versions of the same request return identical parsed objects.
console.log(JSON.stringify(streamed) === JSON.stringify(nonStreamed))Cause 4 of 5
The model refused, so the reply is prose
1Check
Both providers return a refusal as a successful response whose content need not follow your schema; Anthropic states that the refusal message takes precedence over it. Check for a refusal before parsing.
// OpenAI Responses API: a refusal arrives as its own content part
const refusal = res.output
.flatMap((item) => (item.type === 'message' ? item.content : []))
.find((part): part is OpenAI.Responses.ResponseOutputRefusal => part.type === 'refusal')
// Anthropic Messages API
const refused = msg.stop_reason === 'refusal'2Fix
Branch on the refusal before parsing: log it with the input id, show the user a fallback, and do not retry the same input in a loop. Anthropic notes that a schema field asking for step-by-step reasoning can itself trigger a refusal; ask for a short explanation instead.
if (refusal || refused) {
logger.warn({ inputId, reason: refusal?.refusal ?? 'stop_reason=refusal' }, 'model refused')
return { ok: false as const, reason: 'refused' }
}
return { ok: true as const, data: Invoice.parse(JSON.parse(text)) }3Verify
Feed a known-refused input from your logs: the call returns the fallback, and no SyntaxError reaches error tracking.
grep -c 'SyntaxError' error.logCause 5 of 5
The JSON parses, but its shape is wrong
1Check
Validate the parsed value instead of trusting it. This is common with JSON mode, with providers or models that have no schema mode, and with hand-written repair. Anthropic's structured outputs also do not guarantee the casing of enum values.
const result = Ticket.safeParse(JSON.parse(text))
if (!result.success) console.log(result.error.issues)2Fix
Validate every reply with a schema library. When validation fails, retry once with the validation errors added to the prompt, then give up. For models without a schema mode, strip fences and cut to the outermost braces before parsing; that repair is a fallback, not a substitute for constrained output.
import { z } from 'zod'
const Ticket = z.object({
category: z.preprocess(
(v) => (typeof v === 'string' ? v.toLowerCase() : v), // tolerate 'Billing' vs 'billing'
z.enum(['billing', 'bug', 'other']),
),
summary: z.string().max(200),
})
function extractJson(text: string): unknown {
const unfenced = text.replace(/^\s*```(?:json)?\s*/i, '').replace(/\s*```\s*$/, '')
const start = unfenced.indexOf('{')
const end = unfenced.lastIndexOf('}')
if (start === -1 || end <= start) throw new Error('no JSON object in model output')
return JSON.parse(unfenced.slice(start, end + 1))
}
export async function getTicket(ask: (prompt: string) => Promise<string>, input: string) {
let prompt = input
for (let attempt = 1; attempt <= 2; attempt++) {
let data: unknown
try { data = extractJson(await ask(prompt)) } catch { data = undefined }
const result = Ticket.safeParse(data)
if (result.success) return result.data
const problems = result.error.issues.map((i) => `${i.path.join('.')}: ${i.message}`).join('; ')
prompt = `${input}\n\nYour previous reply was rejected (${problems}). Reply with the JSON object only.`
}
throw new Error('no valid ticket after 2 attempts')
}3Verify
Unit-test the parser with failing replies saved from production (fenced, prefaced, wrong casing, missing field); each should either parse to a valid object or fail with a clear error.
npx vitest run src/llm/ticket.test.tsHow do I stop it coming back?
- Use the provider's schema-constrained mode wherever it exists, and keep schemas small and flat; very complex schemas are rejected or slow to compile.
- Log the stop reason and output token count of every structured call, and alert when replies approach the limit.
- Save a handful of real failing replies as test fixtures for the parser.
- Never pass unvalidated model output to database writes, payments or other code that trusts its shape.
Sources
How was this page made?
I drafted this page with AI assistance from the official documents listed under Sources. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.
Which stack is behind this page?
Projects of mine built on the same technologies:
Related fixes
AI
The OpenAI API answers 429: out of credit, or just too fast?
"type": "insufficient_quota"
A 429 from the OpenAI API means one of two unrelated things.
5 causes, each with a fixAI
The OpenAI Node client in Next.js cannot find OPENAI_API_KEY
either provide it, or instantiate the OpenAI client with an apiKey option
The OpenAI client reads process.env.OPENAI_API_KEY when you create it.
3 causes, each with a fixAI
OpenAI API 400: image_url is only supported by certain models
BadRequestError: 400 Invalid content type. image_url is only supported by certain models.
The request carried an image part, but the model you named cannot take image input.
2 causes, each with a fixAI
Azure OpenAI says the API deployment for this resource does not exist
The API deployment for this resource does not exist
Azure OpenAI does not take a model name in the model field.
2 causes, each with a fixAI
Azure OpenAI fails with an SSL certificate verification error
httpx.ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED]
The client checks the endpoint's certificate against its CA bundle, which is certifi by default.
2 causes, each with a fixAI
Gemini API says the model is not found for gemini-1.5-pro or gemini-1.5-flash
models/gemini-1.5-flash is not found for API version v1beta
Google shut down gemini-1.5-pro, gemini-1.5-flash and gemini-1.5-flash-8b on 29 September 2025, so the API no longer finds them, whatever access you have.
3 causes, each with a fix