Maaz Zindani

Loading the index…

↑↓ move↵ openesc close4072 pages
AI and LLM integration

The model's reply breaks JSON.parse

ErrorSyntaxError: Unexpected token '`', "```json ..." is not valid JSON

Short answer

JSON.parse rejects model output for a few repeatable reasons: the JSON sits inside markdown fences or next to a sentence, the reply stopped at the token limit mid-object, a stream was parsed before it ended, or the model refused. Ask for schema-constrained output (OpenAI Structured Outputs, Anthropic structured outputs), check why generation stopped before parsing, and validate the result with a schema.

  • Applies to openai Node.js SDK 7, @anthropic-ai/sdk 1, zod 4, Node.js 22 and 24
  • Updated
  • By Maaz Zindani, Senior Software Engineer

Fix it fast

  1. 1Find out which cause you have

    Before changing the prompt, log one failing call: why generation stopped, and the first and last characters of the raw text. Fences or a polite sentence show up at the start or the end; a reply that was cut off ends in the middle of a value and comes with a token-limit stop reason.

    console.log(JSON.stringify({
      stopped: res.status ?? res.stop_reason,     // OpenAI Responses: status; Anthropic Messages: stop_reason
      why: res.incomplete_details?.reason,        // OpenAI only, e.g. 'max_output_tokens'
      head: text.slice(0, 80),
      tail: text.slice(-80),
    }))
  2. 2Match what you see, then jump to its fix

    1. 1Unexpected token '`' (a fence), Unexpected token 'S' or 'H' (a preface such as Sure, here is...), or Unexpected non-whitespace character after JSON (a closing remark).The JSON comes wrapped in code fences or a sentenceFix
    2. 2Unterminated string in JSON, Unexpected end of JSON input, or Expected ',' or ']' after array element, mostly on long inputs or long lists.The reply hit the token limit and stopped mid-objectFix
    3. 3The errors appear only with stream: true, the position in the message is small, and JSON.parse sits inside the loop that reads stream events.A streamed reply is parsed before it has finishedFix
    4. 4Unexpected token 'I', "I can't he"... and similar; with Anthropic structured outputs the response has HTTP 200 and stop_reason refusal; with OpenAI the output contains a refusal item instead of text.The model refused, so the reply is proseFix
    5. 5No SyntaxError, but later code throws TypeError: Cannot read properties of undefined, or an enum value differs by capitalisation.The JSON parses, but its shape is wrongFix

What does the error look like?

  • SyntaxError: Unexpected token '`', "```json ..." is not valid JSON
  • SyntaxError: Unterminated string in JSON at position 1874
  • SyntaxError: Unexpected end of JSON input
  • SyntaxError: Unexpected non-whitespace character after JSON at position 212
  • SyntaxError: Unexpected token 'I', "I can't he"... is not valid JSON

Cause 1 of 5

The JSON comes wrapped in code fences or a sentence

1Check

The head and tail in the log show the fence or the extra words. This happens when JSON is only requested in the prompt: nothing stops the model from formatting the answer the way chat replies usually look. OpenAI's older JSON mode guarantees valid JSON but not your schema, and it errors unless the word JSON appears in the input.

const looksFenced = /^\s*```/.test(text) || /```\s*$/.test(text)
const hasPreface = !/^\s*[[{]/.test(text)
console.log({ looksFenced, hasPreface })

2Fix

Let the API enforce the shape. OpenAI Structured Outputs (a json_schema text format with strict mode) and Anthropic structured outputs (output_config.format) constrain generation to your JSON Schema, and both Node.js SDKs accept a Zod schema and return a parsed object. Under strict OpenAI schemas every field must be required; express optional values as nullable.

import { z } from 'zod'
import OpenAI from 'openai'
import { zodTextFormat } from 'openai/helpers/zod'
import Anthropic from '@anthropic-ai/sdk'
import { zodOutputFormat } from '@anthropic-ai/sdk/helpers/zod'

const Invoice = z.object({
  vendor: z.string(),
  total: z.number(),
  currency: z.string(),
  due_date: z.string().nullable(),   // optional in practice, but present in the output
})

// OpenAI (Responses API)
const openai = new OpenAI()
const r = await openai.responses.parse({
  model: process.env.OPENAI_MODEL!,
  input: [{ role: 'user', content: invoiceText }],
  text: { format: zodTextFormat(Invoice, 'invoice') },
})
const fromOpenAI = r.output_parsed

// Anthropic (Messages API)
const anthropic = new Anthropic()
const m = await anthropic.messages.parse({
  model: process.env.ANTHROPIC_MODEL!,
  max_tokens: 1024,
  messages: [{ role: 'user', content: invoiceText }],
  output_config: { format: zodOutputFormat(Invoice) },
})
const fromClaude = m.parsed_output

3Verify

Run the same input many times and count parse failures; with schema-constrained output it should be zero.

let failures = 0
for (let i = 0; i < 50; i++) {
  const r = await extractInvoice(sampleText)
  if (!r) failures++
}
console.log({ failures })
Did this fix it?

Cause 2 of 5

The reply hit the token limit and stopped mid-object

1Check

Look at why generation stopped. OpenAI's Responses API marks the response incomplete with the reason max_output_tokens; Anthropic sets stop_reason to max_tokens. Both providers warn that a cut-off reply may not match the schema, even in structured mode.

// OpenAI Responses API
if (res.status === 'incomplete') console.log('cut off:', res.incomplete_details?.reason)

// Anthropic Messages API
if (msg.stop_reason === 'max_tokens') console.log('cut off at', msg.usage.output_tokens, 'output tokens')

2Fix

Treat a cut-off reply as a failed call, never as text to parse or repair. Raise the output limit with headroom, ask for fewer items per call and page through the rest, and keep free-text fields short in the schema description.

function assertComplete(res: { status?: string; stop_reason?: string | null }) {
  if (res.status === 'incomplete' || res.stop_reason === 'max_tokens') {
    throw new Error('model output truncated: raise the token limit or request fewer items')
  }
}

// ask for 25 items per call instead of everything at once
for (let page = 0; page * 25 < rows.length; page++) {
  const chunk = rows.slice(page * 25, (page + 1) * 25)
  results.push(...(await classify(chunk)))
}

3Verify

Log output tokens against the limit for a week of traffic; the largest reply should stay well under the limit, and no call should end with a token-limit stop.

grep -o '"output_tokens":[0-9]*' llm.log | cut -d: -f2 | sort -n | tail -1
Did this fix it?

Cause 3 of 5

A streamed reply is parsed before it has finished

1Check

Each streamed event carries only a fragment of the text. Parsing a fragment fails, and parsing the text so far fails until the last delta arrives.

grep -n -A6 'for await' src/llm/*.ts | grep -n 'JSON.parse'

2Fix

Append the text deltas to a buffer and parse once, after the stream reports completion. If the user needs to see progress, show the raw text while it streams and swap in the parsed result at the end.

const stream = await openai.responses.create({ ...params, stream: true })
let text = ''
for await (const event of stream) {
  if (event.type === 'response.output_text.delta') text += event.delta
  if (event.type === 'response.completed') break
}
const data = Invoice.parse(JSON.parse(text))   // parse once, after the last delta

3Verify

Stream and non-stream versions of the same request return identical parsed objects.

console.log(JSON.stringify(streamed) === JSON.stringify(nonStreamed))
Did this fix it?

Cause 4 of 5

The model refused, so the reply is prose

1Check

Both providers return a refusal as a successful response whose content need not follow your schema; Anthropic states that the refusal message takes precedence over it. Check for a refusal before parsing.

// OpenAI Responses API: a refusal arrives as its own content part
const refusal = res.output
  .flatMap((item) => (item.type === 'message' ? item.content : []))
  .find((part): part is OpenAI.Responses.ResponseOutputRefusal => part.type === 'refusal')

// Anthropic Messages API
const refused = msg.stop_reason === 'refusal'

2Fix

Branch on the refusal before parsing: log it with the input id, show the user a fallback, and do not retry the same input in a loop. Anthropic notes that a schema field asking for step-by-step reasoning can itself trigger a refusal; ask for a short explanation instead.

if (refusal || refused) {
  logger.warn({ inputId, reason: refusal?.refusal ?? 'stop_reason=refusal' }, 'model refused')
  return { ok: false as const, reason: 'refused' }
}
return { ok: true as const, data: Invoice.parse(JSON.parse(text)) }

3Verify

Feed a known-refused input from your logs: the call returns the fallback, and no SyntaxError reaches error tracking.

grep -c 'SyntaxError' error.log
Did this fix it?

Cause 5 of 5

The JSON parses, but its shape is wrong

1Check

Validate the parsed value instead of trusting it. This is common with JSON mode, with providers or models that have no schema mode, and with hand-written repair. Anthropic's structured outputs also do not guarantee the casing of enum values.

const result = Ticket.safeParse(JSON.parse(text))
if (!result.success) console.log(result.error.issues)

2Fix

Validate every reply with a schema library. When validation fails, retry once with the validation errors added to the prompt, then give up. For models without a schema mode, strip fences and cut to the outermost braces before parsing; that repair is a fallback, not a substitute for constrained output.

import { z } from 'zod'

const Ticket = z.object({
  category: z.preprocess(
    (v) => (typeof v === 'string' ? v.toLowerCase() : v),   // tolerate 'Billing' vs 'billing'
    z.enum(['billing', 'bug', 'other']),
  ),
  summary: z.string().max(200),
})

function extractJson(text: string): unknown {
  const unfenced = text.replace(/^\s*```(?:json)?\s*/i, '').replace(/\s*```\s*$/, '')
  const start = unfenced.indexOf('{')
  const end = unfenced.lastIndexOf('}')
  if (start === -1 || end <= start) throw new Error('no JSON object in model output')
  return JSON.parse(unfenced.slice(start, end + 1))
}

export async function getTicket(ask: (prompt: string) => Promise<string>, input: string) {
  let prompt = input
  for (let attempt = 1; attempt <= 2; attempt++) {
    let data: unknown
    try { data = extractJson(await ask(prompt)) } catch { data = undefined }
    const result = Ticket.safeParse(data)
    if (result.success) return result.data
    const problems = result.error.issues.map((i) => `${i.path.join('.')}: ${i.message}`).join('; ')
    prompt = `${input}\n\nYour previous reply was rejected (${problems}). Reply with the JSON object only.`
  }
  throw new Error('no valid ticket after 2 attempts')
}

3Verify

Unit-test the parser with failing replies saved from production (fenced, prefaced, wrong casing, missing field); each should either parse to a valid object or fail with a clear error.

npx vitest run src/llm/ticket.test.ts
Did this fix it?

How do I stop it coming back?

  • Use the provider's schema-constrained mode wherever it exists, and keep schemas small and flat; very complex schemas are rejected or slow to compile.
  • Log the stop reason and output token count of every structured call, and alert when replies approach the limit.
  • Save a handful of real failing replies as test fixtures for the parser.
  • Never pass unvalidated model output to database writes, payments or other code that trusts its shape.

Sources

How was this page made?

I drafted this page with AI assistance from the official documents listed under Sources. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.

Which stack is behind this page?

Projects of mine built on the same technologies:

Related fixes

Next step

Want this handled for good?

I look after servers, apps and releases end to end. Tell me what keeps breaking.