SafetensorError: Error while deserializing header: HeaderTooLarge
safetensors_rust.SafetensorError: Error while deserializing header: HeaderTooLargeShort answer
A safetensors file starts with 8 bytes that give the size of a JSON header, and the library caps that header at 100MB. HeaderTooLarge means the first bytes are not a valid size, so the file is not real weights. It is usually a small Git LFS pointer text file or a damaged download. Look at the file, then download it again.
Fix it fast
1Find out which cause you have
Look at the size and the first bytes of the file. A real weights file is many megabytes; a pointer is a few lines of text.
ls -l model.safetensors head -c 200 model.safetensors2Match what you see, then jump to its fix
What does the error look like?
safetensors_rust.SafetensorError: Error while deserializing header: HeaderTooLarge
Cause 1 of 2
The file is a Git LFS pointer, not the weights
1Check
The Git LFS spec says a pointer is a UTF-8 text file under 1024 bytes with the keys version, oid and size. The safetensors format reads the first 8 bytes as the header size, so text there gives an impossible size, far above the 100MB header cap.
head -n 3 model.safetensors2Fix
Download the real file with the Hugging Face CLI. It prints the local path of the file.
hf download your-user/your-model model.safetensors3Verify
The file is now large and opens.
# use the path that hf download printed
ls -lh PATH_FROM_HF_DOWNLOAD
python -c "from safetensors import safe_open; f = safe_open('PATH_FROM_HF_DOWNLOAD', framework='pt'); print(len(list(f.keys())))"Confirmed by raw.githubusercontent.com
Cause 2 of 2
The download is damaged or is not a safetensors file
1Check
The safetensors README states a 100MB limit on the header, kept to stop parsing of extremely large JSON. A cut-off or wrong file fails this check the same way.
file model.safetensors2Fix
Force a fresh download so the cached copy is not reused.
from huggingface_hub import hf_hub_download
path = hf_hub_download(repo_id="your-user/your-model", filename="model.safetensors", force_download=True)
print(path)3Verify
Open the file with safe_open and list its tensors.
python -c "from safetensors import safe_open; print(len(list(safe_open('PATH_FROM_ABOVE', framework='pt').keys())))"Confirmed by huggingface.co
How do I stop it coming back?
- Download model files with the hf command or huggingface_hub, which handle large files.
- After copying a model between machines, compare file sizes with the Hub page.
- If you use git for models, install the large-file extension the Hub names before cloning.
Sources
How was this page made?
I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.
Which stack is behind this page?
Projects of mine built on the same technologies:
- stegnor.comMy company site: 804 pages from a Python and Jinja generator, with 15 free browser tools.
- MenuloA restaurant discovery platform in Germany, kept current by a Python ingestion pipeline.
- ShopGoAn e-commerce aggregator with a Python scraping engine behind its catalog, past 10K Google Play downloads.
Related fixes
AI
from_pretrained cannot find model.safetensors in the directory
OSError: Error no file named model.safetensors found in directory
from_pretrained loads weights from the folder you give it.
3 causes, each with a fixAI
Azure OpenAI says the API deployment for this resource does not exist
The API deployment for this resource does not exist
Azure OpenAI does not take a model name in the model field.
2 causes, each with a fixAI
Azure OpenAI fails with an SSL certificate verification error
httpx.ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED]
The client checks the endpoint's certificate against its CA bundle, which is certifi by default.
2 causes, each with a fixAI
LangChain says it could not import the chromadb package
ImportError: Could not import chromadb python package. Please install it with `pip install chromadb`.
The message comes from LangChain's old Chroma class, which cannot import chromadb in the Python that runs your code.
2 causes, each with a fixAI
Gemini API returns 429 RESOURCE_EXHAUSTED on the free tier
429 RESOURCE_EXHAUSTED
Gemini limits are applied per Google Cloud project, not per API key, so a new key in the same project does not reset anything.
3 causes, each with a fixAI
ChatPromptTemplate is missing the variable agent_scratchpad
Expected: ['agent_scratchpad', 'input']
A tool-calling agent keeps its tool calls and tool results in a prompt slot named agent_scratchpad.
2 causes, each with a fix