Maaz Zindani

Loading the index…

↑↓ move↵ openesc close4072 pages
AI and LLM integration

from_pretrained cannot find model.safetensors in the directory

ErrorOSError: Error no file named model.safetensors found in directory

Short answer

from_pretrained loads weights from the folder you give it. It fails when that folder holds the wrong kind of file. A GGUF download needs the gguf_file argument, a sharded checkpoint needs its index file and every shard, and a path one level too high or low holds no weights at all. List the folder first.

  • Applies to transformers 4.x and 5, Python 3.12
  • Updated
  • By Maaz Zindani, Senior Software Engineer

Fix it fast

  1. 1Find out which cause you have

    List what is actually in the folder. The file names decide which cause applies.

    ls -la ./my-model
  2. 2Match what you see, then jump to its fix

    1. 1ls shows one *.gguf file and no config.jsonThe folder holds a GGUF file, not safetensorsFix
    2. 2ls shows model-00001-of-0000N.safetensors files, but the index file or some shards are missingA sharded checkpoint is incompleteFix
    3. 3ls shows only sub-folders such as snapshots or the model nameThe path points at the wrong folderFix

What does the error look like?

  • OSError: Error no file named model.safetensors found in directory

Cause 1 of 3

The folder holds a GGUF file, not safetensors

1Check

A GGUF checkpoint is a single file with weights, config and tokenizer. There is no model.safetensors to find.

ls ./my-model | grep -i gguf

2Fix

Pass the folder or repo together with the file name through gguf_file.

from transformers import AutoModelForCausalLM, AutoTokenizer
path = "./my-model"
name = "model-Q4_K_M.gguf"
model = AutoModelForCausalLM.from_pretrained(path, gguf_file=name)
tok = AutoTokenizer.from_pretrained(path, gguf_file=name)

3Verify

The model loads without the OSError.

print(type(model))

Confirmed by huggingface.co

Did this fix it?

Cause 2 of 3

A sharded checkpoint is incomplete

1Check

A large model is saved in shards plus model.safetensors.index.json, which maps each parameter to its shard file. A partial download breaks that map.

ls ./my-model | grep -E 'safetensors'
python -c "import json; print(len(set(json.load(open('./my-model/model.safetensors.index.json'))['weight_map'].values())))"

2Fix

Load the model again from its Hub repo id so every file is fetched, and save a fresh copy if you need a local folder.

from transformers import AutoModelForCausalLM
# load by repo id so every file is fetched, then save a complete copy
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf")
model.save_pretrained("./my-model")

3Verify

The number of shard files in the folder equals the count the index prints.

ls ./my-model/*.safetensors | wc -l

Confirmed by huggingface.co

Did this fix it?

Cause 3 of 3

The path points at the wrong folder

1Check

from_pretrained takes a Hub repo id or a local directory that directly contains the weights and config.json.

find ./my-model -maxdepth 3 -name config.json

2Fix

Point from_pretrained at the folder that contains config.json, not its parent.

from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("./my-model/the-folder-with-config")

3Verify

The load completes.

print(model.config.model_type)

Confirmed by huggingface.co

Did this fix it?

How do I stop it coming back?

  • Fetch models with from_pretrained on the repo id, not by downloading single files by hand.
  • Keep one model per folder and check for config.json before a long job starts.
  • Do not hand-rename weight files; the index file refers to shards by name.

Sources

How was this page made?

I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.

Which stack is behind this page?

Projects of mine built on the same technologies:

Related fixes

AI

LangChain says it could not import the chromadb package

ImportError: Could not import chromadb python package. Please install it with `pip install chromadb`.

The message comes from LangChain's old Chroma class, which cannot import chromadb in the Python that runs your code.

2 causes, each with a fix

Next step

Want this handled for good?

I look after servers, apps and releases end to end. Tell me what keeps breaking.