Maaz Zindani

Loading the index…

↑↓ move↵ openesc close4072 pages
Node.js, PHP and APIs

Mongoose saves duplicate documents although the field is unique

Errorunique: true in the schema, but two documents with the same value are saved

Short answer

The unique option in a Mongoose schema is not a validator. It asks MongoDB to build a unique index, and only that index rejects duplicates. Duplicates get in when the index was never built, was still building, or could not be built because the data already held duplicates. Check the real indexes in MongoDB, remove duplicates, then build.

  • Applies to Mongoose 8, MongoDB 7, Node.js 22
  • Updated
  • By Maaz Zindani, Senior Software Engineer

Fix it fast

  1. 1Find out which cause you have

    List the indexes MongoDB really has on the collection. If there is no unique index on your field, the schema option has not produced one.

    const indexes = await User.collection.indexes();
    console.log(indexes);
  2. 2Match what you see, then jump to its fix

    1. 1No duplicate error, and the index list shows no unique index yet, or it appears laterThe index has not been built when the writes happenFix
    2. 2The unique index is missing although the schema has it, and the Mongoose index event carries an errorThe collection already holds duplicates, so the index cannot be createdFix
    3. 3Indexes never appear in a deployed environment, but do appear locallyautoIndex is switched off for the schema or the appFix
    4. 4You added unique later, or changed the field, and the old non-unique index is still thereThe index in MongoDB does not match the schema after a changeFix

What does the error look like?

  • unique: true in the schema, but two documents with the same value are saved
  • duplicate key error, shown only after the index exists

Cause 1 of 4

The index has not been built when the writes happen

1Check

Mongoose builds schema indexes after connecting, in the background. If MongoDB has no unique index on the field yet, a duplicate write does not error.

const indexes = await User.collection.indexes();
console.log(indexes.filter((i) => i.unique));

2Fix

Wait for the index build before you write. Model.init() returns a promise that resolves when the indexes are finished. The Mongoose FAQ shows the same pattern in tests.

await User.init();
await User.create([{ email: 'a@example.com' }, { email: 'a@example.com' }]); // now throws a duplicate key error

3Verify

The second document is rejected with a duplicate key error.

node test-unique.mjs

Confirmed by mongoosejs.com

Did this fix it?

Cause 2 of 4

The collection already holds duplicates, so the index cannot be created

1Check

MongoDB cannot build a unique index when existing data already breaks the constraint. Several documents with a missing or null value also break it, because a missing field counts as null and only one such document is allowed.

db.users.aggregate([
  { $group: { _id: "$email", n: { $sum: 1 } } },
  { $match: { n: { $gt: 1 } } }
])

2Fix

Remove or merge the duplicates, then build the index. Make sure the field is always set, because documents with a missing value all count as null.

// after cleaning the data
await User.createIndexes();

3Verify

The index list now shows unique: true on the field.

const idx = await User.collection.indexes();
console.log(idx.find((i) => i.key.email));

Confirmed by mongodb.com

Did this fix it?

Cause 3 of 4

autoIndex is switched off for the schema or the app

1Check

The Mongoose guide says automatic index creation can be disabled, and recommends doing so in production because index builds can cost performance.

grep -rn "autoIndex" src/

2Fix

When autoIndex is off, build the indexes on purpose: call Model.createIndexes() in a deploy step, or create them in the MongoDB shell. Do not leave the app without them.

// one-off deploy script
await mongoose.connect(process.env.MONGODB_URI);
await User.createIndexes();
await mongoose.disconnect();

3Verify

List the indexes on the deployed database.

mongosh "$MONGODB_URI" --eval "db.users.getIndexes()"

Confirmed by mongoosejs.com

Did this fix it?

Cause 4 of 4

The index in MongoDB does not match the schema after a change

1Check

Mongoose builds the indexes declared in the schema. An existing index on the same field with different options is not the one you wanted, so compare.

console.log(await User.diffIndexes());

2Fix

Use syncIndexes to make MongoDB match the schema. It drops indexes that are not in the schema, except _id, so run diffIndexes first and avoid it on a busy production system.

await User.syncIndexes();

3Verify

diffIndexes reports nothing left to change.

console.log(await User.diffIndexes());

Confirmed by mongoosejs.com

Did this fix it?

How do I stop it coming back?

  • Treat a Mongoose unique option as a request for an index, and test it with duplicate writes after Model.init().
  • Create production indexes in a deploy step or the MongoDB shell, not by relying on app start.
  • Clean data before adding a unique constraint to a table that already has rows.
  • Catch the duplicate key error in your code and return a clear message to the user.

Sources

How was this page made?

I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.

Which stack is behind this page?

Projects of mine built on the same technologies:

Related fixes

Back end

ng serve says it cannot determine the project or target

Cannot determine project for command.

The Angular CLI looked through angular.json for a project that defines the target you asked for (serve, build, test) and found none, or found several and could not pick one.

3 causes, each with a fix

Back end

ng serve cannot find the dev-server builder package

Could not find the implementation for builder @angular/build:dev-server

The builder named in angular.json (for example @angular-devkit/build-angular:dev-server) is looked up as an npm package starting from the workspace root.

3 causes, each with a fix

Next step

Want this handled for good?

I look after servers, apps and releases end to end. Tell me what keeps breaking.