4

I was creating a website using NextJS and for authentication uses NextAuth. And the database is on free tier in MongoDB Atlas.

I have two versions of code for database connection. One is this:

/**
 *      MongoDB Connection
 * 
 *  */
import mongoose from 'mongoose'

const MONGODB_URI = process.env.MONGODB_URL

if (! process.env.MONGODB_URL) {
  throw new Error(
    'Please define the MONGODB_URI environment variable inside .env.local'
  )
}

/**
 * Global is used here to maintain a cached connection across hot reloads
 * in development. This prevents connections growing exponentially
 * during API Route usage.
 */
let cached = global.mongoose

if (!cached) {
  cached = global.mongoose = { conn: null, promise: null }
}

async function dbConnect() {
  if (cached.conn) {
    return cached.conn
  }

  if (!cached.promise) {
    const opts = {
      useNewUrlParser: true,
      useUnifiedTopology: true,
      bufferCommands: false,
    //   bufferMaxEntries: 0,
    //   useFindAndModify: false,
    //   useCreateIndex: true,
    }

    cached.promise = mongoose.connect(process.env.MONGODB_URL, opts).then((mongoose) => {
      return mongoose
    })
  }
  cached.conn = await cached.promise
  return cached.conn
}

export default dbConnect

So, before making any DB related queries via code, I call await dbConnect(). It is working fine.

But for storing the sessions in DB, in NextAuth, I was not able to use the above function. So for that, am using this custom code (/lib/mongodb.js):

/**
 * 
 *      Used only for Next-Auth
 * 
 */

import { MongoClient } from "mongodb"

const uri = process.env.MONGODB_URL
const options = {
  useUnifiedTopology: true,
  useNewUrlParser: true,
}

let client
let clientPromise

if (!process.env.MONGODB_URL) {
  throw new Error("Please add your Mongo URI to .env.local")
}

if (process.env.NODE_ENV === "development") {
  // In development mode, use a global variable so that the value
  // is preserved across module reloads caused by HMR (Hot Module Replacement).
  if (!global._mongoClientPromise) {
    client = new MongoClient(uri, options)
    global._mongoClientPromise = client.connect()
  }
  clientPromise = global._mongoClientPromise
} else {
  // In production mode, it's best to not use a global variable.
  client = new MongoClient(uri, options)
  clientPromise = client.connect()
}

// Export a module-scoped MongoClient promise. By doing this in a
// separate module, the client can be shared across functions.
export default clientPromise

And the code in my /pages/api/auth/[...nextauth].js is like this:

import NextAuth from 'next-auth'
import { MongoDBAdapter } from "@next-auth/mongodb-adapter"
import mongodb from '../../../lib/mongodb'
//...

export default async function auth(req, res) {
    return await NextAuth(req, res, {    
        //....

        adapter: MongoDBAdapter({
            db: (await mongodb).db("my_db_name_here")            
        }),

        //....
    })
}

Here's the packages am using:

"mongodb": "^4.1.2",
"mongoose": "^6.0.1",    
"next": "11.0.1",
"next-auth": "^4.0.0-beta.6",
"react": "17.0.2",
"react-dom": "17.0.2",

The problem is, am receiving email notifications sometimes like the following:

enter image description here

My website is still in testing phase(tested by two persons only) and is hosting in Vercel server. I believe this could be because NextAuth is creating new db connections each time? Any thoughts on what went wrong?

Akhilesh B Chandran
  • 6,523
  • 7
  • 28
  • 55

1 Answers1

3

clientPromise in next-auth is local, you create new client and 5 connections every time. Just use global.mongoose.conn.

The docs for MongoDBAdapter says it needs a promise that resolves to a client, so it must be something like this:

export default NextAuth({
  adapter: MongoDBAdapter(dbConnect().then(mon => mon.connection.getClient())),
  ...
})

In you case you seem to use db. I couldn't find any references for MongoDBAdapter to accept something liek {db: ...} but you can get the db from mongoose as following:

await (dbConnect().then(mon => mon.connection.getClient().db("my_db_name_here")))

Or without parameter to use the same database as configured in mongoose connection.

UPDATE

The issue with number of connections from Vercel is covered in Vercel creates new DB connection for every request

Alex Blex
  • 34,704
  • 7
  • 48
  • 75
  • I tried using the second solution(passing db) you provided, but got the error in console saying `conn.getClient is not a function` – Akhilesh B Chandran Jun 06 '22 at 17:42
  • When I used the first solution, it thrown this error : `TypeError: Cannot read property 'collection' of undefined. UnhandledPromiseRejectionWarning: TypeError: conn.getClient is not a function` – Akhilesh B Chandran Jun 06 '22 at 17:44
  • @AkhileshBChandran, sorry, missed the "connection" bit. `global.mongoose.conn` in your snippet is not a connection as the name may suggest, but the whole mongoose, so it should be `conn.connection.getClient()` instead of `conn.getClient()`. I updated the answer – Alex Blex Jun 07 '22 at 10:40
  • Thanks. First solution was throwing `TypeError: Cannot read property 'collection' of undefined`. But the second option is not throwing any errors and seems to be working fine. But how would I double check that? I mean is there a way to inspect through console log or something regarding how many connections are there? – Akhilesh B Chandran Jun 08 '22 at 08:57
  • You can check number of connections in Atlas – Alex Blex Jun 08 '22 at 09:00
  • Thanks. But that will display the count of all connections that includes the live website's also. I was just wondering whether there's any way so that I could test from my development machine, like accessing it multiple times from different browsers from the same PC and stuff like that, and check whether the number of connections are not exceeding. Anyway, I will monitor the connections in Atlas. – Akhilesh B Chandran Jun 08 '22 at 09:25
  • Another question is, you mentioned that `clientPromise in next-auth is local, you create new client and 5 connections every time`. You are talking about the second code (/lib/mongodb.js) isn't it? But are there also we are storing it globally and re-using? Sorry I was a bit confused. Would you be kind enough to share more details about it. Thanks – Akhilesh B Chandran Jun 08 '22 at 09:27
  • Btw, I just got multiple email notifications from MongoAtlas : `connections to your cluster(s) have exceeded 500` after pushing it to the server – Akhilesh B Chandran Jun 08 '22 at 09:31
  • Screenshot: https://prnt.sc/C5Sr2mws5iWX – Akhilesh B Chandran Jun 08 '22 at 09:33
  • 1
    In the second snippet you re-use the global promise only when `process.env.NODE_ENV === "development"` The connections chart doesn't look good. Are you sure it is the only place where you connect to atlas, there is only one nodejs server, and the connection pool is default 5 connections per client? I'm afraid the resolution will require a bit of debugging to identify the root cause first. – Alex Blex Jun 08 '22 at 09:47
  • Thanks. Didn't noticed that condition! Regarding the spike in connections, I tried monitoring and found that it was occurring every 30mins. Since I was running a CRON(it pings a particular API endpoint in my app), checked the code and noticed that am only using `updateMany()` of Mongoose. Then added indexes for the two fields which it was using as filter. So far didn't saw the spike. And the application is hosted in Vercel server, and no connection pool number was explicitly passed. – Akhilesh B Chandran Jun 08 '22 at 10:44
  • oh, you should mention it's Vercel from the beginning. Memoising connection in global space won't help in serverless environment, as each HTTP request is processed in isolation. See https://stackoverflow.com/questions/63208960/vercel-creates-new-db-connection-for-every-request There is not much you can do about it. Try to reduce connection pool to 2 connections as a tactical measure. In a long term - upgrade Atlas to dedicated M10 which has no 500 connection limitation, consider serverless Atlas subscription, or Atlas data-api. – Alex Blex Jun 08 '22 at 11:10
  • Thanks. Yeah I forgot to mention it in the first place as I thought people will prefer Vercel for hosting their NextJS projects mostly. I will try reducing the connection pool as you suggested. And also when I checked the limits, Mongo's website says there is a limit of `1500` for M10 : https://www.mongodb.com/docs/atlas/reference/atlas-limits/ – Akhilesh B Chandran Jun 13 '22 at 05:55
  • interesting. I was sure dedicated instances are... well, dedicated to a user, like all 65k connections that the OS can provide. I wonder where this limitations come from. I'd reach out to support to discuss options. They explicitly ask to do this at the top of the page, and right below the connection section. Seems like they are trying to solve some internal issues by introducing these limits but are not quite sure how it hit sales. Regarding Vercel - they position themselves as a frontend app hosting at the first place. Databases are supported, and atlas is just one of the options. – Alex Blex Jun 13 '22 at 08:47
  • sorry for the delay. Am thinking about separating the API part to a different NodeJS+Express application. But am a bit confused regarding how to use the authentication. Like at the moment, am using NextAuth with OneLogin and the sessions are stored in MongoDB, in my NextJS app. So when I moved the API logic to serverful app, how would I handle the authentication? If you can edit your answer and post your inputs, would be greatful. And for API calls, should I call the NextJS app's API endpoint, which would internally call my serverful API endpoints? Or I should directly call ? – Akhilesh B Chandran Jul 06 '22 at 22:42
  • wow, that's what I call one's post raised more questions than answers. Sorry Akhilesh, it's so much out of scope of original question. You are now asking about solution architecture and format SO is not quite suitable for that. I would recommend to do some research, pick a solution, and ask new questions if you have any issues with implementation. I believe the question about connectivity issues of Vercel is answered and there is no need to update my answer. – Alex Blex Jul 07 '22 at 12:20