Semaphore ConnectionThrottlingPipeline for MongoDB c#

Question

I am trying to Implement a pipeline to limit and execute all incoming request from API to mongoDB using this reference. I Have Implemented as Follows,

Singleton.cs

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using System.Threading.Tasks;
    using System.Threading;
    using Orniz.OE.BusinessObjects;
    using MongoDB.Driver;
    using System.Diagnostics;
    using System.Configuration;
    
    namespace DataAccess.Implementations
{
    public class Singleton
    {
        private Singleton() { }

        private static Singleton _instance;

        private static string DBName = DBConnection.Database;
        private static string connectionString = DBConnection.ConnectionString;
        public static string PoolSize { get { return ConfigurationManager.AppSettings["OnlinePoolSize"].ToString(); } }

        public IMongoDatabase db = null;

        private static readonly object _lock = new object();

        public static Singleton GetInstance()
        {
            if (_instance == null)
            {
                lock (_lock)
                {
                    if (_instance == null)
                    {
                        try
                        {
                            _instance = new OnlineExamSingleton();
                            var mongo_url = MongoUrl.Create(connectionString);
                            MongoClientSettings settings = MongoClientSettings.FromUrl(mongo_url);
                            settings.MaxConnectionPoolSize = Convert.ToInt32(PoolSize);
                            _instance.client = new MongoClient(settings);
                            _instance.db = _instance.client.GetDatabase(DBName);

                        }
                        catch (Exception ex)
                        {

                        }
                    }
                }
            }
            return _instance;
        }



        public static int MaxConnectionPoolSize
        {
            get
            {
                int defaultValue = 200;
                string configValue = PoolSize;
                if (string.IsNullOrEmpty(configValue))
                {
                    return defaultValue;
                }
                else
                {
                    int returnValue;
                    try
                    {
                        int configExpiryValue = Convert.ToInt32(configValue);
                        returnValue = configExpiryValue;
                    }
                    catch (Exception ex)
                    {
                        returnValue = defaultValue;
                    }

                    return returnValue;
                }
            }
        }
        public MongoClient client { get; set; }
        public IMongoCollection<TDocument> Collection<TDocument>(string collection)
       where TDocument : class
        {
            return db.GetCollection<TDocument>(collection);
        }
    }
}

**ConnectionThrottlingPipeline.cs * *

    using MongoDB.Driver;
    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using System.Threading;
    using System.Threading.Tasks;
    
    namespace DataAccess.Implementations
{
    public class ConnectionThrottlingPipeline<T> : IConnectionThrottlingPipeline<T>
    {
        private readonly Semaphore openConnectionSemaphore;

        public ConnectionThrottlingPipeline(IMongoClient client)
        {
            //Only grabbing half the available connections to hedge against collisions.
            //If you send every operation through here
            //you should be able to use the entire connection pool.
            openConnectionSemaphore = new Semaphore(client.Settings.MaxConnectionPoolSize / 2,
                client.Settings.MaxConnectionPoolSize / 2);
        }

        public async Task AddRequest(Func<Task> task)
        {
            try
            {
                openConnectionSemaphore.WaitOne();
                try
                {
                    await task();
                }
                finally
                {
                    openConnectionSemaphore.Release();
                }
            }
            catch (Exception ex)
            {

            }


        }
    }
}

**IConnectionThrottlingPipeline.cs * *

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using System.Threading.Tasks;
    
    namespace DataAccess.Implementations
{
    public class IConnectionThrottlingPipeline<T>
    {
    }
}

**methodclass.cs * *

public static Singleton db = Singleton.GetInstance();
public async Task<bool> InsertData(Data data)
{
    try
    {
        ConnectionThrottlingPipeline<Task> connectionThrottlingPipeline = new ConnectionThrottlingPipeline<Task>(Singleton.GetInstance().client);
        var mycollection = db.Collection<Data>("mycollection");
        await connectionThrottlingPipeline.AddRequest(() => mycollection.InsertOneAsync(data));

    }
    catch (Exception exception)
    {

    }
}

It 's giving me the below error upon testing with 4000 concurrent document write requests with 300 as default PoolSize, The exception occurs on Add request method. I wonder should I need to Implemet the class as Singleton too or is there something wrong in my Implementation ? can someone give me some insight on this ?

The wait queue for acquiring a connection to server myserver-shardtag14.mongodb.net:27017 is full.

at MongoDB.Driver.Core.ConnectionPools.ExclusiveConnectionPool.AcquireConnectionHelper.EnterWaitQueue() at MongoDB.Driver.Core.ConnectionPools.ExclusiveConnectionPool.d__38.MoveNext() at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at MongoDB.Driver.Core.Servers.Server.d__34.MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at MongoDB.Driver.Core.Operations.RetryableWriteContext.d__21.MoveNext() at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at MongoDB.Driver.Core.Operations.RetryableWriteContext.d__1.MoveNext() at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at MongoDB.Driver.Core.Operations.BulkMixedWriteOperation.d__44.MoveNext() at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at MongoDB.Driver.OperationExecutor.d__51.MoveNext() at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at MongoDB.Driver.MongoCollectionImpl1.d__1001.MoveNext() at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at MongoDB.Driver.MongoCollectionImpl1.d__30.MoveNext() at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at MongoDB.Driver.MongoCollectionImpl1.<UsingImplicitSessionAsync>d__1061.MoveNext() at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at MongoDB.Driver.MongoCollectionBase`1.d__74.MoveNext() at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.GetResult()

`db.Collection("mycollection").InsertOneAsync(data)` returns `Task`, not `Task`, create an overload method for `AddRequest` which will take as a parameter `Task task` . — Alexandra Petrova, Oct 04 '21 at 10:50
Just add ```public async Task AddRequest(Task task) { openConnectionSemaphore.WaitOne(); try { await task; } finally { openConnectionSemaphore.Release(); } } ``` — Alexandra Petrova, Oct 04 '21 at 10:54

JCompetence · Answer 1 · 2021-10-04T10:54:41.473

1

I believe you need to add <T> to the top level of the class,

public class IConnectionThrottlingPipeline<T>
{
}

ConnectionThrottlingPipeline<T> : IConnectionThrottlingPipeline<T>

== Edited ==

public async T AddRequest<T>(T task)

ConnectionThrottlingPipeline<Task> connectionThrottlingPipeline = new ConnectionThrottlingPipeline(db.client);
                 connectionThrottlingPipeline.AddRequest(db.Collection<Data>("mycollection").InsertOneAsync(data));

edited Oct 04 '21 at 10:54

answered Oct 04 '21 at 09:55

JCompetence

6,997
3
19
26

I tried the first one it is showing a warning parameter T has the same name as the outer type ConnectionThrottlingPipeline and the error is not going away. As for the second one I already tried that and it is showing error cannot convert from Task to Task @Susan Mustafa – Aneesh Oct 04 '21 at 10:34
@Aneesh Please try adding to the inherited class as well – JCompetence Oct 04 '21 at 10:40
Tried that it is still showing the same error although no new error has occured when I added the code you suggested @Susan Mustafa – Aneesh Oct 04 '21 at 10:49
kindly see edit @Aneesh – JCompetence Oct 04 '21 at 10:57
1

if you use `public async T AddRequest(T task)` you won't be able to await the task. And also, now there is nothing async in the method, so you need to remove the `async` keyword. – Alexandra Petrova Oct 04 '21 at 10:58
Thanks the error got solved, but still the issue I was facing The wait queue for acquiring a connection to server mongodb:27017 is full error is not going away I think there is something wrong with my Implementation @Alexandra Petrova , Susan Mustafa – Aneesh Oct 04 '21 at 11:42
@Aneesh can you update the question with more info about the error you are talking about? – Alexandra Petrova Oct 04 '21 at 12:38
Updated feel free to take a look @Alexandra Petrova – Aneesh Oct 04 '21 at 14:38

Semaphore ConnectionThrottlingPipeline for MongoDB c#

1 Answers1