c# - Using LinQ to look for duplicate items in a list and updating object properties if so

Question

I have a list Allflights of object type FLightInfo.

If an object from the list has the same flightNumber and takeofftime. I want to consolidate those entries and add their user count

Example: This is my list:

FlightNumber 123 Takeoff 12:00 Users 5
FlightNumber 256 Takeoff 3:00 Users 6
FlightNumber 123 Takeoff 12:00 Users 8
FlightNumber 651 Takeoff 5:00 Users 3

I want the output to be like:

FlightNumber 123  Takeoff 12:00 Users 13
FlightNumber 256  Takeoff 3:00  Users 6  
FlightNumber 651  Takeoff 5:00  Users 3

My source code:

struct FlightInfo
        {
            public string FlightNumber { get; set; }
            public string Takeoff_time { get; set; }
            public string Landing_time { get; set; }
            public int UserCount { get; set; }
}

static List<FlightInfo> allFlights = new List<FlightInfo>();

//I read several files using multi-threading and create a FlightInfo object   
//and add it to the allFlights list

allFlights.Add(buildFlight(FlightNumber, Origination, Destination, Takeoff_time, Landing_time, UserCount);

//This is what I need to do
//if FlightNumber && testTakeOff object attributes are the same 
// I want to consolidate those entries and add the UserCount from those entries

Can you try to check before adding? How would you do that? You could create a class that takes FlightInfo objects and then does the checking/consolidating. If you are really adding from multiple threads you should consider using a `lock` statement. Bonus: Your new class could enforce this locking! — Dennis Kuypers, Oct 22 '18 at 17:42
Can you please add some simple data and the expected output? — MakePeaceGreatAgain, Oct 22 '18 at 17:45

score 3 · Accepted Answer · answered Oct 22 '18 at 18:29

What you're looking for can be accomplished with GroupBy<TSource,TKey,TElement,TResult>(IQueryable<TSource>, Expression<Func<TSource,TKey>>, Expression<Func<TSource,TElement>>, Expression<Func<TKey,IEnumerable<TElement>,TResult>>, IEqualityComparer<TKey>)

Using your object definiton you could do something like this

var consildatedFlights = allFlights.GroupBy(x => new {x.FlightNumber, x.Takeoff_time}, x => x,
            (key, vals) => ConsolidateFlightInfo(vals));

foreach(var flight in consildatedFlights)
    Console.WriteLine($"FlightNumber: {flight.FlightNumber}, Takeoff Time: {flight.Takeoff_time}, User Count: {flight.UserCount}");

public static FlightInfo ConsolidateFlightInfo(IEnumerable<FlightInfo> flights)
{
    var list = flights.ToList();
    var ret = list[0];
    ret.UserCount = list.Sum(x => x.UserCount);
    return ret;
}

The first argument to .GroupBy specifies an anonymous type describing describing the properties you want to group by. The second item specifies what you want in your results lists (one per group). In this case we want the entire flight info object. The third parameter specifies how you want to transform each grouping. In this case we pass each set of grouped flights to a method that sums UserCount and returns a single FlightInfo with that summed value.

A little late but thank you for this!! this worked perfectly. I just marked your response as the answer :) — Fernando, Jun 03 '20 at 23:17

Dennis Kuypers · Answer 2 · 2018-10-22T18:03:27.577

I made your FlightInfo a class and renamed some properties according to C# code styles. Also I added a constructor that takes the data.

class FlightInfo
{
    public FlightInfo(string flightNumber, string origination, ...)
    {
        FlightNumber = flightNumber;
        Origination = origination;
        // ...
    }

    public string FlightNumber { get; set; }
    public string Origination { get; set; }
    public string Destination { get; set; }
    public string TakeoffTime { get; set; }
    public string LandingTime { get; set; }
    public int UserCount { get; set; }
}

We add a FlightManager class that takes care of

preventing any multithreading issues using the lock statement
consolidating the entries

FlightManager

class FlightManager
{
    private object _lock = new object();
    private List<FlightInfo> _flights = new List<FlightInfo>();

    public void Add(FlightInfo info)
    {
        lock(_lock)
        {
            // look for existing flights
            var existing = _flights.FirstOrDefault(f =>
            {
                return f.FlightNumber == info.FlightNumber
                    && f.TakeoffTime == info.TakeoffTime;
            });

            // FirstOrDefault will return null if none found
            if(existing == null)
            {
                // add as new flight
                _flights.Add(info);
            }
            else
            {
                // add passenger count
                existing.UserCount += info.UserCount;
            }
        }
    }
}

Usage

static FlightManager Manager = new FlightManager();

allFlights.Add(new FlightInfo(FlightNumber, Origination, Destination, TakeoffTime, LandingTime, UserCount);

POCOs (data objects)

There is an alternative syntax to initialize your FlightInfo if you drop the constructor.

var flightInfo = new FlightInfo()
{
    Origination = origination,
    Destination = destination,
    // ...
}

but a constructor makes your intent clear, that the user has to provide all data in advance. If you are using a serialization library this might not be possible though.

There are some more methods, for example you could use a `ConcurrentDictionary` and its `AddOrUpdate` method to do this. — Dennis Kuypers, Oct 22 '18 at 18:05

c# - Using LinQ to look for duplicate items in a list and updating object properties if so

2 Answers2

FlightManager

Usage

POCOs (data objects)