0

Im storing a huge amount of data in JSON format that needs to be committed to DB. This JSON data is being deserialized into a C# class[]. Thing is there are other seed data thats stored in a static readonly Class[] and simply being loaded/sent to DB. My question is which is better assuming we have 20k records for an entity. Having a JSON and deserializing or having a huge static readonly class[]? My objective is to have a single uniform method of storing data instead of different types of files for the same thing.

Edit: My question apparently wasn't well explained so Ill use an example this time. I have 2 options to load data: a Json:

[
 {
   Id: "56d0bdbe-25be-4ea8-a422-dc302deee962",
   Name: "C-1"
   LegacyId: 1
 },
 {
   Id: "7bf2e997-8a8b-43c9-ba08-1770cd3adb38",
   Name: "C-2"
   LegacyId: 2
 }
]

Each item is being deserialized into a UserEntity which is then later stored in db. However there are other files that are being used for the same purpose of loading data to db. But in this case they are stored like this:

     public static class UserSeedData
        {
            public static readonly UserEntity[] UserData = {
                new UserEntity{
                 Id = "56d0bdbe-25be-4ea8-a422-dc302deee962", 
                 Name = "C-1", 
                 LegacyId = 1
                },
                new UserEntity{ 
                 Id = "7bf2e997-8a8b-43c9-ba08-1770cd3adb38", 
                 Name = "C-2", 
                 LegacyId = 2
                }
            };
        }

My question is which is faster or if theres a difference at all between loading and serializing the JSON // loading the static class UserSeedData to insert it on DB.

Im focusing on the deserialization part and not the committing to DB. This is a legacy system which is currently using BOTH ways to load data. Im simply trying to normalize the process by defining which method is better.

In the end the code to commit to DB is either:

From JSON:

List<UserEntity> users = JsonConvert.DeserializeObject<List<UserEntity>>(File.ReadAllText("Users.json"));

            foreach (var userData in users)
            {
                _db.Add(userData);
            }

Or from C# class:

    foreach (var userData in UserSeedData.UserData)
    {
        _db.Add(userData);
    }
Edgar Contreras
  • 11
  • 3
  • Your bottleneck will be the DB write, not the C# code, so you should focus your efforts on improving performance/doing what is best here. That will depend on your data, your model, your DB technology etc. etc. etc. – jason.kaisersmith Apr 12 '21 at 16:44
  • Begs the question: why are you storing JSON in a database, use proper columns and tables. And what is a `static readonly Class[]`, is it an array of a certain class, or is it some other kind of collection (why use an array rather than a `Dictionary`)? Why is the fact that it is `static` or `readonly` relevant? – Charlieface Apr 12 '21 at 16:55
  • @Charlieface I edited my question with more details. Please check – Edgar Contreras Apr 12 '21 at 17:16
  • 20k records isn't really huge, both for JSON and for a DB. For such a quantity, performance won't be a noticeable problem for any reasonable code. Just use whatever you think as more readable. – Alejandro Apr 12 '21 at 17:51
  • I see, so you are storing the data in normal tables via EF. If I understand correctly, you are saying to have the whole 20K records generated in code as above. The static class is almost certainly faster, how much so is probably not very much, and it's more unwieldy IMHO. A third option is to keep the data in a database in the first place. Also, how much of the data can be generated on the fly, such as Id numbers? – Charlieface Apr 12 '21 at 20:16
  • @Alejandro Thats what I thought, however Im actually trying to find an explanation for it. Currently thinking of executing a test bench and comparing loading times. – Edgar Contreras Apr 13 '21 at 17:32
  • It cant be stored in DB since this data's objective is to use it as Seed. So its imported the first time the solution is brought up. Nothing is being generated, we are basically parsing and committing to db. – Edgar Contreras Apr 13 '21 at 17:34

0 Answers0