0

I have a C# Windows Form UI that controls a serial instrument (a digital scale) using NI VISA, and I'm trying to figure out the best way to format the output for results tracking in a SQL Server database. The scale is used to weigh various liquids, and the UI calculates volume with a function like this:

private Decimal GetVolume(string liquid)
{
    Decimal density = 0;
    string match = Regex.Match(netWeight, @"\d+\.*\d*").Value;
    Decimal weight = Convert.ToDecimal(match);

    switch (liquid)
    {
        case "Liquid 1":
            density = 1.00M;
            break;

        case "Liquid 2":
            density = 1.23M;
            break;
    }

    return weight / density;
}

Where netWeight is the string output of the scale. The scale is only accurate to 0.1g, and the densities to .01 g/mL. I'm trying to avoid rounding errors as much as possible since total liquid usage will be tracked over time, and I'm not sure what the best practice is for doing this sort of thing and storing it in a database. Right now I have the SQL Server column for volume defined as Decimal(9,4), but other than that I'm lost.

My programming experience basically amounts to winging it and Googling, so I'm really not used to doing calculations like this in code. How do I select the best datatype, precision, etc. to minimize accumulated rounding errors over time and enable accurate calculations in the SQL Server database?

The volume calculations will eventually be multiplied by prices in the db, so that's something to consider too.

I've looked at every discussion of the various numeric datatypes in SQL Server that I could find, and read about money calculations with Decimal, etc. but I haven't found any direct discussion of a similar situation to mine.

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
longdistancerunner
  • 11
  • 2

1 Answers1

0

There is currently no implementation of BigDecimal in .NET. There is only the BigInteger representation. You can implement a real of infinite precision as a fraction with a numerator and denominator based on BigInteger. But I think Decimal is enough for your task.

The Documentation of Microsoft: https://learn.microsoft.com/en-us/dotnet/api/system.numerics.biginteger?view=net-7.0

For an example, my BigInteger (See UnlimitedInteger): https://github.com/DmitriySidyakin/Blog-BitMath/blob/main/BitMath/UnlimitedInteger.cs

HailToTheVictor
  • 388
  • 2
  • 8
  • 28