Stack Overflow
Stack Overflow

Questions tagged [significant-digits]

Significant digits are a way of describing the precision of measurements in a scale-independent way.

Typically if a number is in normal scientific notation, the number of significant digits is the length (number of digits) of the mantissa or significand. The significant figures of a number are those digits that carry meaning contributing to its precision. This includes all digits except: All leading zeros, Trailing zeros when they are merely placeholders to indicate the scale of the number (exact rules are explained at Identifying significant figures), and spurious digits introduced, for example, by calculations carried out to greater precision than that of the original data, or measurements reported to a greater precision than the equipment supports.

Numbers are often rounded to avoid reporting insignificant figures. For instance, if a device measures to the nearest gram and gives a reading of 12.345 kg, it would create false precision to express this measurement as 12.34500 kg. Significant Figures - Wikipedia

178 questions
0
votes
2 answers

Where are these tailing digits in Python value from?

In mathematics, the value 1/3 is 0.333(3 to infinity). However, Python prints out wrong digits at the tail. print(f"{1/3:.100f}".rstrip("0")); print(f"{1/3:.100f}"); Result: 0.333333333333333314829616256247390992939472198486328125 Result (no…
Dee
  • 7,455
  • 6
  • 36
  • 70
0
votes
1 answer

Keep significant zeros using str(float) in python

Considering the code below. The output of this is 2.76 but for the application I am using it for, the zero after 2.76 is significant thus I am searching for a way to output 2.760. f = 2.7598 r = round(f,3) s = str(r) print(s)
James Cannon
  • 3
  • 1
0
votes
0 answers

Excel calculation - number of digits when copied

So, this might be a very specific use-case, I'm not sure. I'm a Scientist who works with HPLC data. The data is handled in a piece of software called Chromeleon, where reports are built in an Excel-style spreadsheet. While it looks suspiciously like…
CTH
  • 27
  • 7
0
votes
0 answers

Is there an R function for rounding based on the size of the number?

I want the rounding of a set of numbers to automatically change as the numbers get larger. I am making barplot labels. Here is what currently shows up: <39 39 to 11564 11564 to 23090 23090 to 50893 50893 to 89034 89034 to 138900 138900 to…
Tessa Danehy
  • 1
0
votes
1 answer

Can't round to significant figures without floating point errors

I'm populating a DataGridView with a set of data with numbers of varying size. The source of the data is a text file which has numbers with very big precision, but the precision needn't be that huge. For display purposes, 4 significant figures…
djv
  • 15,168
  • 7
  • 48
  • 72
0
votes
1 answer

R formatC gives not same output with the same parameter

Ich have a numeric value (x1=0.01) which I want to format with two significant digits: > p_digits <- 2 > x1 <- 0.01 > str(x1) num 0.01 > formatC(x1, format = "fg", digits = p_digits, flag = "#", big.mark = "'") which gives: [1] "0.010" Now, I…
giordano
  • 2,954
  • 7
  • 35
  • 57
0
votes
1 answer

Add only changed numbers to a running list

I am trying to log values for a meter at one minute intervals. In lieu of entering the full value each time, I want to be able to enter just the numbers that are different: as opposed to entering the difference of the values or the entire…
ThomK
  • 35
  • 1
  • 4
0
votes
1 answer

Loss of significance in a complicated recurrence in R, related to Gaussian quadrature

I am not a programmer (this is my first post here), and don't have much experience with floating point arithmetic. I apologize if I missed something obvious. I have been trying to find the parameters for Gaussian quadrature with custom weight…
Yuriy S
  • 103
  • 6
0
votes
0 answers

choose number of significant digits in matlab for whole code

I'm trying to format my code in such a way that a chosen number of significant digits are saved, throughout the whole code. Basically, I'm looking for something that does the same as the function round(x, d, 'significant'), but will retain this…
Netta Gal
  • 9
  • 1
0
votes
1 answer

BigDecimal Division using Significant Figures

I am using BigDecimal for division. I would like the quotient to be rounded to the correct number of significant figures. For example @Test public void testBigDDivision() { BigDecimal top = new BigDecimal("0.25"); BigDecimal bottom =…
Oliver Reimer
  • 15
  • 1
  • 4
0
votes
1 answer

Setting decimal places in BlueSky Statistics

In BlueSky Statistics, how can I control the number of decimal places used to display the results of the analysis? It's showing four value after the decimal place and I would like it to round off after two.
Carla F
  • 1
  • 2
0
votes
1 answer

Suggestions for a user-defined function for significant figures and trailing zeros in Excel/VBA

I have written the user-defined function (UDF) below for an Excel sheet at work. It rounds to significant figure and is able to handle trailing zeros. Actually, the UDF works as intended! The only thing, that you need to be aware off is that it…
reuss
  • 121
  • 1
  • 1
  • 9
0
votes
0 answers

Errors with long integer in Python 3

I have this program that changes a number into words, but the problem is that when you have more than 16 digits in the number, it starts changing the numbers. This is done in Python 3 and we are wondering why this is happening and if there is a…
James Abela
  • 97
  • 1
  • 3
0
votes
2 answers

How do I force R to write out scientific notation with 3 significant figures after a decimal?

Because Fortran, I need to write a program that, among other wonderful features, will force R to output a number in scientific notation with 3 significant figures after the decimal point. Specifically, I need 170.5 to be written to output as…
Jared Brewer
  • 132
  • 2
  • 12
0
votes
1 answer

Is there a function in R, which returns number of significant digits?

Somewhere I found this code signdig = function(x) { length(gregexpr("[[:digit:]]", as.character(x))[[1]]) } But this returns strange numbers, like > L=matrix(c(15,5,9,3.111),nrow=2) > kappa(L) [1] 239.5819 > signdig(kappa(L)) [1] 15 Can…
Harshvardhan
  • 479
  • 1
  • 3
  • 12
1 2 3
…
11 12