Stack Overflow
Stack Overflow

Questions tagged [numerical]

This tag is for questions concerning problems using numbers which either cannot be exactly solved, or where the exact solution may be much more difficult to acquire than by using numerical methods.

This tag is for questions concerning problems using numbers which either cannot be exactly solved, or where the exact solution may be much more difficult to acquire than by using numerical methods.

It includes questions about algorithms, precision and accuracy of underlying numbers, advantages and disadvantages of several methods, error propagation, problems with the underlying numbers and data structures.

508 questions
5
votes
3 answers

Haskell 64 bit numerical type

I am writing a function in Haskell that deals with numbers beyond the length of a 32 bit int. I cannot find the type to do this and I seem to be searching for the wrong terms. It needs to be able to hold numbers with the length of about 2^40 without…
Yacoby
  • 54,544
  • 15
  • 116
  • 120
5
votes
1 answer

compiler-optimization for numerical stability

do GCC or similar compilers perform optimizations that are aimed at improving the numerical stability of floating-point operations. It is known that seemingly simply operations like addition or computing the norm of a vector are numerically unstable…
shuhalo
  • 5,732
  • 12
  • 43
  • 60
5
votes
2 answers

Finding the "discrete" difference between close floating point numbers

Suppose I have two floating point numbers, x and y, with their values being very close. There's a discrete number of floating point numbers representable on a computer, so we can enumerate them in increasing order: f_1, f_2, f_3, .... I wish to…
Szabolcs
  • 24,728
  • 9
  • 85
  • 174
5
votes
4 answers

Writing into a text file without overwriting it

I'm doing a numerical simulation of gravity in C++ and I want to back up my results every time a single step is counted. However, the way I do it now, the programm always overwrites the file. I guess I would be able to solve in by always saving the…
Tom83B
  • 2,059
  • 4
  • 18
  • 28
5
votes
2 answers

Numerical Optimization of PL/SQL, or alternatives

We have a requirement to do some computational heavy lifting to connect with an Oracle database. Up until now, we have performed our numerical calculations in PL/SQL and largely put up with the lack of performance. Now we are implementing a…
Matti Wens
  • 740
  • 6
  • 24
5
votes
4 answers

undefined / NaN error in output

This is my first post on stackoverflow, i am encountering error in the following code , no error shows in inspect element / JS console in firefox but for some reason the ouput after calculation is displaying undefined / NaN error . The input from…
Prabh Rai
  • 53
  • 6
5
votes
3 answers

How to find the length of a numerical variable using PROC SQL

I have a dataset with a column of phone numbers. I want to filter this dataset using PROC SQL WHERE the length of the numbers is at least 7 digits. In normal SQL I can just apply a length function around a number and it works, however in SAS it…
shecode
  • 1,716
  • 6
  • 32
  • 50
5
votes
3 answers

How to compute an exponent in matlab without getting inf?

The title says it all: I want to calculate an exponent in matlab with big numbers, but I get overflow and it just returns infinity. >> 100^1000 ans = Inf Last time I checked, 100^1000 is decidedly smaller than infinity :)
rhombidodecahedron
  • 7,693
  • 11
  • 58
  • 91
5
votes
2 answers

I need to calculate Stirling's approximation very fast

I'm writing a small library for statistical sampling which needs to run as fast as possible. In profiling I discovered that around 40% of the time taken in the function is spent computing Stirling's approximation for the logarithm of the factorial.…
pg1989
  • 1,010
  • 6
  • 13
5
votes
2 answers

Differences between Matlab and Numpy and Python's `round` function

Simplified question Can I make Numpy agree with Matlab and Python's round? Matlab 2013a: >> round(-0.5) ans = -1 Python (using a Numpy array, or just a scalar, same result): >>> import numpy >>> round(numpy.array(-0.5)) -1.0 Numpy, the odd…
Ahmed Fasih
  • 6,458
  • 7
  • 54
  • 95
5
votes
1 answer

Machine epsilon, ε: What is step to next smaller value from 1?

We know that machine epsilon, ε, is ”the difference between 1 and the least value greater than 1 that is representable in the given floating point type”.1 However, the definition says nothing about the next smaller value. What is the difference…
Sean
  • 713
  • 1
  • 10
  • 28
5
votes
2 answers

xsl:sort: sorting by numeric value

I have to sort out the codes in numerical order. The codes have four characters and four numerals. for example, COMP2100 COMP2400 COMP3410 LAWS2202 LAWS2250 when I just do it displays above…
Jane Doe
  • 543
  • 5
  • 9
  • 19
4
votes
2 answers

What to do with “Inferred type is less polymorphic than expected”?

I need the Numeric.FAD library, albeit still being completely puzzled by existential types. This is the code: error_diffs :: [Double] -> NetworkState [(Int, Int, Double)] error_diffs desired_outputs = do diff_error <- (diff_op $ error' $ map…
Astro
  • 858
  • 1
  • 8
  • 8
4
votes
1 answer

Complex number array format efficiency for common operations

Another person in my office and I got into a discussion about which complex number matrix array format is more efficient: storing the real and imaginary parts interleaved, as in struct { double real; double imag; } Complex foo[m][n]; or by…
telotortium
  • 3,383
  • 2
  • 23
  • 25
4
votes
0 answers

Numerical integration - Boost vs GSL

I've found two good libraries for numerical integration in c++ that are easy enough for me to use. I have to integrate fairly complex functions with the possibility of singularities at endpoints and it appears that an adaptive integration method…
Raul Laasner
  • 1,475
  • 1
  • 17
  • 30
1 2 3
…
33 34