Questions tagged [numerical]

This tag is for questions concerning problems using numbers which either cannot be exactly solved, or where the exact solution may be much more difficult to acquire than by using numerical methods.

This tag is for questions concerning problems using numbers which either cannot be exactly solved, or where the exact solution may be much more difficult to acquire than by using numerical methods.

It includes questions about algorithms, precision and accuracy of underlying numbers, advantages and disadvantages of several methods, error propagation, problems with the underlying numbers and data structures.

508 questions
9
votes
7 answers

Python arithmetic with small numbers

I am getting the following unexpected result when I do arithmetic with small numbers in Python: >>> sys.float_info sys.float_info(max=1.7976931348623157e+308, max_exp=1024, max_10_exp=308, min=2.2250738585072014e-308, min_exp=-1021, min_10_exp=-307,…
user1389890
  • 335
  • 1
  • 4
  • 9
9
votes
6 answers

How do I save a floating-point number in 2 bytes?

Yes I'm aware of the IEEE-754 half-precision standard, and yes I'm aware of the work done in the field. Put very simply, I'm trying to save a simple floating point number (like 52.1, or 1.25) in just 2 bytes. I've tried some implementations in Java…
Robin Rodricks
  • 110,798
  • 141
  • 398
  • 607
8
votes
2 answers

Is there a way to reduce scipy/numpy precision to reduce memory consumption?

On my 64-bit Debian/Lenny system (4GByte RAM + 4GByte swap partition) I can successfully do: v=array(10000*random([512,512,512]),dtype=np.int16) f=fftn(v) but with f being a np.complex128 the memory consumption is shocking, and I can't do much more…
timday
  • 24,582
  • 12
  • 83
  • 135
8
votes
2 answers

Categorical & Numerical Features - Categorical Target - Scikit Learn - Python

I have a data set containing both categorical and numerical columns and my target column is also categorical. I am using Scikit library in Python34. I know that Scikit needs all categorical values to be transformed to numerical values before doing…
USC.Trojan
  • 391
  • 5
  • 14
8
votes
2 answers

Best way to check if double equals negative infinity in C++

I found this: http://en.cppreference.com/w/cpp/numeric/math/isinf but it appears to check for either positive or negative infinity. I just want to check if a value is equal to exactly negative infinity, or in otherwords is log(0) Thanks for answer!…
evolvedmicrobe
  • 2,672
  • 2
  • 22
  • 30
8
votes
3 answers

Dealing with floating point errors in .NET

I'm working on a scientific computation & visualization project in C#/.NET, and we use doubles to represent all the physical quantities. Since floating-point numbers always include a bit of rounding, we have simple methods to do equality…
Hank
  • 8,289
  • 12
  • 47
  • 57
8
votes
5 answers

Numerical Java Libraries

Most of my code is in Java. For standardized algorithms: matrix operations, FFT, ... I would prefer to not use my own pure Java implementations, and are perfectly happy using unsafe FFI/JNI calls. What are the libraries I should look into? Thanks!
anon
  • 41,035
  • 53
  • 197
  • 293
8
votes
9 answers

Java performance in numerical algorithms

I am curious about performance of Java numerical algorithms, say for example matrix matrix double precision multiplication, using the latest JIT machines as compared for example to hand tuned SSE C++/assembler or Fortran counterparts. I have looked…
Anycorn
  • 50,217
  • 42
  • 167
  • 261
8
votes
3 answers

How to test numerical analysis routines?

Are there any good online resources for how to create, maintain and think about writing test routines for numerical analysis code? One of the limitations I can see for something like testing matrix multiplication is that the obvious tests (like…
kaybenleroll
  • 16,794
  • 16
  • 54
  • 66
8
votes
2 answers

How does the size of a binary influence the execution speed

How does the size of a binary influence the execution speed? Specifically I am talking about code written in ANSI-C translated into machine language using the gnu or intel compiler. The target platform for the binary are modern computers with intel…
highsciguy
  • 2,569
  • 3
  • 34
  • 59
8
votes
7 answers

Identifying common periodic waveforms (square, sine, sawtooth, ...)

Without any user interaction, how would a program identify what type of waveform is present in a recording from an ADC? For the sake of this question: triangle, square, sine, half-sine, or sawtooth waves of constant frequency. Level and frequency…
endolith
  • 25,479
  • 34
  • 128
  • 192
7
votes
3 answers

Computationally efficient three dimensional arrays in C

I am trying to solve numerically a set of partial differential equations in three dimensions. In each of the equations the next value of the unknown in a point depends on the current value of each unknown in the closest points. To write an efficient…
Coffee on Mars
  • 988
  • 6
  • 21
7
votes
6 answers

How to do numerical integration with quantum harmonic oscillator wavefunction?

How to do numerical integration (what numerical method, and what tricks to use) for one-dimensional integration over infinite range, where one or more functions in the integrand are 1d quantum harmonic oscillator wave functions. Among others I want…
7
votes
4 answers

Logsoftmax stability

I know how to make softmax stable by adding to element -max _i x_i. This avoids overflow and underflow. Now, taking log of this can cause underflow. log softmax(x) can evaluate to zero, leading to -infinity. I am not sure how to fix it. I know this…
Abhishek Bhatia
  • 9,404
  • 26
  • 87
  • 142
7
votes
3 answers

C++ Float Comparison Around Zero Fails With Gtest

we are struggling with one unit test of mine for quite some while. During investigation we have found the root cause, which seems to be the comparison in floats (see the following code snippet where I have simplified the computation but it still…
MattW
  • 461
  • 3
  • 10