Stack Overflow
Stack Overflow

Questions tagged [decimal]

Decimal is the name for our common base-ten numeral system. It may also refer to non-integer values expressed with a decimal point.

Decimal is the base-10 positional numeral system. It is the ordinary numeral system we use; it can be used to represent both integers and (using a decimal point) non-integer values.

Related tags

7532 questions
2
votes
3 answers

c# decimal to hex & hex to decimal / hex to ascii in LINQ

I'm trying to do some conversion and would like to use Linq to achieve the following, Decimal to Hex, Decimal to Ascii, Hex to Decimal, Hex to Ascii Can someone please show me how to do this efficently in Linq? I'll be displaying the output into…
Clu
  • 339
  • 3
  • 8
  • 18
2
votes
0 answers

How to format scientific E notation into human readable numbers? (numbers larger than BigDecimal)

I'm trying to create a number class which can hold values from 4.9E-324e-1.7976931348623157E308 to 1.7976931348623157E308e1.7976931348623157E308 in both signs. (yes, the exponent is also a double) My number class is in base 2. I have all the…
LXIV
  • 21
  • 3
2
votes
2 answers

Yup.number().integer() is not considering 1.0 as a decimal, how to fix this?

I have a field that should only accept integers because my backend can't accept float type. When using Yup.number().integer() to achieve this, it works fine for whole numbers like 2, 3, 100, etc. However, when I enter a value like 1.0, Yup treats it…
Hoa Do
  • 23
  • 1
  • 4
2
votes
1 answer

Javascript: adding decimal to input field when a button is clicked

I have the following code: document.getElementById("decimal").addEventListener("click", () => { if (!inputBox.value.includes(".")) { inputBox.value += "."; } });
stilts15
  • 21
  • 1
2
votes
0 answers

'Property doesn't exist' error with imported module in Svelte

I want to use big.js or decimal.js to perform accurate calculations of decimal numbers in my basic Svelte app. To test it, I have imported the module in my App.svelte file: import Decimal from "../node_modules/decimal.js/decimal.mjs"; import Big…
Sidders
  • 124
  • 10
2
votes
1 answer

Is it possible to preview github.com/shopspring/decimal values in VSCode during debugging?

This is a preview of a normal int64/float64 variable in Go in vscode during debugging. And this is how it looks when using a variable from Decimal library. If that would be possible then debugging would be so much easier. Right now, being a value…
Alex
  • 78
  • 1
  • 6
2
votes
1 answer

Convert a System.Double to an unscaled value and scale

Given a double value: double value = 123.456; How would I obtain an unscaled representation of the value (123456) and scale(3)? This was fairly trivial with decimal because it's not an IEEE-754 floating point implementation, but it seems much…
Matthew Layton
  • 39,871
  • 52
  • 185
  • 313
2
votes
3 answers

How can I verify that a swt text widget is a decimal value

In swt, text widget allow any string. But What is the most appropriate SWT widget in which to enter a Decimal value? I found two answers : First, implement the VerifyKeyListener and VerifyListener, work for french decimal notation, but simple and…
Grégory
  • 1,473
  • 1
  • 17
  • 28
2
votes
2 answers

Rounding numbers to the nearest decimal places in R

Consider the following data set: df <- data.frame(id=1:10, v1=c(2.35456185,1.44501001,2.98712312,0.12345123,0.96781234, 1.23934551,5.00212233,4.34120000,1.23443213,0.00112233), …
iGada
  • 599
  • 3
  • 9
2
votes
3 answers

How to make that f"..." string formatting uses comma instead of dot as decimal separator?

I tried: import locale print(locale.locale_alias) locale.setlocale(locale.LC_ALL, '') locale.setlocale(locale.LC_NUMERIC, "french") print(f"{3.14:.2f}") but the output is 3.14 whereas I would like 3,14. How to do this with f"..." string…
Basj
  • 41,386
  • 99
  • 383
  • 673
2
votes
2 answers

C# decimal type equality

Is the equality comparison for C# decimal types any more likely to work as we would intuitively expect than other floating point types?
Dan
  • 379
  • 3
  • 12
2
votes
1 answer

How to create a Span from a pointer to a decimal variable in F#?

In C# I can do the following: decimal v = 123.456M; var span = new ReadOnlySpan(&v, 4); var flags = span[0]; var hi32 = span[1]; var lo32 = span[2]; var mi32 = span[3]; Is there a way to do the same in F# 5.0? Context: I’m trying…
Andrew Rybka
  • 23
  • 3
2
votes
5 answers

Check precision to nth decimal place?

I have to calculate the values of a hot plate and have it accurate only to the first decimal place. I am stumped on trying to figure out how to check all the array values if they changed. I found out that 724 runs made no change after that to the…
krizzo
  • 1,823
  • 5
  • 30
  • 52
2
votes
4 answers

Round decimals to the closest non zero digit in Kotlin

I am trying to print multiple doubles with different decimal points, I have search for a while but I cannot find the exact solution for my need. Let me explain it as an example, so there are a few double variables here: 1.23 0.330 1.00 1.001 The…
William Tan
  • 85
  • 6
2
votes
1 answer

DecimalField validation error, returns incorrect value Django

Ive got a DecimalField in one of my forms where a user enters a price. Say the user enters 11.00, now when i retrieve it as in (priceform.cleaned_data) it returns Decimal('11.00') so it looks like price = Decimal('11.00') which triggers a…
jeddah
  • 49
  • 5
1 2 3
…
99
100