Share via


How to apply precision and scale in .net

Question

Thursday, April 19, 2012 1:43 PM

Hi,

   i need to apply  precision and scale for a value:

say precion=5,scale=2  the the result should be 78890.251 how to do in .net

 

All replies (5)

Thursday, April 19, 2012 2:22 PM ✅Answered

Floats and doubles in C# have a fixed range of precision.  You can't change that. 

The SQL type decimal(5, 2) specifies that the value has 5 digits of precision with 2 digits after the decimal - ###.##  Both floats and double easily handle this range of values so you actually have more precision (and scale) then you need.  The only time you'd like care about actually reducing the precision/scale is when you're displaying the values and for that the format specifiers are used.  Internally you'll always want to use the highest precision possible to avoid roundoff errors.

Note that the decimal type can also be used but it is even more precise than doubles and is probably overkill for the SQL equivalent of 5,2.

Michael Taylor - 4/19/2012
http://msmvps.com/blogs/p3net


Friday, April 20, 2012 7:01 AM ✅Answered

You can use an int for that, and divide by 100 for display. For example, do all computations in cents and only display in euros.


Thursday, April 19, 2012 1:52 PM

If you're referring to displaying a value as a string then use either String.Format or ToString with a format specifier.

If you're referring to the numeric value itself (similar to how SQL might work) then you can't.  Floating point values have fixed precision so all you can do is control how much of that you see.

Michael Taylor - 4/19/2012
http://msmvps.com/blogs/p3net


Thursday, April 19, 2012 2:10 PM

thanks cool,

     this is not i want i need. Actually i am having a calculated double value. I have to apply precision and scale to this value. We found something like casting to decimal(5,2), which is having precision as 5 and scale as 2. I need to apply this to my double value. Is it possible? Kindly help me out.

Regards

Sajith


Saturday, April 21, 2012 11:59 AM

thanks Louis.fr & Cool