A Microsoft Office (Excel, Word) forum. OfficeFrustration

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

Go Back   Home » OfficeFrustration forum » Microsoft Excel » Worksheet Functions
Site Map Home Register Authors List Search Today's Posts Mark Forums Read  

subtraction returning incorrect calue



 
 
Thread Tools Display Modes
Prev Previous Post   Next Post Next
  #1  
Old March 18th, 2010, 06:31 PM posted to microsoft.public.excel.worksheet.functions
Ashish_Vaidya
external usenet poster
 
Posts: 2
Default subtraction returning incorrect calue

We have a spreadsheet that subtracts one number from another. We did not
understand why Excel is rounding the way it did so we decided to add decimal
places until we see the value change. For instance, one of our number was
going out to x,xxx.499999999999999999999999999999900000 while the other
number was x,xxx.5000000000000000000000000000000000. When we rounded the
numbers, one rounded up while the other rounded down. This is fine. Now
that we fixed the formula to TRUNC instead of ROUND, I thought the issue
would go away but now when I subtracted two numbers that were like
x,xxx.xx000000000000000000000000 and another number that is also
x,xxx.xx000000000000000000000000, why would I receive an answer that comes
back as x,xxx.xx4999999999900000000000000000? This doesn't make sense to me.
Has anyone else run into this issue? How did you fix it?

--
Ashish
 




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump


All times are GMT +1. The time now is 07:57 PM.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 OfficeFrustration.
The comments are property of their posters.