I have a spreadsheet that has 29 numbers with up to 2 decimal place precision
(44.29, 36.89, etc). I am taking the 1st number and subtracting the other 28
numbers from it
(=A1-B1-B2-B3-B4-B5-B6-B7-B8-B9-B10-B11-B12-B13-B14-C1-C2-C3-C4-C5-C6-C7-C8-C9-C10-C11-C12-C13-C14).
When I do this I get a result of 775,817.3700000010. My question is: where is
that 1 coming from in the 9th decimal place? I am displaying all of my 29
input numbers to 10 decimal places, and they are all 0's past the 2nd decimal
place. They are all entered as raw values (none are the result of a formula).
One other point: if I take out ANY of the 28 cells that are being
subtracted, the problem goes away - the 1 in the 9th decimal place becomes a
0.
I can send you the sample spreadsheet if that would help.
Not a bug but a feature of almost all computer math.
See Excel IEEE error support.microsoft.com/kb/78113
best wishes
--
Bernard V Liengme
www.stfx.ca/people/bliengme
remove caps from emailquot;Randy Spquot; lt;Randy gt; wrote in message
...
gt;I have a spreadsheet that has 29 numbers with up to 2 decimal place
gt;precision
gt; (44.29, 36.89, etc). I am taking the 1st number and subtracting the other
gt; 28
gt; numbers from it
gt; (=A1-B1-B2-B3-B4-B5-B6-B7-B8-B9-B10-B11-B12-B13-B14-C1-C2-C3-C4-C5-C6-C7-C8-C9-C10-C11-C12-C13-C14).
gt; When I do this I get a result of 775,817.3700000010. My question is: where
gt; is
gt; that 1 coming from in the 9th decimal place? I am displaying all of my 29
gt; input numbers to 10 decimal places, and they are all 0's past the 2nd
gt; decimal
gt; place. They are all entered as raw values (none are the result of a
gt; formula).
gt;
gt; One other point: if I take out ANY of the 28 cells that are being
gt; subtracted, the problem goes away - the 1 in the 9th decimal place becomes
gt; a
gt; 0.
gt;
gt; I can send you the sample spreadsheet if that would help.
- Jul 20 Thu 2006 20:08
Calculation Error?
close
全站熱搜
留言列表
發表留言