Simple Calculation Error

Question

0.00/5 (No votes)

See more:

Here is a short code snippet that I am having trouble with:

C#

private double DetermineBatteryDepthOfDischarge(ComboBox DepthOfDischarge)
{
    int k = DepthOfDischarge.SelectedIndex;
    double DoD = DoD = 10 / (k + 1);
    //MessageBox.Show(DoD.ToString());
    return (DoD);
}

The combobox has 10 items: 10%, 20%, ... , 90%, 100%.
Now if I select say the third item (30%) in the combobox, the DoD variable stores 3 and not 3.33333333...
If I select say the 7th item, (70%) in the combobox, then the DoD reads 1 and not 1.42857...

The decimal seems to be truncated. Why is this happening in a 'double' or 'decimal' data type?
Am I missing something important?

Posted 30-Jan-12 1:06am

Winston_D

Updated 30-Jan-12 1:09am

v3

Add a Solution

3 solutions

Solution 3

C#

private double DetermineBatteryDepthOfDischarge(ComboBox DepthOfDischarge)
{
    Double k = cdbl(DepthOfDischarge.SelectedIndex);
    Double DoD = 10.0 / (k + 1.0);
    //MessageBox.Show(DoD.ToString());
    return (DoD);
}

Posted 30-Jan-12 1:17am

Rakesh Meel

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

dolbysys · Accepted Answer · 2012-01-30T01:09:00

Solution 1

Use floating-point values instead of integers:

C#

private double DetermineBatteryDepthOfDischarge(ComboBox DepthOfDischarge)
{
    double k = Convert.ToDouble(DepthOfDischarge.SelectedIndex);
    double DoD = 10.0 / (k + 1.0);
    //MessageBox.Show(DoD.ToString());
    return (DoD);
}

Posted 30-Jan-12 1:09am

dolbysys

Comments

Winston_D 30-Jan-12 7:13am

It works like you suggested but why does a simple integer not work like expected?

CRDave1988 30-Jan-12 7:15am

because the rule of exp is that if all element r int than result will be int

Winston_D 30-Jan-12 7:16am

Thnx a million. I'll keep it in mind for next time

Abhinav S · Accepted Answer · 2012-01-30T01:10:00

Solution 2

Try using double DoD = DoD = 10.0 / (k + 1.0);

Posted 30-Jan-12 1:10am

Abhinav S