Donate Now Goal amount for this year: 2500 USD, Received: 1627 USD (65%)

Results 1 to 3 of 3

Thread: Unexpected Value Rounding with a Calculation Formula

  1. #1
    Moderator cellophane's Avatar
    Join Date
    August 9, 2011
    Location
    Louisville, KY
    Posts
    6,140
    Current Local Time
    12:41 AM

    Unexpected Value Rounding with a Calculation Formula

    Per the title - I have a formula setup to calculate a height based on a triangle calculation, and the values are randomly popping up with fractions, and then rounding back down to a non-fraction value and I have never seen this before without using a Round() function. Any clues to what is happening?!

    https://www.screencast.com/t/reNsDihtjN

  2. #2
    The Moderator with No Imagination MPwuzhere's Avatar
    Join Date
    December 14, 2010
    Location
    Coeur d Alene, ID
    Posts
    4,783
    Current Local Time
    09:41 PM
    Your angle settings are set to 2 decimal places....change it to 3 or 4....as 1/256 is .00390625. So it's rounding to 0.

  3. #3
    Moderator cellophane's Avatar
    Join Date
    August 9, 2011
    Location
    Louisville, KY
    Posts
    6,140
    Current Local Time
    12:41 AM
    so... there seems to be some sort of change in values (or value display) between 2018 & 2019. note that the angles are set to 2 decimal places in both.

    https://www.screencast.com/t/qQca6faLP (18 on the left, 19 on the right)

Similar Threads

  1. Incremental Rounding and Unit Conversion Formula Question
    By SuperDave in forum Architecture - Family Creation
    Replies: 5
    Last Post: July 12th, 2016, 04:47 AM
  2. Replies: 0
    Last Post: September 3rd, 2012, 10:45 PM
  3. Replies: 0
    Last Post: August 28th, 2012, 12:45 AM
  4. some unexpected things
    By gaby424 in forum Out There
    Replies: 12
    Last Post: August 24th, 2012, 10:29 AM
  5. Replies: 0
    Last Post: August 16th, 2012, 12:45 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •