Results 1 to 3 of 3

Thread: Unexpected Value Rounding with a Calculation Formula

  1. #1
    Moderator cellophane's Avatar
    Join Date
    August 9, 2011
    Location
    Louisville, KY
    Posts
    6,229
    Current Local Time
    02:37 AM

    Not allowed! Not allowed!

    Unexpected Value Rounding with a Calculation Formula

    Per the title - I have a formula setup to calculate a height based on a triangle calculation, and the values are randomly popping up with fractions, and then rounding back down to a non-fraction value and I have never seen this before without using a Round() function. Any clues to what is happening?!

    https://www.screencast.com/t/reNsDihtjN

  2. #2
    The Moderator with No Imagination MPwuzhere's Avatar
    Join Date
    December 14, 2010
    Location
    Coeur d Alene, ID
    Posts
    4,848
    Current Local Time
    11:37 PM

    Not allowed! Not allowed!
    Your angle settings are set to 2 decimal places....change it to 3 or 4....as 1/256 is .00390625. So it's rounding to 0.

  3. #3
    Moderator cellophane's Avatar
    Join Date
    August 9, 2011
    Location
    Louisville, KY
    Posts
    6,229
    Current Local Time
    02:37 AM

    Not allowed! Not allowed!
    so... there seems to be some sort of change in values (or value display) between 2018 & 2019. note that the angles are set to 2 decimal places in both.

    https://www.screencast.com/t/qQca6faLP (18 on the left, 19 on the right)

Similar Threads

  1. Incremental Rounding and Unit Conversion Formula Question
    By SuperDave in forum Architecture - Family Creation
    Replies: 5
    Last Post: July 12th, 2016, 04:47 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •