Page 1 of 3 123 LastLast
Results 1 to 10 of 30
Like Tree11Likes

Thread: RFO Benchmark v2 2016 call for testers

  1. #1
    Administrator Gordon Price's Avatar
    Join Date
    December 7, 2010
    Location
    Rotterdam, South Holland
    Posts
    2,928
    Current Local Time
    01:32 AM

    RFO Benchmark v2 2016 call for testers

    All,
    I am looking for a few testers to give this all new benchmark a go. Ideally at least one to two who have been getting results in the low 100s for the main test.
    If you are game, please PM me.

    Thanks!
    Gordon

  2. #2
    Administrator Gordon Price's Avatar
    Join Date
    December 7, 2010
    Location
    Rotterdam, South Holland
    Posts
    2,928
    Current Local Time
    01:32 AM
    Just wanted to update everyone. Testing is proceeding, but I did find a problem in the RST main journal. Smarter folks than I are looking into it, so I suspect that will be resolved quiet quickly. Also, it looks like there may be a mechanism that allows for the "heavy" main test I was hoping to do. It will require learning a metric crap ton about the inner workings of journal files, but I have a 12 hour flight home to Berlin day after tomorrow, so I have time. Also, I am nearly done with the v2 test for 2015 as well, and I want to do some extensive testing of both to ensure that v2 "Standard" test is indeed producing the same results as the v1 test for 2015.
    Also, in looking at some of the possibilities for expanding the test, now that the structure is there, I suspect I will
    push for a test that mimics the previous tests, plus a little extended testing, and a new lightweight test, for the initial release. Then I can start looking at additional tests for things like view template and filter application, mass view creation, etc.

    Net result, I think we are looking at testing and (minor) additions this week, with a true release early next week. And an extended test made available later.

    Thanks!
    Gordon

  3. #3
    New Member
    Join Date
    September 2, 2015
    Posts
    2
    Current Local Time
    06:32 PM
    Gordon,
    I am doing some testing on various hardware and would love to test in 2016 Revit.
    Is that possible?

  4. #4
    New Member
    Join Date
    September 2, 2015
    Posts
    2
    Current Local Time
    06:32 PM
    Gordon, I would love to test this.

    Quote Originally Posted by Gordon Price View Post
    Just wanted to update everyone. Testing is proceeding, but I did find a problem in the RST main journal. Smarter folks than I are looking into it, so I suspect that will be resolved quiet quickly. Also, it looks like there may be a mechanism that allows for the "heavy" main test I was hoping to do. It will require learning a metric crap ton about the inner workings of journal files, but I have a 12 hour flight home to Berlin day after tomorrow, so I have time. Also, I am nearly done with the v2 test for 2015 as well, and I want to do some extensive testing of both to ensure that v2 "Standard" test is indeed producing the same results as the v1 test for 2015.
    Also, in looking at some of the possibilities for expanding the test, now that the structure is there, I suspect I will
    push for a test that mimics the previous tests, plus a little extended testing, and a new lightweight test, for the initial release. Then I can start looking at additional tests for things like view template and filter application, mass view creation, etc.

    Net result, I think we are looking at testing and (minor) additions this week, with a true release early next week. And an extended test made available later.

    Thanks!
    Gordon

  5. #5
    Administrator Gordon Price's Avatar
    Join Date
    December 7, 2010
    Location
    Rotterdam, South Holland
    Posts
    2,928
    Current Local Time
    01:32 AM
    All, thanks so much for the interest. And the patience. After a ton of "in-house" testing, which was necessitated by some pretty extensive revisions to the journals, I am doing one last round of validations while on the train tomorrow, to verify that year to year and v1 to v2 comparisons are meaningful, and then the whole kit and caboodle will go live. This is definitely a 1.0 release, since the script that ties it all together is 100% new code, and the journals have been revised and added to, so on Sunday everyone can start testing!

    Thanks again.
    Gordon

  6. #6
    Junior Member SysQueKenny's Avatar
    Join Date
    March 28, 2013
    Posts
    44
    Current Local Time
    04:32 PM
    Quote Originally Posted by Gordon Price View Post
    All, thanks so much for the interest. And the patience. After a ton of "in-house" testing, which was necessitated by some pretty extensive revisions to the journals, I am doing one last round of validations while on the train tomorrow, to verify that year to year and v1 to v2 comparisons are meaningful, and then the whole kit and caboodle will go live. This is definitely a 1.0 release, since the script that ties it all together is 100% new code, and the journals have been revised and added to, so on Sunday everyone can start testing!

    Thanks again.
    Gordon
    Gordon I would like to be included in this test also.

  7. #7
    Administrator Gordon Price's Avatar
    Join Date
    December 7, 2010
    Location
    Rotterdam, South Holland
    Posts
    2,928
    Current Local Time
    01:32 AM
    All,
    RFO Benchmark version 2 beta 1 is available for Revit 2015 & 2016!

    The new benchmark has three “test sets” by default. Each set is initiated by simply double clicking the shortcut in the main folder. Results files will land here too.

    Standard is basically the same benchmark as previous years, but running three times and averaging the results. This benchmark is best for posting to RFO for comparisons.

    Simplified removes the render test and non hardware accelerated graphics test, and only runs once, to provide a quicker test with the most pertinent numbers. This benchmark is best for doing a quick evaluation.
    Additionally the Simplified test demonstrates the new messages behavior, which lets you know what specific test is running. This can be helpful when monitoring progress with Task Manager or Resource Monitor so you can see exactly what is hammering CPU, RAM, drive access, etc.

    Expanded adds ‘previous version upgrade’ and ‘link and array’ tests, as well as a new ‘autodesk raytracer’ rendering test (for 2016 only of course), expends on the export tests by exporting all 40 views and at a higher resolution for PNG export, and lastly the remainder of the model creation tests plus the graphics test are done with 24 additional copies of the final model arrayed around the work area, resulting in an overall heavier and more realistic benchmark. Additionally the benchmark is run 5 times with the high and low outliers thrown out and the remainder averaged. And some additional times are included in the results. This benchmark is best for comparing hardware when making purchase decisions, and for those of you with machines getting numbers in the double digits on the old test.

    Some things to be aware of. The journals got an extensive rationalization this year, thanks to some great info from the Factory. This addressed some places where the old journals could power through some errors and give erroneous times, as well as addressing some tests that where reporting too much or two little time. I’ll go into more detail on this stuff later, but for now just know that you should not compare these new v2 results with results from the old test. If you want to compare years, run the new v2 benchmark for both years. Additionally, 2014 v2 will be available shortly.

    Lastly, this is a beta. If you see any odd or missing machine config information, have problems running on a localized version of Revit, or drop into interactive mode, let me know.

    So, knock yourselves out. And thanks again!
    Gordon

    2015
    2016
    iru69, kubsix and Alex Cunningham like this.

  8. #8
    Moderator
    "RFO 2012 Spelling Bee Champion"
    Alex Cunningham's Avatar
    Join Date
    December 8, 2010
    Location
    Northern California
    Posts
    3,547
    Current Local Time
    04:32 PM
    Are you going to update the original RFO Benchmark thread?

  9. #9
    Member colliert's Avatar
    Join Date
    May 2, 2012
    Location
    Little Rock, Arkansas
    Posts
    74
    Current Local Time
    06:32 PM
    Started the Revit 2015 Standard version of the test, and even after running through the test 3 times, it started another one. Is this normal? I thought it was supposed to run 3 and average the results, which I assume to mean it stops after the third time through?

    Let me add that the results did shown 3 Raw Data lists even after I terminated the 4th running test.

    Also, the results file output doesn't TAB so I can't grab the results and copy/paste into Excel without it placing all information on a line in one cell. Compare the results lines from the old version of the test output.
    Last edited by colliert; September 8th, 2015 at 06:24 PM.

  10. #10
    Member
    Join Date
    May 31, 2012
    Location
    Syracuse, NY
    Posts
    168
    Current Local Time
    07:32 PM
    Ran the expanded version on a 2015 Zbook15 and a older zelitebook

    throw the results in here or in a sticky benchmark thread?

Page 1 of 3 123 LastLast

Similar Threads

  1. Replies: 1
    Last Post: August 17th, 2015, 02:48 PM
  2. Replies: 0
    Last Post: March 29th, 2015, 10:45 PM
  3. RFO Benchmark: call for beta tester(s)
    By Gordon Price in forum Hardware and Infrastructure
    Replies: 6
    Last Post: July 25th, 2013, 07:36 PM
  4. Call for beta testers
    By Gordon Price in forum Hardware and Infrastructure
    Replies: 0
    Last Post: April 8th, 2012, 03:52 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •