Thread: RFOBenchmark

RFOBenchmark is here.
This is an extension of the AUBench tool, with some added tests, verified function for all three verticals, and modularized to make adding tests and porting to new versions of Revit much easier.

Please post your results in
RFOBenchmark_2011
RFOBenchmark_2012
RFOBenchmark_2013



Some things to be aware of...

• The RFOBenchmark folder doesn't need to be renamed. Just copy the folder from the ZIP to your disk and run the _RFOBenchmark.vbs file. You will then be asked what vertical you want to test, progress will be shown in a popup between tests, and a final popup will let you know the benchamrk is done and let you open the results file.
  2013: With 2013, you have more vertical options. RVT is OneBox, and RVT-S is OneBox with ONLY Structural UI enabled.
• 2013: The new test also asks if you want to run full screen or not. PLEASE run full screen. The impact is not trivial. if we all do full screen tests, we can then look at the screen size information reported in the hardware section and make much more meaningful comparisons.
• The test can ONLY be run via the VBS file. Running the journals manually will not produce the results you want.
  Simply double click the _RFOBenchmark_2011.vbs or _RFOBenchmark_2012.vbs file or _RFOBenchmark_2013.vbs file.
• The test is longer than before. Up to 15 minutes. I think the expanded data is worth it, and since most people won’t be running it regularly, hopefully this won’t be a problem.
• Read the readme. Especially the part about using _Kill Test.vbs
• The test attempts to quantify GPU performance. However, the results you see are in no way a complete picture. Over time we will see what the data tells us. It may be that the results are in fact misleading, and we decide to remove that part of the test. In the mean time, use the benchmark as a datapoint for decision making, but remember that nothing is as meaningful as a butt-dyno comparison of two different cards, on your machines, on your projects.
  2013: The new test expands a bit on the GPU test, which when combined with running full screen is getting close to being a meaningful metric. Not there yet, but closer!
• On the GPU test, you will notice that Consistent Colors is very fast on the non hardware accelerated test in 2011, but on a par with the other views in 2012. This is because CC doesn’t do shadows in 2011, so the major hardware acceleration benefit is lost. In 2012, where shadows are available in Consistent Colors, the results match expectations.
• 2013: Lastly, I regret to say that they new test will not run in 32 bit Windows XP. If I can find a way to make this happen I will update ASAP, but for the moment it looks like that is not going to happen. That said, I hope that Win XP x32 has become an edge case and this won't affect too many people.

Please keep all discussions about the tools in this thread, and use the Result thread ONLY for results.

One last note. Some folks have had issues with the downloads being corrupted. If you have this issue, I recommend you try the download with a browser that is NOT Internet Explorer. Maybe IE9 is better, but even IE8 sometimes results in corrupted download. Chrome and Safari for Windows have both worked well for me, and others have reported good results with Firefox.

Thanks all, I hope you find RFOBenchmark useful.
Gordon

Great job Gordon! Rep. for you

I´ll post my results shortly

It still throws my work machine into interactive mode at the exact same spot the other one did.

Would that be a corrupt download, or is that some other issue? All I have is IE8 at work. IT won't let me use chrome or firefox.

Dan, I´ve just sent the Benchmark by mail... maybe that´ll work...

Gah, I was really hoping for a nice clean working benchmark for everyone. Dan, can you PM me with a copy of the journal file that was made as the benchmark ran? I'll see if that sheds any light.

Gordon

Very slick! Fun stuff.

Originally Posted by Gordon Price

• The test attempts to quantify GPU performance. However, the results you see are in no way a complete picture. Over time we will see what the data tells us. It may be that the results are in fact misleading, and we decide to remove that part of the test. In the mean time, use the benchmark as a datapoint for decision making, but remember that nothing is as meaningful as a butt-dyno comparison of two different cards, on your machines, on your projects.

Unfortunately, I don't think this part of the test is saying anything about the GPU (video card). The test that uses the HWA is so minimal that every video card is going to be about the same. The test that doesn't use the HWA is pretty much just benchmarking the CPU (as far as I would guess).

p.s. - sorry I didn't think to mention it when you first had me take a gander at it (I wasn't really paying any attention to what was actually being benchmarked, only that it didn't crash).

Iru69,
I am sure you are correct. The numbers are useful, in that they tell you that shadows and HA go hand in hand. But beyond that I think we will want to have a discussion here about what the numbers really mean.

BUT, I find it VERY interesting that Consistent Colors AND Realistic View are faster than Hidden line, with and without shadows, with and without hardware acceleration (I have run those variations of the test seperately, in 2011 and 2012, and will do a writeup of the results soon). My guess is that the "new" view types where done with optimized code to start with, and the Hidden Line view still uses older (original?) unoptimized code. I know people who specifically avoid Consistent Colors thinking hidden line must be faster, so just for that data point I am happy with the GPU test. But it isn't saying anything about the performance of a particular card, which I had hoped for. And thus my extended caveat on the GPU test specifically.

Gordon

Sure thing. Let me try the one Munk sent.

Well, it hasn't shown up yet. So.......uh...........how do I attach to a PM or a RFO email for that matter? lol

Well, I thought you could do it in a PM as well. But here in the thread, click the Go Advanced button, then you will get a slick little Attach Files section lower down the screen. Click Manage Attachments, Add Files, Select Files. Choose your file the click Upload File.

Thanks,
Gordon

Yeah, I knew I could do it in a thread, but you said PM. LOL
I wasn't sure if it was because you didn't want everyone to see it for some reason.

Here it is. Have fun! lol