Announcement

Collapse
No announcement yet.

Dr. StrangeSwitch or “How I learned to Stop Worrying and Use 64 bit Windows”

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Dr. StrangeSwitch or “How I learned to Stop Worrying and Use 64 bit Windows”

    First off, please be aware that nothing I say would pass muster with anyone who holds a CS degree. Or even a really savvy hobbyist. And some of it likely won't pass muster with IRU69. That said, I think I have come to understand this stuff well enough to make informed decisions, and if I can help anyone else do that also, while improving my ability to explain things (which is kinda my job) then Woot!

    Revit is a “Professional” software tool. As such, it must be run on “Professional” hardware and a “Professional” OS. With regards to Operating Systems, I am going to argue that 32 bit Windows is NOT a “Professional” environement any more. For even a small office I would argue that Windows 7 x64 and 4 GB of RAM is a baseline. That could be Windows 7 Home x64, but not Windows 7 Professional x32. At least for Revit, 64 bit is the only game in town. Knowing that now is a bad time to be fighting for hardware and software upgrades, I offer this information as ammunition for that fight.

    The 32 bit stepchild
    In 32 bit Windows (XP, Vista & Windows 7) with default settings, any given app only get's access to 2 GB of User Virtual Address space. That virtual address space references both physical System RAM and “Page File” on the hard drive, but the application just thinks it has 2 GB of dedicated “memory” to play in. The Operating System decides what of that needs to be in RAM, and what can be “paged out” to disk, and is constantly moving “pages” of memory back and forth as needed, for each application and itself. That is virtual memory in a nutshell.
    So, in 32 bit Windows any given app can crash if it tries to use more than 2 GB of address space, no matter how much memory is present as System RAM or swap file. The 3 GB switch makes 3 GB of address space available to applications, while only 1Gb is available to Windows itself (Kernal Address Space). This is because 32 bit Windows can only handle 4Gb of total address space, and at any given moment it is dealing with the virtual address space for itself and a single app. Without the 3 GB switch it is 2/2 (the default setting I first mentioned), with the 3 GB switch it is a 3/1 split, and using the /USERVA (User Virtual Address) switch you can tweak it further, to say 2.8/1.2.
    Now why would you want to tweak this value? Because of how 32 bit Windows maps graphics RAM mostly. Windows has to use part of the Kernal Virtual Address space to map to the RAM on the Graphics card as well as the Network card; as well as addressing it’s own memory needs, again some in RAM and some “paged”. How this Graphics RAM is mapped is dependent on the graphics driver, with some trying to map all the Graphics RAM at once, and others being able to access only a fraction of the Graphics RAM at once, and paging the address space for the rest. And there is no guarantee that the driver bothers to even look at the 3GB switch and USERVA settings and tune the way it works for the environment it is working in. In any case, whatever portion of the Kernal Address Space that is alocated for mapping Graphics RAM and Network card RAM is of course unavailable to map to System RAM. This the situation where you have 4 GB of RAM installed, but Windows reports that only 3.4 GB is “available”. That last .8 GB is lost to things like .5 GB of graphics ram mapping (a 512 MB card with a driver that maps all graphics RAM directly) and .1 GB for a gigabit network card, etc. And this mapped area also takes away from how much Kernal Virtaul Address space Windows has to run in. With the 3GB switch set, the graphics and network cards above might only leave .4 GB of that 1 GB of Kernal Address Space to run Windows in, and even WindowsXP needs at least .5 GB or so. Thus the tweaking with /USERVA to maximize address space for apps, while still allowing enough for Windows to function.

    So the main thing to realize is that the 3 GB switch is NOT about using more RAM, it is about using more Memory. Memory being comprised of both RAM and Page File. And Revit or Windows can have problems when Revit tries to access Memory beyond the configured User Virtual Address space, or Windows tries to access memory beyond the Kernal Virtual Address space, or either need more of their address space in System RAM than is available, or either need try to page out virtual memory and there is not enough room there. LOTS of ways to crash, all hard to track down. I have seen situations where a machine with 2 GB of RAM was stable with a big model as long as the 3 GB switch was enabled. Perhaps Revit was accessing 2.1 GB of Memory, and maybe only 1 GB was in RAM, while another 1.1 GB was paged. Without the 3 GB switch that would cause Revit to crash because it was trying to access memory beyond the 2 GB limit. Now with 4Gb and the 3 GB switch enabled it might be faster, with maybe 2 GB in RAM and only .1 GB paged. But with 4 GB of RAM and no 3 GB switch it will still crash, because of that 2 GB limit on addressable memory. Of course with a 512 MB graphics card it might then crash WITH the 3 GB switch, because now the 1 GB of kernel address space isn't enough for Windows. Edit USERVA to roll back to maybe 2.8 GB for User space and 1.2 GB for Kernel Space might then be the sweet spot. It all depends on the graphics driver, the network card, installed software, all sorts of things can impact how much Kernel address space is needed.

    64 bit Windows solves most of this
    The benefit of 64 bit Windows is that the address space goes to something like 2 TB, so there is no way you will crash from running out, even if Windows has half of that dedicated to itself, we are just FAR from Revit trying to access 1000 Gb of memory, as that would be a 50 GB file on disk! That alone means that 64 bit Windows can be more stable than 32 bit Windows on the same amount of RAM, even 2 GB. It may be slower because it has to page more, but it is more stable because it CAN address and page more. Add more RAM and it is stable and fast. Also, graphics RAM is mapped way up around 2 TB, beyond any phycical RAM we will see installed in our life times. So even a future 64 GB graphics card isn’t going to cause conflicts.
    Thus 64 bit Windows and 4 GB of RAM is to me a minimal machine these days, even for small residential work. Just no reason not to, and upgrades to 8 GB, 16 GB or more can be done as needed, with zero impact on other system dynamics. New graphics cards also cause no problems. But only if you have a 64 bit OS to start with.


    More on Virtual Memory
    The key idea with protected virtual memory as a concept is that each application is aware of itself and nothing else. So each app believes it has access to 2 GB of memory at addresses 0 to 2 GB in 32 bit Windows, or 0 to 1 TB in 64 bit Windows, and Windows makes sure that one app never writes data to a physical RAM location that another app is using, as that would cause the kind of crash we had back in the Windows 3.1 days when device drivers and apps shared memory and could walk all over each other when poorly written or the machine poorly configured.

    So at the moment maybe my (32 bit) Revit thinks it has stuff stored at 0-1.9 GB, while Chrome that is also running thinks it has stuff stored at 0-.25 GB. And Windows thinks it has stuff stored way up at 2.0-3.5 GB and beyond. Obviously Revit and Chrome can't actually both have stuff at .2 GB, so Windows uses a Lookup Table to map the location of each app's memory in use to an actual location, which could be in RAM if that data is in use, or could be in the Page File if not. So the lookup table may say that Revit address .2 GB is in physical RAM at location 2.8 GB, while Chrome address .2 GB is in my 16 GB Page File at location 1.8 GB, which is then mapped to a track & sector on the Hard Drive.
    And that Lookup Table takes up some RAM, which Windows doesn't want to waste so it puts currently unneeded bits of the lookup Table in the Page File as well. Thus, the more apps you have open but not currently doing anything, the more the Lookup Table is taking up Page File space.
    And since you don't actually have dedicated RAM for every app, Windows wants to ensure that as much physical RAM is available as possible, so memory locations that are not currently needed by an app are put in the Page File and the Lookup Table updated to reflect this. When an app askes for a particular memory location. Windows checks the lookup table to see if that data is in the PF or RAM. If the latter, it points the app to the data, if in the PF it moves the data out of the PF to some location in RAM, updates the Lookup Table and points the App to the proper location in RAM. Actually it translates the location because as far as the app is concerned, everything is at the virtual memory location and all the behind the scenes stuff is invisible.

    So if at one moment each app and Windows can address 2 GB of Memory, I might have...
    Windows .5 GB RAM, 1 GB PF
    Revit 1 GB RAM, .9 GB PF
    Chrome .125 GB RAM, .125 GB PF
    At which point I have 1.625 GB of physical RAM and 2.25GB of Page File in use, and with only 2 GB of physical RAM and a 4 GB Page File I am fine. But if my Page File was only 2 GB I would be using 1.875 GB of physical RAM and getting close to running low, while my PF would be maxed out. And since every open app uses some memory, and Windows will try to keep as much physical RAM free as it can, every open app, especially inactive ones, will eat up Page File. That's why the user with 15 copies of Internet Explorer open but minimized is actually causing themselves potential problems. All those minimized apps are in the Page File, potentially limiting the availability of PF for Revit.
    But Windows will only Page out memory when it is not in use, because getting data from the Page File, aka a Hard Fault, is SLOW compared to getting it from RAM. That is also why you can be slow and yet stable (enough PF but not enough RAM) or fast but unstable (enough RAM up to a point, and not enough Page File to free up RAM beyond a certain usage).
    And if I upped my RAM to 4 GB and my Page file to 8 GB, I could have three Revit sessions open with the same resource needs because my total RAM use would be 3.625 GB (3x 1 + .5 + .125) and PF would be 3.825 GB (3x .9 + 1 + .125).
    However, with only one session of Revit open and needing 1.1 GB of RAM and .9 GB of PF, I crash. I only need 1.725 GB of my 4.0 GB of physical RAM, and only 2.25 of my 16 GB swap file, but Revit needs 2.1 GB of total memory, and it only has access to 2.0 GB of addressable memory. So there is the condition where I can open three models fine, but one very slightly bigger model alone crashes, with plenty of RAM & PF still available.
    And if you use the 3 GB switch you still crash, because now Windows only has 1 GB of addressable memory, and the need is still 1.5 GB (.5 as RAM and 1 as PF). Use the USERVA setting to give all apps 2.4 GB and Windows 1.6 GB and you would be stable. For the moment. Throw a larger graphics card in, which Windows must map to directly out of it's Kernel address space, and Windows could need 1.9 GB of address space and I am crashing again. Or up the size of that one Revit model just a bit and again you go past the addressable memory limit.
    And of course all those numbers are actually changing thousands of times a second, so you can go for weeks without a hitch, or perhaps with a slow down due to thrashing (paging memory a lot because you don't have enough RAM) and then suddenly crash because some combination of things pushed you over a limit somewhere. And you can simultaneously be near the limit on addressable space for both Revit and Windows, as well as near the limit on physical RAM and/or Page File with a lot of apps open, and on any given day any one of those could cause your crash.
    Oh, that memory mapping of graphics RAM? Remember that different drivers handle it differently. So a driver update or different card could change the formula, require a little more kernel address space at certain times, and suddenly Windows is crashing, but only when Revit is running, and all because the graphics driver changed. But the trigger condition for the crash didn't occur till weeks after the driver update, and might actually be caused by the network card driver using more kernel address space when moving a big animation file, but only when the network isn't saturated and it can actually use a lot of bandwidth and it is configured to dynamically size it's packets. What?!
    Any doubts why IT folks would rather just get new 64 bit machines with lots of RAM and not have to troubleshoot that mess?

    Notes:
    File size on disk is only part of the picture. How the app uses RAM also impacts things. In Revit 2010 and earlier, an export to DWG would not release memory used to make each DWG until the entire DWG export was complete. Basically a memory leak due to code logic. So a 10 MB file could crash in 64 bit Windows with 32 GB of RAM and 64 GB of swap file. Just ask it to export 1000 DWGs and watch the available resources drop till you crash. And yet the "x20" rule suggests that you have no memory problems at all!
    2011 releases the memory used to create each DWG as the DWG is written to the hard drive, so even a huge file that is using 90% of the available RAM could export 20,000 DWGs with no issues. But things like Audit and upgrade still don't release memory till the process completes, so you need some memory headroom to complete those tasks. Headroom being address space, physical ram and swap file in 32 bit windows, and only the latter two in 64 bit Windows. But still, run out of any of the three and you crash. So there are still genuine bugs that can cause problems, even in 64 bit Windows.

    Also, the swap file is a shared resource, every open app, plus Windows, is dipping into whatever swap file you have allocated. So if Revit needs to use 800 Mb of swap file, and you have 2 GB allocated but only 600 MB is available to Revit, again you crash.
    The general rule in 32 bit Windows XP was to set the swap file to twice the physical ram, and make it a fixed size so it couldn't fragment. I have seen machines where this was set when the machine had 1 GB of RAM, and thus a fixed 2 GB swap file. A year later the machine had 3 GB of ram and still that 1 GB swap and was crashing, even with more RAM, because windows was trying to page to a very tiny swap file. Again, in 64 bit Windows I would recommend you just let Windows handle the Swap File size, and make sure you have as much RAM as needed to minimize swapping anyway. And make sure you have plenty of room on your hard drive for the swap file. Whan your work files are on a network this is easy even with the smallest hard drives available today. But even a sole proprietor with a single machine shouldn’t have a problem given the price for multi TB hard drives.

    For those wanting some other resources, the Factory has a post here that talks about some of this as well. They have just simplified things and talk about RAM where what they are really talking about is User Virtual Address space.

    And Wikipedia has some info here and here.

    Physical RAM limits for Windows 7 NEW 01.18.2011
    I found a reference from Microsoft with some good info about actual physical memory limits in Windows 7. It seems that Windows 7 Ultimate/Enterprise/Pro are limited to 192 GB of physical RAM. This is different from the User & Kernel Address Space, and no one is going to have anything close to 192 GB of RAM any time soon, so this is not a real limitation. However, some of us save some money by using Windows 7 Home Premium for single machine or small office situations, so it is worth noting that even in 64 bit, this flavor of the OS is limited to only 16 GB of RAM, which could actually be a problem for some people now. And Windows 7 Home Basic only supports 8 GB of RAM. But Windows 7 Home Basic is for "emerging markets" only, and most of us will not likely have access to it anyway.
    Also, according to this wikipedia article, the actual address space is 8 TB for User and 8 TB for Kernel. Yeah, not gonna be running out in this lifetime.

    And for the purists, you wouldn't really use a term like 1.2 GB as a memory location, that just seems to make conceptual sense to people who just want to understand the gist. A real address would be 1024 rather than 1.0 GB, and would be better expressed in hexadecimal not decimal.

    Lastly, my apologies for the length. I compiled this from an old thread from some time back, and probably should have edited more. But I didn’t.

    Gordon

    P.S. Thanks to DoTheBIM, patricks, swomack & eric.piotrowicz for the lively discussion that was the genesis for this information.
    Last edited by Gordon Price; January 18, 2011, 08:18 PM. Reason: Edited for consistent use of units
    Pragmatic Praxis

    #2
    Holy crap...
    Martijn de Riet
    Professional Revit Consultant | Revit API Developer
    MdR Advies
    Planta1 Revit Online Consulting

    Comment


      #3
      Its a one time read. Maybe. And only if you have 32 bit windows and are crashing. And I run off at the mouth. Going to :beer: now.

      G
      Pragmatic Praxis

      Comment


        #4
        It was a positive "holy crap". Very extensive and well-informed post. I would say thanks, but I don't want Mulkolms raft all over me. So I just edit some credit...
        I do have 32-bit on my laptop. Never did enable 3G-switch though, and I'm surely going to read this post before making a descision about that. Not just right now though, I have work to be done...
        Martijn de Riet
        Professional Revit Consultant | Revit API Developer
        MdR Advies
        Planta1 Revit Online Consulting

        Comment


          #5
          Gordon,

          Your post is a sensational and lucid discourse on operating systems. A must read for pretty much everyone (IRU69 excluded of course)

          Well done - if I could add more than one credit to your rep I would do so in a flash. :beer:

          Cheers,
          Last edited by Ian.Kidston; January 12, 2011, 09:28 AM. Reason: typo
          Ian Kidston
          http://allextensions.com.au

          Comment


            #6
            Seems like this will be a good read with my morning apple and tea.
            But first I have to get get a license of a whole nother sorts for this new Imperial country.
            Thanks for the effort Gordon.
            Great to have people in the user community shair their knowledge and thoughts.
            Cheers
            MW

            Comment


              #7
              Originally posted by RabbitHole View Post
              Seems like this will be a good read with my morning apple and tea.
              But first I have to get get a license of a whole nother sorts for this new Imperial country.
              Thanks for the effort Gordon.
              Great to have people in the user community shair their knowledge and thoughts.
              Cheers
              Spitpromise it won't be deleted... :thumbsup:
              Martijn de Riet
              Professional Revit Consultant | Revit API Developer
              MdR Advies
              Planta1 Revit Online Consulting

              Comment


                #8
                This is an excellent write up on how operating systems behave in regards to memory usage. Something I fight with constantly with clients when they refuse to upgrade to a 64 bit OS but wonder why the 32 bit OS crashes constantly in 100+ mb models. I may just copy/paste this to send to them, with appropriate credit and link information given.

                Comment


                  #9
                  If it draws people to RFO then I shall be a happy camper. If it makes life a little easier on IT Managers and consultants, I am down with that too.

                  Thanks!
                  Gordon
                  Pragmatic Praxis

                  Comment

                  Related Topics

                  Collapse

                  Working...
                  X