Accessing > 2 GB of RAM in Windows XP 32-bit

Ethan Winer wrote on 10/27/2010, 11:43 AM
Some of you may have seen my problem with Low Memory errors that prevented me from rendering a project built from a bunch of WMV files:

http://www.sonycreativesoftware.com/forums/ShowMessage.asp?MessageID=733226&Replies=9

I found the solution, and felt this information is significant to warrant a new thread if not a sticky.

When I first started having memory problems a few months ago, I bought an additional 2 GB of RAM for my Dell XPS computer, for a total of 4 GB. I know that Windows 32-bit can't access more than 3 GB, but having the extra 1 GB didn't seem to make any difference.

I was telling a friend my saga last night, and he told me that Windows will not use more than 2 GB until you add the switch /3GB to the C:\boot.ini file. So I did that, and son of a gun it fixed my problem! Apparently this important information is not very well known, though I can't for the life of me understand why Windows wouldn't just use the extra RAM automatically. Sheesh! So now I actually have the 3GB of RAM I paid for, which is finally enough to be able to render.

This is the last line of my boot.ini file, to which I added a space followed by the /3GB switch at the end:

multi(0)disk(0)rdisk(0)partition(2)\WINDOWS="Windows XP Media Center Edition" /fastdetect /NoExecute=OptIn /3GB

So if you have more than 2 GB of RAM and your boot.ini file does not contain /3GB, that RAM is not being used.

I hope others find this useful.

--Ethan

Comments

TheHappyFriar wrote on 10/27/2010, 12:14 PM
The issue is that Windows can still only use ~3gb. So you're talking away more from other programs & the OS.

I did that & had nothing but trouble. I made a separate boot option for it for testing.
Ethan Winer wrote on 10/27/2010, 12:50 PM
In my case the only two options are running Windows with only Vegas while I render, or not being able to render at all. But I'm now able to render successfully AND ALSO browse the web and do email etc, so for me it's a win-win. I can't speak for anyone but myself, and for my friend who told me this tip which he also needs to edit large video projects. (Though my friend uses Premier, not Vegas.)

More here:

http://blogs.technet.com/b/askperf/archive/2007/03/23/memory-management-demystifying-3gb.aspx

--Ethan
rmack350 wrote on 10/27/2010, 3:22 PM
As that Technet article points out, there's more to this than just setting that switch.

The switch adjusts how much virtual address space applications and the OS environment can use. Virtual address space is not exactly the same as RAM. You could think of it as RAM + page file.

What that switch does is increase the available virtual space for applications and *decrease* the available space for the OS. The caveat here is that the 32-bit application needs to be "large address aware", I don't know about VP10 (32-bit) but VP9 wasn't large address aware.

Okay, what does all this mean with less technobabble?

--- 32-bit Windows can address a total of 4GB of physical RAM, minus memory-mapped devices like BIOS, cards, PCI devices, etc. After all these have been assigned addresses by the BIOS there's usually about 3GB left, plus or minus. (More hardware often results in less addressable RAM)

-- So, in actual practice, 32-bit Windows has ~3GB of physical RAM to work with even if you install 4GB.

-- Under normal circumstances, 32-bit Windows allots 2GB of virtual address space to applications and another 2GB of virtual address space to the OS environment. Virtual address space is not RAM, BTW. You could have a total of 512 MB installed and Windows would still allot 2GB to each of the above.

-- 32-bit Windows will parcel out the actual physical RAM as needed. An application would never get more than 2GB, nor would the OS. Since there's really only about 3GB available each would probably get a bit less, depending on need.

-- Under normal circumstances, all 32-bit applications are compiled to use only the 2.0 GB of virtual space allotted to them. They won't normally use more even if it's available.

--Given all this, if all you do is set the /3GB switch then applications will still be limited to 2GB of *virtual space* because they won't take the extra available space. The difference is that the OS environment is limited to 1GB. This might make more physical RAM available to your application but the application still won't use more than 2GB. In the mean time the OS may be starved for memory.

So what about making Vegas Large Address Aware? It turns out that this is possible, and there is a HUGE thread about this on this site. Making Vegas and some of it's DLLs Larg Address Aware will allow it to use more than 2GB of virtual address space, and if you're running it under a 64-bit version of Windows it'll actually have more access to physical RAM. And if you're using 64-bit Windows then you don't need to muck about with that /3GB switch.

Rob Mack
Ethan Winer wrote on 10/28/2010, 11:39 AM
Thanks for the clarification Rob. As it happens, one of the things that Vegas support had me try previously was the CFF Explorer utility that patches EXE headers to access more RAM. So it may be that adding /3GB would not have solved my render problem had I not also patched Vegas9 a few week ago. However ...

I've watched Task Manager closely as I've had these problems, and now, after adding /3GB, I can see that the total "commit" slightly exceeds 2 GB. So even with nothing else running other than Windows' own background stuff, the total during rendering now exceeds 2 GB. So it makes sense to me that adding /3GB allows the sum of all apps to exceed 2 GB, which is why I can now render without error.

Anyway, my point in posting this thread was that /3GB solved a long-standing fatal problem for me, so maybe it will do the same for others.

--Ethan
TheHappyFriar wrote on 10/28/2010, 6:14 PM
The GPU memory counts to the 4gb max too from my understanding. I'd assume that if people use GPU rendering they'd have even less memory for Vegas to use.
rmack350 wrote on 10/29/2010, 9:12 AM
I don't really know if the switch would have solved 32-bit Vegas' problems without first making Vegas Large Address Aware. I think you get a lot more mileage out of "LAA" if you're running Vegas on a 64-bit OS because the OS has access to more RAM. Anyway, the two combined on Win32 probably gives Vegas more space to work with even if some of that space is paged to disk rather than being actual RAM.

The other thing the /3GB switch is doing for you is throttling back the amount of address space the OS can have. It's the other side of the seesaw in this story.

As for GPU memory consuming address space it's my understanding that it has a direct effect on the amount of system memory remaining. Install a 1GB graphics card and you lose a 1GB of addressability from system RAM. (That might be a flawed understanding. Maybe it's not a 1:1 relationship.)

The 32-bit address space runs from 00000000 to FFFFFFFF. When the system boots the BIOS starts assigning addresses for "memory mapped devices" at FFFFFFFF and then works its way downward. After that, RAM is assigned the remaining addresses starting at 00000000 and working upward until the BIOS runs out of addresses.

If there's more RAM than address space then one of two things happens. If the BIOS isn't 64-bit capable then the surplus RAM just doesn't get addresses. If the BIOS *IS* 64-bit capable then the surplus RAM is assigned to 64-bit addresses after FFFFFFFF. If you're running a 64-bit OS then the OS can *see* those addresses above FFFFFFFF and it can use that RAM, if it's a 32-bit OS then those addresses don't exist as far as the OS knows.

The BIOS always assigns addresses to hardware starting at FFFFFFFF, never higher. It does this because the hardware must be accessible to a 32-bit OS. It uses the top end of the address range because most PCs didn't come equipped with that much RAM so there was never any conflict.

(Some of the details above might be slightly fanciful. Feel free to point it out if you like.)

---------------------------

The moral of any of these stories is that it's time to migrate to 64-bit Windows.

Hogwild wrote on 11/5/2010, 7:10 AM
ok, so now I'm completely confused as this is all way over my head.

I have 4gb ram on a 32 bit xp pro system, so if I add more physical ram (I can expand to 16), then I will not see any increase in vegas performance on the timeline?

When I playback single HDV clips on my timeline, my cpu is running between 65% & 70% most of the time at half resolution in the preview window. If I add a title or crossfade, it goes to 100% and stuttery playback. I have to set the preview to quarter rez to get smooth playback. Will adding memory help this or not? I'm new with vegas.

Should I be capturing my HDV files differently?
Ethan Winer wrote on 11/5/2010, 10:07 AM
> I have 4gb ram on a 32 bit xp pro system, so if I add more physical ram (I can
> expand to 16), then I will not see any increase in vegas performance on the timeline?

Right, more than 4 GB memory will not help Vegas. However, adding the /3GB switch to the c:\boot.ini file as explained in my first post will give 1 GB more memory to Vegas. So that might well help. At least it sure helped me!

--Ethan
rmack350 wrote on 11/5/2010, 11:36 AM
Sorry, Hogwild, my comments have been too long. That's probably not going to change since there's like 6 or 7 little ideas to get for this.

Short answers with less exposition:

I have 4gb ram on a 32 bit xp pro system, so if I add more physical ram (I can expand to 16), then I will not see any increase in vegas performance on the timeline?

32-bit Windows can only "see" the first 4GB and can only use ~3GB of that. Upgrade to 64-bit Win7 if you want to have access to more than 3GB.

Additional RAM will probably not improve playback performance, unless you're hitting the page file a lot during playback. Watch Task Manager to see if the page file is growing during playback. If so, then more RAM would help. You either manage what you have or upgrade to Win64.

When I playback single HDV clips on my timeline, my cpu is running between 65% & 70% most of the time at half resolution in the preview window. If I add a title or crossfade, it goes to 100% and stuttery playback.

That answers your question. Your CPU is running flat out but not keeping up. You need a faster CPU. If RAM were the problem then the CPU wouldn't be pegged.

Rob
BigD4077 wrote on 11/5/2010, 11:54 AM
I farted around tossing more ram at an XP-32-bit system running a Pentium-4 dual core. Waste of time as the CPU is the bottleneck. Build a new system, or if your on a new platform (sounds like your not) upgrade to Windows 7 64-bit. You'll be happy, HaPpY, HAPPY.

http://forums.creativecow.net/readpost/24/918305
Hogwild wrote on 11/5/2010, 12:12 PM
Thanks, I think I get it now.

BTW, my pagefile usually hits about 1.95gb steadily during playback. I think I'm going to manage with what I have for a while, but will be looking to upgrade in the near future.

Thanks again!
rmack350 wrote on 11/5/2010, 2:40 PM
my pagefile usually hits about 1.95gb steadily during playback

That seems high to me but maybe it's fine. One thing to keep in mind is that if you've set your RAM Preview setting very high then you could be forcing your PC to use the page file excessively. If the PC has to use the page file while playing out the timeline then that could be making things choppy. Drop the setting to something very modest and see how things go.

Vegas caches frames to RAM while it plays the timeline. You can set this so high that something has to be cached to disk. It might not be Vegas itself but if the OS is struggling to find memory it could be writing to and reading from the page file way too much.

Rob
Hogwild wrote on 11/5/2010, 3:52 PM
Well, I did set my ram setting high because whenever I tried to build a dynamic ram preview at half rez, it would only give me about two seconds of rendered playback. Increasing the ram setting allowed me longer ram previews. My max is 1024mb. - I guess it's a trade off.

I did set the ram preview back at sony's default of 350mb and it lowered the page file substantially during playback. With a single HDV clip, at half rez, it runs at 1.27gb with CPU usage at 50% - 55%. With a title overly and cross fades between clips, it ramps the page file to 1.35gb and CPU usage tops out at about 95%.

When I drop to quarter rez, everything lowers quite a bit more, but I can't stand operating at quarter rez. I used to edit on Avid Liquid with this machine and it used background rendering. I wish Vegas did that.

Should I consider capturing my HDV clips with another application or converting them with clipwrap or something like that. I would hat to have to convert every clip. I am doing a season of high school football games with 80-100 clips each.

Would capturing with Intensity Pro & Neoscene be worth while or advantageous?
rmack350 wrote on 11/5/2010, 4:21 PM
We're hijacking the thread here but this question leads back to getting access to more RAM for Vegas.

You can modify 32-bit Vegas and some of it's DLLs to be Large Address Aware. That would allow them to use up to 3GB. Then, since you're running Win32 you need to also set the /3GB switch so that Win32 will let Vegas use more RAM. That should allow you to run with a higher preview RAM setting. It's a cheap solution that would at least help you out with RAM previews.

Once you've gotten to the point of doing this you should probably just bite the bullet for a CPU upgrade, more RAM, and Windows 7 64-bit. Maybe your current PC would support an upgrade to a quad core CPU but you'd get more bang (for more bucks) from a complete upgrade to something more current. I think the CPU you've got right now is on the wrong side of the bleeding edge for HDV.

I haven't been trying to answer the HDV question. I don't think you should jump through that kind of hoop for this, but someone else may feel differently.

Rob
BigD4077 wrote on 11/5/2010, 4:57 PM
I configured my XP32 with 4GB of ram of which the OS could see 3.4gb. It made no appreciable difference other than being able to create a longer ram preview. That's useless as well as it takes too damn long to create the preview.
Hogwild wrote on 11/5/2010, 6:20 PM
I appreciate everyones responses, I guess I'm at a wall and will have to deal with it for a while. If I do any real upgrading, it will be with a complete new system. I don't like trying to upgrade current my pc too much, because I'll end up crossing that line of $$$ for performance ratio.

Maybe Santa will be nice this year....?