Paul Yao
The latest version of the Microsoft WindowsÔ graphical environment, Version 3.0, has three different memory managers. From a memory management point of view, then, Windows1 is actually three products: real mode Windows, standard mode Windows, and 386Ô enhanced mode Windows. Each exploits different Intel 80x86 processors, yet a common architecture allows a program to run unchanged under all modes.
In Windows, you can determine the current operating mode by looking at the Program Manager's About box. A program can learn the current mode by calling the GetWinFlags routine. This Windows library routine provides a 32-bit value that contains a set of 1-bit flags. A code fragment showing one way to use these flags can be found in Figure 1.
All three memory managers are accessed by running the start-up stub, WIN.COM. If you don't choose a mode, Windows picks one for you. On 8086/8088 machines, real mode is selected. Standard mode is selected on 80286 machines or 80386 machines with less than 2Mb of memory. On 80386 systems with at least 2Mb of RAM, Windows chooses 386 enhanced mode. Alternatively, you can force Windows into a specific mode from the command line. Real mode is started by specifying:
win /r
On 80286 and higher CPUs, standard mode is requested with:
win /s
And on computers with 80386 and higher CPUs, 386 enhanced mode is requested with:
win /3
Each mode has unique characteristics, which I will describe. I'll start by reviewing how the 80x86 segmented address scheme works.
Segmented Addressing
Windows runs on top of DOS2, which means that Windows runs on members of the Intel-86 family of processors. This family includes the 8086, 8088, 80188, 80186, 80286, 80386SX, 80386, and i486Ô microprocessors. Each processor addresses memory in segments. Under real mode Windows, segment values are converted directly into physical addresses. Standard and enhanced mode Windows run the processors in protected mode to gain the memory management support built into the higher-end CPUs.
While there are important differences between real mode and protected mode operation, both access memory in segments. Windows and its programs inherit this segmented orientation, so that segments are the organizing units of memory used by Windows.
There are basically four types of segments available to a Windows program: code segments, a single default data segment, dynamically allocated segments, and resource segments. Code segments are managed by the Windows dynamic linking mechanism, so that code can be demand-loaded from disk when needed and discarded when it is not.
The default data segment contains a program's static data, stack, and local heap. Since Windows supports only a single default data segment, programs must be built using the small or medium compiler model. While a single data segment seems to limit a Windows program to a maximum of 64Kb of data, this apparent limitation is easily overcome through the use of dynamically allocated segments. This third category of memory segment
is allocated by calling GlobalAlloc.
The fourth type of segment, resources, is created for read-only data. Resource segments are used to store various user interface objects, such as menus, dialog box templates, and cursors. Resource segments are also used for GDI objects like fonts and bitmaps. A resource is often in a discardable segment and can be purged from system memory when not needed. This helps Windows conserve memory.
Addressing Memory
All of the Intel processors mentioned address memory with a 16-bit segment identifier paired with a 16-bit offset value. (The 80386 and higher CPUs can use a 32-bit offset, but the Windows Version 3.0 API only supports a 16-bit offset.) The segment identifier describes the segment being addressed, while the offset value describes the byte in that segment.
This two-part memory address is sometimes referred to as a logical address. Software must access memory using a two-part logical address. Together the CPU and the operating system translate a logical address into a physical address. The physical address is a 20-bit, 24-bit, or 32-bit address (depending on the CPU) used to reference the physical memory installed in the system.
In enhanced mode, a logical address is first translated into a 32-bit virtual address. The paging hardware built into the 80386 and 80486 CPUs then resolves this reference into a physical address in RAM.
A diagram of the translation process for all three of the Windows operating modes is shown in Figure 2.
To calculate the maximum amount of physical memory a CPU can address, count the number of address lines. That number is the power of two for the maximum addressable memory. With twenty address lines, the 8088 can address 220 or one megabyte of memory. The maximum addressable memory for each member of the
Intel-86 family is shown in Figure 3.
Although the 80386 and 80486 processors have 32 address lines, only 24 are available on the standard AT-style bus. For this reason, Windows limits itself to 16Mb of physical memory when running on these systems.
The segmented address scheme is the reason that Windows programmers use the NEAR and FAR keywords. At any point in time, the current code and data segments are referenced using the code segment register (CS) and the data segment register (DS). To reference code or data outside the default segments requires FAR addressing. A FAR address requires more work, and is therefore slightly less efficient, than NEAR addressing. FAR addressing requires both the segment and the offset value. When a program references code or data inside one of the current default segments, the simpler NEAR addressing can be used. It is more efficient because only one value-the offset-must be specified.
Windows Segment Attributes
In Windows, a memory segment is managed according to an attribute selected when the segment is allocated. Figure 4 lists the basic segment attributes.
New Windows programmers may find it hard to imagine a discardable memory object, but it is at the heart of Windows' dynamic-linking mechanism. You can think of dynamic linking as a sophisticated overlay manager, which permits any code segment to be purged from system memory. When code segments are needed again, they are loaded from disk into memory in a completely transparent manner. In this respect, dynamic linking is far superior to traditional overlay management software. Discardable segments are also used for segments containing resources, which can be purged from memory when not needed.
The presence of moveable memory is not really new in the world of operating systems. It is, however, somewhat unusual for the movement of memory to be detectable from within a program. Real mode operation requires the presence of moveable memory.
When a moveable or discardable segment is allocated, Windows issues a memory handle to identify the segment. To read or write in such a segment, the program calls the GlobalLock routine to retrieve the object's address. When a segment is not locked, the memory manager is free to move the segment to minimize the fragmentation of the global heap. GlobalLock also increments an object's lock count to prevent the object from being moved or discarded. (An important exception in protected mode is that GlobalLock does not increment the lock count of moveable objects. As you'll see, the protected mode hardware allows a moveable object to maintain its logical address even when it has been moved.)
A fixed memory segment can be allocated for situations in which the first two types of memory cannot be used. This creates the most restrictions for the Windows memory manager, so it should be reserved for special circumstances, such as interrupt service routines in device drivers. Fixed memory objects restrict the memory manager's ability to reorganize system memory, which it must be able to do to recover the memory that would otherwise be lost due to fragmentation of the global heap.
These three types of segments operate in a relatively straightforward manner in real mode Windows. In the other two modes, things get a little fuzzy. The built-in memory management of protected mode in the higher end CPUs takes over some of the tasks that Windows itself must do in real mode. As I discuss the Windows operating modes, I'll describe the unique twist put on these three types of segments by each mode. However, a description of the segment-locking process is needed first.
Segment Locking
As I've mentioned, when you work with moveable or discardable memory segments, you must first call GlobalLock, which asks Windows to tie down a segment. If you are an experienced Windows programmer, you can continue working with dynamically allocated segments in the ways
you are familiar with. However, Version 3.0 makes some new things available.
There are now four methods for locking a segment, as listed in Figure 5. The first method, a simple lock, is the most common way to fix segments. A simple lock is created with a call to GlobalLock, which returns a far pointer. The pointer is guaranteed to be valid until the lock is disabled with a call to GlobalUnlock. In real mode, when you access dynamically allocated segments, you'll lock the segment for a very short period of time-only as long as it takes to process a single message. Otherwise, you risk creating sandbars in the global heap that prevent the memory manager from reorganizing memory when it needs to.
In protected mode, the situation is a little different. As you'll see, the Windows memory manager has help from the hardware. In protected mode, you can lock a segment once, and leave it locked until you are finished using the memory. Windows protected mode memory managers can still move memory around, and even swap memory to disk in enhanced mode.
The other three types of locks are for special circumstances (see Figure 5). For example, in real mode, to lock a segment for a relatively long period of time, you should wire the segment by calling GlobalWire instead of GlobalLock. A wired segment is moved first so that it doesn't obstruct the movement of other memory objects.
The third type of segment lock is for protected mode, to fix a segment at a given physical address. When a segment has a fixed physical address, a low-level lock prevents it from moving in the linear address space of protected mode Windows. Ordinarily, segments can be moved in protected mode Windows, but the hardware memory management fools you because the segment's logical address doesn't change when the segment moves. A fixed physical address lock prevents such movement. This is the most secure way to tie down a segment in standard mode but is overkill in most applications. It is required mainly for device drivers that need to make sure that the physical address of a dynamically allocated segment doesn't change.
The fourth type of segment lock prevents a segment from being swapped to disk in 386 enhanced mode. The virtual memory system of this mode swaps memory pages to disk in a manner that is entirely transparent to your program. However, when a segment is page-locked, it cannot be swapped to disk. Page-locking also fixes the physical address. This is the most restrictive type of lock in Windows enhanced mode. Like the fixed physical address lock of standard mode, a page-locked segment is overkill for most applications but is useful for device drivers that need to be present in memory at all times. Consider the delay that might be caused if critical parts of the display or keyboard drivers were swapped to disk. Or imagine the problems that would be caused if the kernel's swapping code itself were swapped to disk.
Inside the module definition file (DEF) of Windows programs and dynamic-link libraries (DLLs), there are entries that describe the memory disposition of code and data segments. For example, an entry like the following makes code moveable and discardable by default:
CODE MOVEABLE DISCARDABLE
In general, segments in a DLL that are FIXED inherit the most secure locking of each of the operating modes in Windows. For example, if a DLL's default data segment were declared fixed in real mode and standard mode, it would be fixed in the physical address space:
DATA FIXED
In enhanced mode, it would be page-locked to prevent it from being swapped. The rationale is that device drivers are implemented in DLLs, and therefore they should get this service automatically.
In a Windows program, however, the FIXED attribute is treated a little differently. For example, assume an application has the following in its DEF file:
CODE FIXED
DATA FIXED
In real mode, both code and data segments would be fixed. But in protected mode, the Windows memory manager overrides these flags and does not make them fixed or page-locked. The reason is simple: the Intel protected mode mechanism already gives fixed logical addresses to code and data segments without invoking the more restrictive types of memory locks.
As this article goes to press, there is a known problem in Windows 3.0 running in protected mode. It involves the use of the GMEM_FIXED flag when dynamically allocating a segment. If you allocate a segment like the following from an application or a DLL, the segment will be allocated as fixed and page-locked.
hmem = GlobalAlloc (GMEM_FIXED, dwSize);
This is a problem because these higher segment- locking attributes should only apply when such a segment is allocated from a DLL. From an application, it should be sufficient to give it a fixed logical address and still allow the physical address to change. This will be altered in a future version of Windows.
Real Mode
Experienced DOS programmers will find much that is familiar in Windows real mode. Also, if you started programming for Windows with Version 1.x or Version 2.x, everything is pretty much the same in real mode under Windows Version 3.0. For those previous versions, Windows operated only in real mode.
All 80x86 chips can run in real mode. In real mode, the size of the physical address space is 1Mb. For the sake of compatibility, the higher-end members of this family inherit this limitation when running in real mode. This is true even for chips that have more address lines than the 8088/8086 CPUs.
Real mode gets its name from the fact that memory is accessed using real, physical addresses. As depicted in Figure 6, the logical-to-physical address translation process involves multiplying the segment identifier by 16 (shifting it left by 4 bits) to create a 20-bit base address. The offset is added to this to access a particular byte or set of bytes in the memory segment.
The three Windows segment attributes-fixed, moveable, and discardable-were created with real mode in mind. They allow the Windows memory manager, in cooperation with Windows programs, to manage memory effectively.
Even though a segment's logical address is identical to its physical address, memory in real mode can be moved because Windows does not provide a segment's address at segment allocation time. Instead, a memory handle is issued. As long as a segment isn't locked, the memory manager is free to move or discard segments as needed. To retrieve the address of a segment, a program locks the segment. The address is guaranteed to be valid until the segment is unlocked.
The segment allocation and segment-locking mechanism work in an identical manner in real and protected mode. Thus, it is a simple matter to create programs that work the same way in all operating modes.
But, as you'll see when I discuss protected mode operation, there are shortcuts that can be taken by a Windows program in protected mode. Before looking at the two protected modes in Windows, there is one more real mode topic to explore: expanded memory.
Real Mode and Expanded Memory
Expanded memory is memory that is made available according to the Expanded Memory Specification (EMS). It describes a bank switching mechanism for using memory that is temporarily mapped into unused portions of the real mode address space.
From the beginning, Windows made use of EMS memory. In version 1.x, the use of EMS memory was limited to support for DOS applications running under Windows. When EMS was available, Windows 1.x swapped idle DOS applications to EMS memory instead of swapping them to disk.
In Version 2.0, introduced in the fall of 1987, Windows started using EMS memory for Windows applications themselves (see "EMS Support Improves Microsoft Windows 2.0 Application Performance," MSJ, Vol 3., No. 1). The support under real mode Version 3.0 is identical to Windows 2.x EMS support.
In Version 3.0, real mode Windows uses EMS memory, when available, to increase the size of the address space for each Windows program. An EMS bank is automatically allocated for every Windows program (see Figure 7). As a program runs, the Windows memory manager treats the EMS memory as an extension of the global heap. In fact, since each program has its own EMS bank, each program has its own private extension to system memory that does not have to be shared with other programs. EMS memory is largely transparent to programs, since Windows works behind the scenes to switch the correct EMS memory into view. The result is less competition for system memory and better overall system performance.
When running in real mode, Windows uses EMS memory if it finds an EMS driver in the system. While EMS memory helped relieve the memory shortage in Version 2.x, it no longer plays a crucial role in Windows memory management.
Banked EMS memory is abandoned by the other two operating modes, because they have access to extended memory. Extended memory is not banked; instead all extended memory is simultaneously accessible. Extended memory provides a more flexible approach to breaking the 1Mb boundary that has hounded Windows and DOS programmers for so long.
Standard Mode
From the beginning, it was intended that Windows would eventually run in protected mode, as it does in Windows Version 3.0. The most obvious difference between protected mode and real mode operation is that a larger address space is available: up to 16Mb of RAM can be addressed.
Standard mode is sometimes described as the way Windows takes advantage of the protected mode of the 80286 processor. In fact, standard mode can run on the 80386 and higher processors as well. Even though standard mode is associated with the less powerful 80286 CPU, you might prefer it over enhanced mode. Standard mode actually runs a little faster than enhanced mode because it doesn't rely on the virtual device drivers. These drivers provide some nice features, such as allowing DOS programs to run in a window. But they slow things down a bit, particularly for those programs that access the disk heavily.
Understanding how the protected modes of the Intel processors access memory will help you understand exactly how Windows uses these modes. As in real mode, protected mode uses a segmented addressing scheme. But the segment portion of the address does not refer to a physical memory location. Instead, it contains an index into a segment lookup table. In protected mode, there are two types of lookup tables: the global descriptor table (GDT) and the local descriptor table (LDT).
Intel designed its protected modes flexibly, enabling different operating systems to use these two types of tables in several different configurations. For example, a single GDT could be used to reference all of an operating system's shared memory. Each program could then have a private LDT with which to access private segments.
If you have programmed in OS/2 systems, you are probably aware that
OS/23 uses the GDT itself and gives each process its own LDT. This minimizes the possibility of one program adversely affecting the operation of other programs. A program can only access data through its own LDT.
While this helps protect programs from each other, it requires a little more work to share data. Data sharing in OS/2 requires the duplication of selector information. Each process that needs access to a shared segment must have an entry in its LDT. To simplify things, if a segment is allocated as shareable, its segment selector is the same for all processes in the system. A single segment is shared, then, by creating an entry in the LDT of every process that wishes to access a segment.
Windows 3.0 sets up protected mode addressing using another configuration. The GDT is reserved for the protected mode memory manager, and a single LDT is created for all Windows libraries and applications. Every segment allocated for Windows programs and every segment allocated for DLLs is referenced from one systemwide LDT. A single LDT means that all programs share a common address space, which makes it easy to share segments between programs. But watch out!
In a future version of Windows, it is likely that every program will be allocated a private LDT, just as is done in OS/2. When this happens, each Windows program will have a private address space. At that time, sharing data will require one of three methods: the Clipboard, Dynamic Data Exchange, or DLLs.
Figure 8 depicts how a logical address is translated into a physical address using the LDT. The segment identifier, which is called a segment selector in protected mode, contains a descriptor table index. When a segment is referenced, its descriptor table information is loaded into a set of registers known as the shadow registers. The name comes from the fact that these registers are entirely invisible to application software.
Among the values loaded into the shadow registers are a segment's base address, its size (also known as its limit), and various flags. Translating a logical address into a physical address involves adding the descriptor table's base address to the offset. Since the location of a segment is not part of the segment address itself, but instead is kept in the descriptor table, segments can be moved freely in protected mode without changing their logical address. This represents an important improvement over real mode addressing, in which the segment's logical address is the same as its physical address.
Another improvement over real mode involves automatic error checking. Several checks are performed to make sure that the contents of memory are protected from unauthorized access. For one thing, when a program addresses memory, the CPU automatically checks for a valid segment selector. A valid segment selector is one that has a valid entry in one of the current descriptor tables (one LDT and one GDT table is current at any time). If a valid entry is not found, a protection exception occurs.
At the hardware level, a protection exception creates a hardware interrupt 13H General Protection Fault (also known as a GP fault). Windows responds to this interrupt by displaying a message box with the fatal error message: "Unexpected Application Error," sometimes known as a "UAE Error."
Other errors can cause a GP fault. For example, if a program tries to reference memory beyond the limit of a segment, a GP fault occurs. This prevents a program from overrunning the boundaries of a segment and possibly overwriting another program's data.
Another cause of GP faults is attempted access to a protected segment, a violation of the privilege levels built into protected mode addressing. Associated with every segment is a 2-bit privilege flag that defines
four privilege levels: 0
through 3.
The highest privilege level, 0, is given to the most trusted software. OS/2 programmers know that the OS/2 kernel runs at privilege level 0, also known as Ring 0. OS/2 programs, and a large part of Presentation Manager, run at Ring 3. The rest of Presentation Manager, and all display and printer device drivers, run at Ring 2.
In Windows 3.0, the protected mode control programs (DOSX.EXE in standard mode and WIN386.EXE in enhanced mode) run at Ring 0. Windows programs, and the Windows DLLs, run at Ring 1. If a future version of Windows runs at Ring 3, Windows will have room to grow. It will also minimize the differences between the protected mode environment of OS/2 and the protected mode of Windows and ease porting Windows programs to
OS/2.