It simplifies the base operating system. Moving each API set (Win32, MS-DOS, 16-bit Windows, POSIX, and OS/2) into a separate server removes conflicts and duplications from the operating system base and makes it easy to add new APIs.Helen Custer
In the fall of 1988, Microsoft hired Dave Cutler to lead an effort to design an advanced operating system. Cutler, well-known as a key developer of the VMSÒ operating system and MicroVAXÔ while at Digital Equipment Corporation, assembled a team of engineers to design what is now known as the Windows NTÔ (New Technology) operating system. Early in 1989, Microsoft reviewed the operating system specifications Cutler’s group had defined.
The design goals established for Windows NT include:
Extensibility The code must comfortably grow and change as market requirements change.
Portability As dictated by market goals, the code must move easily from one processor to another.
Reliability and robustness The system should protect itself both from internal malfunction and external tampering. It should behave predictably at all times and applications should not be able to affect the operating system negatively.
Compatibility While Windows NT should extend existing technology, its user interface and application programming interfaces should be compatible with existing MicrosoftÒ systems.
Multiprocessing and scalability Applications should be able to take advantage of the broad range of computers available today. Users must be able to run the same application on single- and multiprocessor computers.
Distributed computing Windows NT should be “network-aware.” It should be able to distribute its computing tasks to other computers on the network to give users more power than is present on any single machine. It should employ both local and remote computers as needed without user intervention.
POSIX compliance In the mid- to late 1980s, U.S. government agencies began specifying POSIX as a procurement standard for government computing contracts. POSIX is an acronym rather loosely defined as “a portable operating system interface based on UNIXÒ.” To meet the government’s procurement requirements, Windows NT was designed to provide an optional POSIX application execution environment.
Government-certifiable security The U.S. government also sets computer security guidelines for many types of government applications. The security guidelines specify such capabilities as protecting one user’s resources from another’s and establishing resource quotas to prevent one user from garnering all the system resources. The initial target for Windows NT security is the Class C2 level, defined as providing “discretionary (need-to-know) protection and, through the inclusion of audit capabilities, for accountability of subjects and the actions they initiate.” This means that the owner of a system resource has the right to decide who can access it and the operating system can detect when data is accessed and by whom.
U.S. government security levels range from level D (least stringent) to level A (most stringent), with levels B and C containing several sublevels each. Windows NT, while initially written to support the C2 security level, can be enhanced for higher security levels in future releases.
Performance The system should be as fast and responsive as possible on each hardware platform.
Windows NT consists of a privileged Executive and a set of nonprivileged servers called protected subsystems. The term privileged refers to a processor’s modes of operation. Most processors have a privileged mode in which a program may use all machine instructions and system memory is accessible, and a nonprivileged mode in which certain instructions are disallowed and system memory is inaccessible. In Windows NT, the privileged processor mode is called kernel mode and the nonprivileged processor mode is called user mode.
Usually, an operating system executes in kernel mode only and application programs execute in user mode except when they call operating system services. The design of Windows NT is unique because its protected subsystems execute in user mode like applications. This allows protected subsystems to be modified or new ones to be added without affecting the integrity of the Executive, making for an extensible operating system.
Windows NT was designed for easy porting. It is written almost entirely in C. The developers selected C because it is standardized and because C compilers and software development tools are widely available. Assembly language is used only for parts of the system that must communicate directly with the hardware (the trap handler, for example) and for components that require optimum speed (such as multiple precision integer arithmetic). Nonportable code is carefully isolated within the components that use it. A single Windows NT kernel is developed for a given processor, which is then used for all platforms based on that processor.
In addition to processor dependency, platform dependency is another portability concern. Windows NT encapsulates platform-dependent code inside a DLL known as the hardware abstraction layer (HAL). The HAL abstracts platform-specific hardware such as caches with a layer of software.
Windows NT can be ported easily to machines that use 32-bit linear addresses and provide virtual memory capabilities. It can move to other machines as well, but at a much greater cost.
Windows NT was designed to be robust, responding predictably to error conditions, even those caused by hardware failures. It also actively protects itself and its users from accidental or deliberate sabotage by user programs.
Structured exception handling is used by Windows NT for capturing error conditions and responding to them uniformly. The operating system or processor issues an exception whenever an abnormal event occurs; exception handling code, which exists throughout the system, is automatically invoked to respond to the condition, ensuring that no undetected error wreaks havoc on users or on the system itself.
Software compatibility is a complicated subject. Whether a new operating system is binary- or source-code-compatible with an existing system depends on several things. Foremost among them is the architecture of the new system’s processor. If the processor uses the same instruction set (with extensions, perhaps) and the same size memory addresses as the old, then binary compatibility can be achieved.
Binary compatibility is not so easy between processors from different vendors. Each processor architecture ordinarily carries with it a unique machine language. This means that cross-architecture, binary compatibility can only be achieved if an emulation program converts one set of machine instructions to another. Without an emulator, all applications moving from the old architecture to the new must be recompiled and relinked (and likely debugged).
Through protected subsystems, Windows NT provides execution environments for applications other than the Win32Ô API, its primary programming interface. When running on IntelÒ processors, the protected subsystems supply binary compatibility with existing Microsoft software, including the MS-DOSÒ operating system, 16-bit WindowsÔ, OS/2Ò, and LAN Manager. On RISC processors manufactured by MIPS, binary-level compatibility is achieved for MS-DOS and 16-bit applications for the MicrosoftÒ Windows1 operating system by using an emulator; source-level compatibility is provided for OS/2 and LAN Manager programming interfaces. Windows NT also provides source-level compatibility with POSIX applications that adhere to the POSIX interfaces defined in IEEE Standard 1003.1.
In addition to compatibility with programming interfaces, Windows NT supports existing file systems including the MS-DOS file system (FAT), the OS/2 high-performance file system (HPFS), the CD-ROM file system (CDFS), and the new, recoverable Windows NT file system.
Each component of Windows NT was designed with an eye toward performance. Performance testing and modeling were done for performance-critical parts of the system. System calls and page faults were carefully optimized to ensure the fastest possible processing speeds.
The protected subsystems that perform operating system functions must frequently communicate with each other and with the privileged part of the operating system. To guarantee that this communication does not hinder their performance, a high-speed message-passing mechanism called the local procedure call (LPC) facility is included as an integral part of the operating system.
Each protected subsystem that provides an operating system environment (environment subsystem) was designed to maximize the speed of frequently used system services.
An operating system is a complex program, a layering of detail upon detail. A unifying model is required to ensure that the system can accommodate its required features, without compromising its design goals.
The Windows NT design was guided by a combination of several models. Windows NT uses a client/server model to provide multiple operating system environments (initially, Windows, MS-DOS, OS/2, and POSIX) to its users. It uses an object model to uniformly manage operating system resources and dispense them to users. A third model, symmetric multiprocessing (SMP), allows Windows NT to achieve maximum performance from multiprocessor computers.
Operating system code can be structured in a number of ways. One approach, particularly common in smaller systems such as MS-DOS, is to create the operating system as a set of procedures and allow any procedure to call any other. This “monolithic” structure enforces no data hiding in the operating system; assumptions about how the system fits together are embedded throughout the operating system code (see Figure 1). Extending such a system can be tough, because modifying a procedure in one place can induce bugs in seemingly unrelated parts of the system. In all but the simplest monolithic operating systems, applications are separated from the operating system itself. While the operating system code runs in a privileged processor mode (kernel mode) with access to system data and to the hardware, applications run in a nonprivileged processor mode (user mode), with a limited set of interfaces and limited access to system data. When a user-mode program calls a system service, the processor traps the call, then switches the calling thread to kernel mode. When the system service completes, the operating system switches the thread back to user mode and allows the caller to continue.
Figure 1 A Monolithic Operating System
A different approach is to divide the system into modules and layer them (see Figure 2). Each module provides a set of functions that other modules can call. Code at any particular layer calls code only at lower layers. On some operating systems, such as VMS or Multics, the layering is even enforced by hardware (using multiple, hierarchicalprocessor modes).
Figure 2 A Layered Operating System
One advantage of a layered structure is that each layer of code is given access only to the lower-level interfaces and data structures that it requires, limiting the amount of code that wields unlimited power. This structure also allows the operating system to be debugged starting at the lowest layer, one layer at a time. Layering also makes it easier to enhance the operating system; an entire layer can be replaced without affecting other parts of the system.
A third operating system model is called the client/server model. The operating system is divided into several processes, each of which implements a single set of services (memory services, for example). Each server runs in user mode, executing a loop that checks whether a client has requested one of its services. The client, which can be either another operating system component or an application program, requests a service by sending a message to the server (see Figure 3). An operating system kernel (or microkernel) running in kernel mode delivers the message to the server, the server performs the operation, and the kernel returns the results to the client in another message.
Figure 3 A Client/Server System
The client/server approach results in an operating system whose components are small and self-contained. Because each runs in a separate user-mode process, a single server can fail, and perhaps be restarted, without crashing or corrupting the rest of the operating system. Furthermore, different servers can run on different processors in a multiprocessor computer or even different computers, making the operating system suitable for distributed computing environments.
The theoretical model shown in Figure 3 depicts a client/server system in which the kernel consists only of a message-passing facility. In reality, client/server systems fall along a spectrum, some doing very little work in kernel mode and others doing more. For instance, the Mach operating system, a contemporary UNIX-like system that uses a client/server architecture, implements a minimal kernel that comprises thread scheduling, message passing, virtual memory, and device drivers. Everything else, including various API sets, file systems, and networking, runs in user mode.
The structure of Windows NT borrows from both the layered model and the client/server model. The kernel-mode portion of Windows NT is called the Windows NT Executive. It’s made up of a series of components that implement virtual memory management, object (resource) management, I/O and file systems, interprocess communication, and portions of the security system. For the most part, these components interact with one another in a modular rather than a layered fashion: each component calls the others through a set of carefully specified internal functions.
The layered operating system model comes into play in the Executive’s I/O system, which uses layers of drivers to process a single request. The layered model also appears in the bottommost portions of the Executive—the kernel and the HAL. All other parts of the Executive are layered on top of these two components. The kernel performs low-level operating system functions, much like those found in small kernel, client/server operating systems: thread scheduling, interrupt and exception dispatching, and multiprocessor synchronization. It also provides a series of functions and basic objects that the rest of the Executive uses to implement higher-level constructs. Below the kernel is the HAL DLL, a layer of platform-specific code that protects the kernel and the rest of the Executive from platform differences. The HAL manipulates hardware directly.
Windows NT uses the client/server model primarily to provide APIs and the facilities that one ordinarily regards as an operating system environment (see Figure 4). The Win32 server provides the user interface and is fundamental to the system’s operation. (The terms server and subsystem are used interchangeably—Ed.) The other servers plug into the Executive and can be loaded as desired, with several in operation at once. The servers communicate with application processes using a message-passing facility in the Executive.
Figure 4 Windows NT Client/Server Structure
Using the client/server model has several benefits:
It simplifies the base operating system. Moving each API set (Win32, MS-DOS, 16-bit Windows, POSIX, and OS/2) into a separate server removes conflicts and duplications from the operating system base and makes it easy to add new APIs.
It improves reliability. Each server runs in a separate process, partitioned into its own memory, and is thus protected from other processes. Further, because the servers run in user mode, they cannot directly access hardware or modify memory in which the base operating system is stored.
It lends itself well to a distributed computing model. Since networked computers are based on a client/server model and use messages to communicate, local servers can easily send messages to remote machines as well. Clients don’t need to know whether certain requests are being serviced locally or remotely.
Bertrand Meyer, in his book Object-oriented Software Construction (Prentice-Hall International Ltd., 1988), characterizes an operating system as a program that has no top. As with other large software systems, it is difficult to identify a single “main program” that drives an operating system. Therefore, instead of attempting to design such a system from the top down, object-oriented software design focuses first on the data that the software must manipulate to do its job. For an operating system, such data takes the form of system resources: files, processes, blocks of memory, and so on.
The goal of designing systems around data is to create software that is easy and cheap to change. The ability to modify software is important when you consider the oft-quoted statistic that 70 percent of software cost is attributable to maintenance.
One way that object-oriented software minimizes change is to hide the physical representation of data within objects. An object is a data structure whose physical format is hidden behind a type definition, a set of services that may be requested on data of that type, and a set of formal properties the object embodies.
Though not strictly an object-oriented system as Meyer defines it, Windows NT uses objects to represent system resources and to hide their structure from user-mode code. Any system resource that can be shared by more than one process, including files, shared memory, and physical devices, are implemented as objects and manipulated using object services. This approach lessens the impact of changes that will occur in the system over time. If a hardware change, for example, forces a change in the operating system, only the object that represents the hardware resource and the services that operate on the object must change; code that merely uses the object remains the same. Likewise, when the system needs to support new resources, a new object is created and added to the system without disturbing existing code.
Basing an operating system on objects has several advantages. First, the operating system accesses and manipulates its resources uniformly. It creates, deletes, and references an event object in the same way that it does a process object—by using object handles. And because each resource is an object, keeping track of resources is easy to do just by monitoring the creation and use of objects.
Second, security is simplified because all objects are protected in the same way. Every time someone tries to access an object, the security system intervenes and validates the operation, whether the object is a process, a section of shared memory, or a communication port.
Third, objects are a convenient and uniform paradigm for sharing resources between two or more processes. Object handles are used to manipulate all types of objects. Two processes share an object simply by opening a handle to it. The operating system can keep track of how many handles are open to an object to determine whether it is still in use. The operating system can then delete the object when all handles to it are closed.
Multitasking shares a single processor among multiple threads. When a computer has more than one processor, the multitasking model must be upgraded to a multiprocessing model. While a multitasking operating system appears to execute multiple threads at once, a multiprocessing operating system actually does it, one on each of its processors. Multiprocessing operating systems support either asymmetric or symmetric hardware (see Figure 5).
Figure 5 Multiprocessing Operating Systems
Asymmetric multiprocessing (ASMP) operating systems typically select one processor to execute operating system code, while other processors run user jobs. ASMP operating systems are relatively easy to create by extending existing uniprocessor operating systems. They are especially well-suited to running on asymmetric hardware, such as a processor with an attached coprocessor or two processors that don’t share memory. However, it’s hard to make ASMP operating systems portable. Hardware from different vendors, and even different versions of hardware from the same vendor, tends to vary in type and degree of asymmetry. Either the hardware vendors must target their products for specific operating systems or the operating system must be substantially rewritten for each hardware platform.
Symmetric multiprocessing (SMP) systems, including Windows NT, allow the operating system to run on any free processor or on all processors simultaneously, sharing memory between them. This approach makes better use of multiple processors because the operating system itself can use a significant percentage of a computer’s processing time, depending on the application(s) running. Running the operating system on only one processor can tax that processor, leave others idle, and decrease the system’s throughput; as the number of processors on the system increases, operating system activities are more likely to bottleneck. SMP systems are also less likely to experience down time than ASMP systems because operating system code can execute on other processors if one fails. Finally, since symmetric hardware is implemented similarly from vendor to vendor, it is possible to create a portable SMP operating system.
Unlike ASMP systems, SMP systems are often designed and written from the ground up, because they must adhere to strict coding guidelines to ensure correct operation. In addition, numerous performance considerations arise in multiprocessing systems that do not occur in single processor systems.
Several Windows NT constructs are crucial to its success as a multiprocessing operating system:
The ability to run operating system code on any available processor, or on multiple processors at once. With the exception of its kernel component, which handles thread scheduling and interrupts, operating system code can be preempted (forced to give up a processor) when a higher priority thread needs attention.
Multiple threads of execution within a single process. Threads allow one process to execute different parts of its program on different processors simultaneously.
Servers that use multiple threads to process requests from more than one client simultaneously.
Convenient mechanisms for sharing objects between processes and flexible interprocess communication capabilities, including shared memory and a message-passing facility.
Windows NT can be divided into two parts: the user-mode portion of the system, which consists of the Windows NT servers, and the kernel-mode portion, the Executive (see Figure 6).
Windows NT servers are called protected subsystems because each one resides in a separate process whose memory is protected from other processes by the Executive’s virtual memory system. Because the servers do not automatically share memory, they communicate by passing messages. The black lines in Figure 6 represent paths that messages may take between clients and servers or between two servers. These messages pass through the Executive, but for simplicity’s sake, that is not shown in the figure.
As mentioned before, the Executive is an operating system engine capable of supporting any number of server processes. The servers give the Executive its user and programming interfaces and provide execution environments for various types of applications.
Figure 6 Windows NT
Each protected subsystem provides a set of APIs. When an application or another server calls an API, a message is sent to the server that implements the API via the Executive’s local procedure call (LPC) facility, a message-passing mechanism. The server replies by sending a message back to the caller.
Windows NT has two types of protected subsystems: environment subsystems and integral subsystems. An environment subsystem is a user-mode server that provides the APIs for a specific environment (say, Windows or POSIX). The most important environment subsystem is Win32, which of course supplies the 32-bit Windows APIs to application programs. The Win32 environment subsystem provides the graphical user interface and controls all application I/O. Windows NT will also have POSIX and OS/2 environment subsystems. These subsystems supply their own APIs to application programs, but use the Win32 subsystem for I/O.
The remaining protected subsystems, the integral subsystems, are servers that perform important operating system functions. Several integral subsystems have come and gone during the development of Windows NT, but one has remained throughout: the security subsystem.
The security subsystem runs in user mode and records the security policies in effect on the local computer. It keeps track of which users have special privileges, which system resources are audited for access, and whether audit alarms or audit messages should be generated. The security subsystem also maintains a database of information about user accounts, including account names, passwords, any groups the user belongs to for security purposes, and any special privileges the user owns. It accepts user logon information and initiates its authentication.
The Windows NT Executive is the kernel-mode portion of Windows NT. The Executive consists of a series of components, each of which implements two sets of functions: system services, which environment subsystems and other Executive components can call, and internal functions available only to components within the Executive (see Figure 6).
The Executive is fundamentally different from the environment subsystems. It does not run continually in a process of its own. Rather, it runs in the context of an existing process by taking over an executing thread when important system events occur. For example, when a thread calls a system service and is trapped by the processor, or when an external device interrupts the processor, the kernel gains control of the thread that was running. The kernel calls the appropriate system code to handle the event and then returns control to the code that was executing before the interrupt.
Executive components maintain independence from one another, each creating and manipulating the system data structures they require. Because the interfaces between components are carefully controlled, it is possible to completely remove a component from the operating system and replace it with one that operates differently. As long as the new version implements all of the system services and internal interfaces correctly, the operating system runs as before. Maintaining the operating system is also an easier task because NT Executive components interact in predictable ways.
The responsibilities of the Executive components are described in Figure 7.
Windows NT is a portable operating system, designed to limit the amount of code that relies on a particular hardware architecture. Some processor-specific code (for instance, Intel 486 or MIPS R4000) is required, however, and it is located in the lowest layers of the kernel, with smaller portions found in the virtual memory manager (VMM). These components, particularly the kernel, hide processor differences from the rest of the operating system.
Platform-dependent code is located in the HAL. This code is written by individual computer manufacturers. Device drivers also contain device-specific code, of course, but they avoid processor- and platform-dependent code by calling kernel routines and HAL routines.
Figure 7 Executive Components
| Object manager | Creates, manages, and deletes Executive objects. |
| Process manager | Creates and terminates processes and threads. It also suspends and resumes the execution of threads and stores and retrieves information about Windows NT processes and threads. |
| Local procedure call (LPC) facility | Passes messages between a client process and a server process on the same computer. LPC is a flexible, optimized version of remote procedure call (RPC), a communication facility for clients and servers across a network. |
| Virtual memory manager (VMM) | Implements virtual memory, a memory management scheme that provides a large, private address space to each process, and protects each process’s address space from the others. When memory use is too high, it pages selected memory contents to disk and reloads them when they are used again. |
| Security reference monitor | Enforces security policies on the local computer. It guards operating system resources, performing run-time object protection and auditing. |
| Kernel | Responds to interrupts and exceptions, schedules threads for execution, synchronizes the activities of multiple processors, and supplies lower-level objects and interfaces that the rest of the Executive uses to implement higher-level objects. |
| I/O system | A group of components responsible for processing input from and delivering output to a variety of devices. It includes the following parts: |
| I/O manager | Implements device-independent input/output facilities and establishes a model for Executive I/O. | |
| File systems | Windows NT drivers that accept file-oriented I/O requests and translate them into I/O requests bound for a particular device. | |
| Intermediate drivers | Drivers that provide intermediate I/O processing between a file system or other high-level driver and a device driver. | |
| NT Executive device drivers | Low-level drivers that directly manipulate hardware to write output or retrieve input from a physical device. | |
| Cache manager | Improves the performance of file-based I/O by storing the most recently read disk information in system memory. It uses the VMM paging facility to automatically write modifications back to the disk in the background. |
| Hardware abstraction layer (HAL) | A layer of code between the Executive and the hardware platform on which Windows NT is running. It hides hardware-dependent details of I/O interfaces, interrupt controllers, multiprocessor communication, and so on. Executive components maintain maximum portability by calling the HAL routines when they need platform-dependent information, rather than accessing hardware directly. |
With minor exceptions, Windows NT does not appear to be a unique, new operating system from a user’s point of view. It looks like Windows 3.1 and runs Windows-based programs. Underneath its user interface, however, it is radically different. Let’s look at how the pieces of Windows NT fit together, beginning from its user interface and working into the Executive.
Windows NT requires users to establish an account and to log on to that account before they access the system. Each user account has associated with it a security profile, which is a collection of security-related information stored in a system database. The security subsystem employs this information to authenticate users (that is, to verify that they are who they say they are). The components of the system involved in logon are highlighted in Figure 8.
Figure 8 Logging On
A security system process, called a logon process, sits waiting for user input. Several logon processes can be active, each one monitoring a separate class of logon devices (a keyboard/mouse combination, for example, versus a network connection). A thread in the process detects when a user attempts to access the system and prompts the user for an account name and password.
From there, the logon process passes the user’s information to the security subsystem, which checks it against its security database. If the logon is authentic, the subsystem creates an object that uniquely identifies this user in all subsequent transactions. The object, called an access token, is the key to security in Windows NT; it determines which system resources the user’s threads may access.
Once the user’s identity is established, the security subsystem creates a process, attaches the user’s access token to it, then passes the process to the Win32 subsystem, which runs the Win32 Program Manager in the process. The user has now established a logon session. Windows NT supports both local and remote logons, and a server machine running Windows NT is likely to have numerous logon sessions active at once.
Once an interactive user successfully logs on to Windows NT, the Win32 subsystem takes control of the screen. In its first release, Windows NT looks like, and is compatible with, Windows 3.1 (see Figure 9).
Figure 9 Windows NT
The Win32 environment subsystem not only controls the video display, but also the keyboard, the mouse, and other input devices attached to the machine. In addition, it is a server for Win32-based applications, implementing the Win32 API.
Windows NT can run applications other than those based on the Win32 API. The Win32 subsystem does not control the execution of these applications. When the user runs an application the Win32 subsystem does not recognize, the subsystem determines what type of application it is, then either calls another subsystem to run it or calls code to initialize an MS-DOS environment in which to run the application (see Figure 10).
Figure 10 All I/O Passes Through the Win32 Subsystem
The environment subsystems each supply a set of APIs that its client applications use. The Win32 subsystem supplies 32-bit Windows APIs, the OS/2 subsystem supplies OS/2 APIs, and so on. Applications cannot mix APIs from different subsystems because each operates differently. A file handle created by the Win32 subsystem will not mean anything to the POSIX subsystem, for example.
MS-DOS and 16-bit Windows emulation are not supplied by environment subsystems per se. Instead, a Win32-based application called a virtual DOS machine (VDM) provides a complete MS-DOS machine environment. MS-DOS and 16-bit Windows-based applications are run within VDMs, which run as clients of the Win32 subsystem (and also call native services from time to time). Each MS-DOS application runs within its own VDM process. Windows 3.1 is considered one MS-DOS application, so all Windows 3.1-based applications share one VDM.
Because the Win32 subsystem handles all video output, the other environment subsystems and VDMs must direct the output of their applications to the Win32 subsystem for display. The VDM running 16-bit Windows-based applications translates their output calls into Win32 calls and sends them via messages to the Win32 subsystem. The OS/2 and POSIX subsystems, as well as any VDMs running MS-DOS-based applications, direct their application’s character-mode output to the Win32 subsystem; this output is displayed in character-mode windows called consoles.
An environment subsystem can have many client applications running. Each subsystem keeps track of its clients and maintains any global information the client applications share.Though several subsystems and VDMs may be running, Win32 is the only environment that makes itself visible. To the user, it appears that Windows runs all the applications.
Environment subsystems (and VDMs, in some cases) implement their APIs by calling Windows NT native services, the system services provided by individual components of the Executive. The VMM supplies memory allocation and deallocation services, for example, while the process manager provides services to create and terminate processes and threads. As Figure 11 illustrates, when a subsystem calls a native Windows NT service, hardware detects the call and traps into the Executive. The service then runs in kernel mode.
Figure 11 Native System Service Call
Because the native services are used by different environment subsystems, they must be general, even primitive. They must be flexible, without unnecessary built-in constraints. They must not generate side effects that might conflict with the diverse needs of the environment subsystems.
One way in which the native services are flexible is in their ability to act on any process the caller specifies. The caller supplies a handle to a process and the service operates on that process. For example, a subsystem can call a native service to create a thread or allocate memory for one of its client processes. Of course, most normal processes cannot do such operations to other processes. Environment subsystems have powerful access tokens that grant them control over their clients. Protected subsystems, DLLs, and components of the Executive are the primary users of native Windows NT services.
Many, perhaps most, native Windows NT services are object services. That is, they do something to some object in the Executive. A thread opens a handle to an object, then uses that handle when calling services to operate on the object.
Shareable resources, including processes, threads, files, shared memory, and so on, are implemented as objects in the Executive. This allows the operating system to take advantage of the similarities between resources and to use common code to manage them wherever possible. The Windows NT object system is a focal point for several types of resource management tasks: resource naming, placing limits (called quotas) on the amount of resources each process can use, sharing resources between two processes, and securing resources against unauthorized access.
Environment subsystems frequently call object services to create, open a handle to, manipulate, or delete objects. If the user starts a Win32-based application, a spreadsheet for example, the Win32 subsystem calls the Windows NT process manager to create and open a handle to a process object, the process in which the spreadsheet will run. The process manager calls the object manager to create a process object and a thread object. Similarly, if the user saves a new spreadsheet, the Win32 subsystem calls an I/O service to create and open a handle to a file object representing the file in which the document is stored. The I/O system calls the object manager (see Figure 12).
Figure 12 Creating Windows NT Objects
Much of the resource management in Windows NT takes place when some process creates and/or opens a handle to an object. For example, when a process (in this case, the Win32 subsystem) creates an object, it can optionally give it a name. Giving an object a name makes it available for sharing by other processes. A process that wants to share the object simply retrieves the object’s name by calling the Windows NT object manager, then opens a handle to the object.
Objects are allocated from operating system memory. To keep any one process from using too much system memory, processes are charged a set amount of their quota each time one of their threads opens a handle to an object. If a process exhausts its quota limits, the object manager does not allow it to open any more object handles.
In addition to managing resources and facilitating resource sharing, the object system is a focal point for resource security. When a thread opens a handle to an object, the Windows NT security system is activated. Each object has attached to it a little database of information, called an access control list (ACL), that specifies which processes’ threads can access the object and what they can do to it. When a thread opens a handle to an object, it cites the operations it wants to perform. Suppose it wants to open a file for read access. The security system checks whether the process in which the thread resides is allowed read access to the file object in question. If so, the object manager returns an object handle containing read access. The caller can then use the handle to read from that file. If the caller also needed write access to the file, then it could either have requested both read and write when it opened the first handle or it could open a second handle for write access. Since a thread must open a handle to an object before it can do anything to it, and since opening a handle invokes the security system, no thread can bypass security.
On Windows NT, application programs run in an operating system environment that behaves like Windows, MS-DOS, POSIX, or OS/2. The challenge is to allow all types of applications to run without their being rewritten and without them bumping into each other in memory.
Each environment subsystem provides a view of memory that corresponds to what its applications expect. Underneath the environment subsystems, the Executive has its own memory structure that the environment subsystems access by calling native Windows NT services.
The memory architecture of Windows NT is a virtual memory system based on a flat, linear address space accessed via 32-bit addresses. A process’s virtual address space is the set of addresses available for the process’s threads to use. At run time, the VMM translates, or maps, the virtual addresses into physical addresses where the data is actually stored. By controlling the mapping, the operating system can make sure that individual processes don’t bump into each other or overwrite the operating system.
Each process’s virtual address space is 4GB (232 bytes), with 2GB reserved for program storage and 2GB reserved for system storage. 4GB is, of course, much larger than the amount of physical memory likely to be available on ordinary machines. When physical memory gets full, the VMM transfers some of the memory contents to disk. Paging data to disk frees some of physical memory so it can be used for other things. When a thread accesses a virtual address that has been paged to disk, the VMM loads the information back into memory from disk.
In Windows NT, the operating system resides in high virtual memory and the user’s code and data reside in low virtual memory (see Figure 13). A user-mode thread cannot read or write system memory directly. A portion of the system memory, called the nonpaged pool, is never paged to disk and is used to store some objects and other important data structures. Another portion of system memory, called the paged pool, is paged to disk. All of user memory can be paged.
Figure 13 Virtual Memory in Windows NT
As with memory, environment subsystems provide whatever I/O facilities their applications expect. They implement those individual facilities by calling native NT I/O services.
The native I/O system uses an asynchronous I/O model, but it provides system services that allow environment subsystems to use either a synchronous or asynchronous model. Asynchronous I/O allows a caller to request an I/O operation, then do other work while the device finishes transferring the data. The I/O system automatically notifies the caller when the I/O is complete so the caller can do any subsequent processing. Since I/O devices are generally much slower than processors, a program that does a lot of I/O can often improve its performance by using asynchronous I/O.
Windows NT supports multiple file systems, including FAT, HPFS, and the new Windows NT file system. The capabilities of this new file system include:
File system recovery allows for quickly restoring disk-based data after a system failure
The ability to handle (ridiculously) large storage media, up to 264 bytes or approximately 17 billion GB
Security features, including execute-only files
Unicode filenames
POSIX support, including hard links, case-sensitive names, and information regarding when a file was last changed
Extensibility features, such as transaction-based operations to support fault tolerance, user-controlled version numbers for files, multiple data streams per file, flexible options for file naming and file attributes, and the ability tosupport popular file servers such as NovellÒ, AppleShareÒ, and SunÒ NFS
The I/O manager allows device drivers and file systems (which it also views as device drivers) to be loaded dynamically into and out of the system, based on the needs of the user. Drivers are modular and can be layered one on top of another, which, for example, allows different file systems to call the same floppy or hard disk driver to access files (see Figure 14).
Figure 14 File I/O Drivers in Windows NT
The layered driver model also gives the user the ability to insert additional drivers in the hierarchy. For example, logical file system drivers or fault tolerant drivers can occupy intermediate levels in the driver hierarchy.
When LAN Manager was first created for OS/2, it ran as an application, implementing network services and network security features. In contrast, networking was built into Windows NT from the system’s inception. Network support was an important consideration and the Executive’s interfaces reflect the needs of network servers and file systems.
Windows NT provides access to files on the network through file system drivers called network redirectors. Redirectors accept requests for remote files and direct them to a network server on another machine. Different redirectors can be loaded into the system to support a variety of networks.
Windows NT is meant to be a truly multilingual operating system, a solid foundation for developing and using international applications.
To facilitate localization, the Win32 subsystem provides national language support (NLS) APIs that give applications access to culturally correct string comparisons, collation tables for sorting different languages, date, time, and currency formatting functions, and functions to determine what locale is in effect and what other locales are present on the system. (For general information on implementing multilanguage software, see “Adapt Your Programs for Worldwide Use with Windows Internationalization Support,” MSJ, Vol. 6, No. 6.) In addition, APIs can convert between the international code set used by Windows NT and other common code sets in use. Both the Win32 subsystem and the C run-time library offer their own APIs based on NLS. Using these facilities allows applications to support localization, without having to duplicate the substantial database of information (tables, code sets, and so on) required to do so.
The lowest layer of localization support is code sets. The U.S. has traditionally employed ASCII for representing data. For other countries, ASCII is inadequate because it lacks common symbols and punctuation.
The International Standards Organization established a standard code set called Latin1 (ISO standard 8859-1), which defines codes for all the European characters omitted by ASCII. Microsoft Windows uses a slight modification of Latin1 called the Windows ANSI code set. Windows ANSI is a single-byte coding scheme.
The set of letters required to write in a particular language is referred to as a script. The same script is often used for several languages. Windows ANSI, and other single-byte coding schemes, can encode enough characters to express the letters of most Western scripts. But Asian scripts, for example, cannot be encoded using a single byte. Asian languages are typically stored using a double- or multi-byte coding scheme, in which some characters are represented by a 1-byte sequence while others are represented by a 2-, 3-, or 4-byte sequence. Such schemes require complicated parsing algorithms to determine the storage width of a particular character. Furthermore, a proliferation of different code sets means that a particular code might yield entirelydifferent characters on two different computers, depending on the code set the computers use.
To address the problem of multiple coding schemes and to accommodate a more comprehensive set of scripts, Windows NT employs the new Unicode standard for data representation. Unicode, a 16-bit coding scheme, can represent 65,536 (216) characters. This is enough to include all the characters of all languages in use today, as well as several archaic or arcane languages with limited applications such as Sanskrit and, eventually, Egyptian hieroglyphs (see Figure 15). Unicode also includes representations for punctuation marks, mathematical symbols, and dingbats, with plenty of room left for future expansion.
Figure 15 Layout of Scripts with Unicode
Unicode separates the “essence” of a character from the formatting information used to display it. Each code corresponds to one (and only one) character; formatting information is applied to Unicode characters to display them in various styles and shapes.
While the Win32 subsystem provides both ANSI and Unicode string APIs, Unicode is the native code set of Windows NT . All character strings in the system including object names, path names, and file and directory names are represented with 16-bit Unicode characters. The Win32 subsystem converts any ANSI characters it receives into Unicode strings before manipulating them. It converts them back to ANSI, if necessary, upon exit from the system.
Unicode effectively removes all limitations on the set of characters that Windows NT can represent. Since Unicode establishes a unique code for every character of every script, Windows NT can ensure that the roundtrip character translation into and out of the system is accurate.
Structured exception handling is a special architecture supported by Windows NT. Exceptions are synchronous errors or atypical events that cause the execution of code outside the normal flow of control. Unlike interrupts (which are generated from an external source), exceptions occur when a program executes a particular code sequence, and they are reproducible.
For example, a call to the C function malloc typically results in malloc allocating memory and returning a pointer to it. The exceptional condition occurs when some problem, such as a lack of available memory, causes the allocate to fail. In this case, the function returns a NULL pointer.
Returning a special value to indicate an exception is a common but primitive form of exception handling that has some drawbacks. First, a programmer must religiously check the return value and either act on any errors or propagate them to a higher layer of software. If one layer omits the check, then bugs can surface in unrelated parts of the program. Second, the code becomes cluttered with if/then/else clauses to handle the atypical cases rather than the typical ones. Third, information about why the operation went wrong may not be easily available to the code that must address the problem.
Exceptions can be detected by either hardware or software. For example, hardware generally detects divide-by-zero exceptions, while software detects memory access violations. Structured exception handling is the method used in Windows NT for processing both hardware and software exceptions, using the control structure of a programming language (hence the name). Structured exception handling allows any block of code to determine what type or types of exceptions it wishes to guard against and to register a special code sequence, the exception handler, that is executed if such exceptions occur within the guarded block of code.
Following is a simple C function that includes an exception handler. It is a modified version of the standard strlen function, which returns the length of a null-terminated string.
/* safelen: return length of string s, even if the
/* string pointer was bad */
int safelen(char *s)
{
int count = 0;
try {
while (*s++ != ‘\0’)
/* possible access violation */
count++;
return (count);
}
except (GetExceptionCode() = = ACCESS_VIOLATION ?
EXCEPTION_EXECUTE_HANDLER :
EXCEPTION_CONTINUE_SEARCH)
{
/* pointer was bad/string not terminated */
return (count);
}
}
The normal strlen function merrily plods through memory a character at a time until it finds a NULL character. But if the string is not NULL terminated or if the string pointer is invalid, strlen can terminate unexpectedly with an access violation exception.
This modified version of the code captures the exception and returns a valid value (not necessarily correct, just valid), rather than terminating the program. A new C keyword “try” marks the beginning of the block of code that might cause an access violation. If an exception occurs in this block, control is transferred to the “except” keyword that is followed (in parentheses) by an exception filter. The exception filter allows the programmer to execute the exception handler only for selected types of exceptions. If the exception filter resolves to TRUE, then the exception handler (the return (count) statement, in this case) executes. The transfer of control to an exception handler is called raising an exception. Notice how the error-handling code is removed from the main line of the program.
Each block of code can have a separate exception handler, and exception handlers can even be nested inside one another. When an exception occurs, the exception filter can test the type of exception and conditionally tell the operating system to execute the exception handler, continue the program, terminate the program, or look for an exception handler in an enclosing block of code.
Operating system exceptions aren’t the only ones applications might want to respond to. Using Microsoft C language extensions, any C code can generate an exception using the run-time library routine RtlRaiseException and cause control to transfer to a registered exception handler. The operating system supports this operation by registering exception handlers and searching for them in the proper order when exceptions are raised. If no exception handler takes care of the problem, the operating system terminates the program that caused the error. In Windows NT itself, an unhandled exception would cause the operating system to shut down. Such an occurrence is unlikely since exception handlers exist throughout Windows NT, ensuring reliable and robust behavior at all levels.
Windows NT is a symmetric multiprocessing operating system base that supports multiple operating system requirements. It has a Windows graphical user interface and runs both 16-bit and 32-bit Windows-based programs, as well as MS-DOS, POSIX, and OS/2 programs. It employs advanced operating system principles such as virtual memory, preemptive multitasking, structured exception handling, and operating system objects. It is secure, powerful, reliable, and flexible. Windows NT brings to the PC the kind of capabilities and power once found only in mainframes and minicomputers.
This article is adapted from the forthcoming book Inside Windows NTÔ by Helen Custer (Microsoft Press, 1992).
References and further reading:
Accetta, M., and Baron, R., Bolosky, W., Golub, D., Rashid, R., Tevanian, A., and Young, M. “Mach: A New Kernel Foundation for UNIX Development,” Carnegie Mellon University Computer Science Department, April, 1986.
Freytag, Asmus. “Program Migration to Unicode,” Proceedings of the First Unicode Implementers Workshop, The Unicode Consortium, Mountain View, California, August, 1991.
Hall, William S. “Adapt Your Program for Worldwide Use with Windows Internationalization Support,” Microsoft Systems Journal, Vol. 6, No. 6, Nov./Dec. 1991.
Meyer, Bertrand. Object-oriented Software Construction, Prentice-Hall, Int’l, Hertfordshire, United Kingdom, 1988.
Sheldon, Kenneth M. “ASCII Goes Global,” Byte, July 1991.
Tanenbaum, Andrew S. Operating Systems Design and Implementation, Prentice-Hall, Inc., Englewood Cliffs, New Jersey, 1987.
1For ease of reading, “Windows” refers to the Microsoft Windows operating system. “Windows” is a trademark that refers only to this Microsoft product.