Microsoft Corporation
Desktop and Business Systems Division
May 1996
Over time, concepts and techniques that allow us to build large applications have been discovered. Modularity, or structuring an application as independent modules, allows you to build complex systems from simpler parts and reuse software. Object-oriented concepts and the Microsoft® Component Object Model (COM) provide a technique that allows you to write modular applications.
Components can give both modularity and natural distribution. When an application is structured as components, the individual parts can reside together in a single computer, or they can interact by using remote-procedure calls across a network.
Structuring an application into independent components can create problems with managing the components. Monolithic programs fail and restart as a unit. But, with a modular system, the failure of one component must not corrupt the others. There must be a way to isolate faults and limit fault propagation. Transactions provide modular execution, simplify and automate fault handling, and provide a simple conceptual execution framework for both implementers and users.
The user thinks of a transaction as a single change event that either happens or doesn't happen. Implementers think of a transaction as a programming style that allows them to write modules that can participate in distributed computations. Consider a transfer of money from one bank account to another. The implementers and the users want to make sure that either both accounts change or neither changes. It is hard to make this work in a distributed system—computers can fail and messages can be lost. Transactions provide a way to bundle a set of operations into an atomic execution unit.
The atomic all-or-nothing property is common; it appears in many everyday proceedings. When several parties enter into a contract, an escrow officer coordinates the transaction and collects the signatures of each party to the contract. The contract is final only when the escrow officer announces that everyone has signed. A minister conducting a marriage ceremony first asks the bride and groom, "Do you take this person to be your spouse?" If they both respond "I do," the minister pronounces them married. A director on a movie set first asks, "Ready on the set?" If all respond yes, the director then calls, "Action!" A helmsman on a sailboat preparing to tack first asks the crew, "Ready about?" If they all respond yes, then the helmsman shouts, "Helm's a'lee!" and turns the boat.
These scenarios illustrate the basic principle of a transaction: several independent entities must agree. If any party disagrees, the deal is off. Once they agree, the transaction can occur. The Microsoft Distributed Transaction Coordinator (MS DTC) performs this transaction coordination role for the other components of the COM architecture.
In MS DTC terminology, the director is called the transaction manager. The participants in the transaction that implement transaction-protected resources, such as relational databases, are called resource managers.
An application begins a transaction by calling the transaction manager's BeginTransaction method. This creates a transaction object that represents the transaction. The application then calls the resource managers to do the work of the transaction.
The application's first call to each resource manager identifies the application's current transaction. For example, if the application is using a relational database, it calls the Open Database Connectivity (ODBC) interface, which associates the transaction object with the ODBC connection. Thereafter, all database calls made over that connection are performed on behalf of the transaction until the transaction is ended.
When a resource manager first does work on behalf of a transaction, it enlists in the transaction by calling the transaction manager. As the transaction progresses, the transaction manager keeps track of each of the resource managers enlisted in the transaction.
Typically, the application completes the transaction with a Commit transaction method. If the application is unable to complete, the application calls the Abort transaction method, which undoes the transaction's actions. If the application fails, MS DTC aborts the transaction.
When the application successfully completes the transaction's work, it calls the MS DTC to commit the transaction. MS DTC then goes through a two-phase commit protocol to get all of the enlisted resource managers to commit. The two-phase commit protocol ensures that all the resource managers commit the transaction or all abort it. In the first phase, the MS DTC asks each resource manager if it is prepared to commit. If all participants say yes, then in the second phase MS DTC broadcasts the commit message to all of them. If any part of the transaction fails, if a resource manager fails to respond to the prepare request, or if a resource manager responds no, then MS DTC notifies all of the resource managers that the transaction aborted.
Transaction managers are a key part of most database systems. Transaction managers are also an optional part of some operating systems. Microsoft believes that transactions are essential for distributed applications—transactions provide modular execution, which complements COM's modular programming. So, Microsoft implemented transaction management software for both the Microsoft Windows® 95 and Microsoft Windows NT® operating systems.
Combining the transaction concept with COM required innovation. Traditional transaction systems required considerable skill to install and manage. The challenge of integrating MS DTC with the Microsoft operating systems was to automate installation, management, and use. Many of the concepts and techniques had to be reinvented for the new client/server, object-oriented, and visual management environments.
In its first release, MS DTC works with one resource manager: Microsoft SQL Server. It also operates with several transaction processing monitors, including Encina Top End, and Novell tuxedo. MS DTC implements the OLE transaction interfaces. All OLE transaction interfaces are public so that any resource manager can become an OLE transaction resource manager. In the future, Microsoft and other software companies will add other transactional resource managers, such as distributed object systems, transactional file systems, transaction queuing systems, and workflow management systems.
Transactions provide the ACID (atomicity, consistency, isolation, durability) properties.
Atomicity
A transaction either commits or aborts. If a transaction commits, all of its effects remain. If it aborts, all of its effects are undone. For example, in renaming an object, the new name is created and the old name is deleted (commit), or nothing changes (abort).
Consistency
A transaction is a correct transformation of the system state. It preserves the state invariants. For example, by adding an element to a doubly linked list, all four forward and backward pointers are updated.
Isolation
Concurrent transactions are isolated from the updates of other incomplete transactions. These updates do not constitute a consistent state. This property is often called serializability. For example, a second transaction, traversing the doubly linked list mentioned in the consistency example, will see the list before or after the insert, but it will see only complete changes.
Durability
Once a transaction commits, its effects will persist even if there are system failures. For example, after the rename in the atomicity example, the object will have the new name even if the system fails and reboots right after the commit completes.
It is up to the application to define consistency and bracket its computation with BeginTransaction and Commit transaction methods to delimit these consistent transformations. Transactional resource managers provide consistent, isolated, and durable transformations of the objects they manage. MS DTC manages transactions that involve multiple resource managers, even those distributed among multiple computers. MS DTC creates transaction objects, tracks migration of transactions among resource managers, and implements the two-phase commit protocol to make these transactions atomic and durable.
We have already discussed the end user's view of transactions; transactions are ACID execution units that either commit or abort. If a transaction commits, its effects are durable. If a transaction aborts, its effects are undone. Application programmers, resource managers, and transaction managers cooperate to implement the ACID properties. Let's look at the role of each of these participants.
The application programmer's model of transactions is quite simple: applications either succeed or fail. The application begins a transaction by getting a transaction object. All subsequent work is associated with that transaction object. When the program reaches a consistent state, it calls the Commit transaction method. If the commit succeeds, the transaction is durably committed. If the commit fails, the transaction is aborted. If the program finds that it cannot complete the transaction, it may call the Abort transaction method to undo the transaction's effects. This is a simple way to clean up complex failure cases.
If the application fails before it commits the transaction, the transaction manager will abort the transaction and tell each enlisted resource manager to undo the transaction's effects. If a computer or resource manager fails, the transaction will also be aborted. Once the transaction has successfully committed, the resource managers and the transaction manager will ensure that the transaction's effects are durable, even if there are subsequent failures.
When a resource manager first comes on the scene, it contacts its local transaction manager to declare the resource manager's presence. Then the resource manager waits for execution requests from applications. When a request arrives tagged with a new transaction object, the resource manager enlists in the transaction. By enlisting, the resource manager ensures that it will get callbacks from the transaction manager when the transaction commits or aborts. The resource manager then performs the transaction's requests. For example, the transaction might insert, delete, or update records in a relational database. The resource manager keeps enough information to allow it to either undo or redo the transaction's work on those resources. There are many ways to do this; keeping versions of data or keeping a log (journal) of the changes are two common techniques.
When the application commits the transaction, the transaction manager initiates the two-phase commit protocol. The transaction manager first asks each enlisted resource manager if it is prepared to commit the transaction. The resource manager must prepare to commit—it readies itself to either commit or abort the transaction.
Typically, the resource manager records the old and new data in stable storage so that the resource manager can recover it even if the system fails. If the resource manager cannot prepare successfully, it informs the transaction manager that it cannot prepare and the transaction manager aborts the transaction. If the resource manager can prepare, it tells the transaction manager and waits for the transaction manager's decision whether to commit or abort the transaction.
Once prepared, a resource manager must wait until it gets a commit or abort callback from the transaction manager. Most transactions commit, although a few transactions abort. Typically, the entire prepare and commit protocol completes in a fraction of a second. If there are system or communication failures, the commit or abort notification may not arrive for minutes or hours. During this period, the resource manager is in doubt about the outcome of the transaction. It does not know whether the transaction committed or aborted. While the resource manager is in doubt about the transaction, it keeps the data modified by keeping the transaction locked, thereby isolating these changes from any other transactions.
The discussion so far shows how distributed transactions are made atomic in a fault-free environment. Now we consider how MS DTC helps resource managers make transactions atomic and durable when a resource manager fails. If a resource manager fails and then restarts, the resource manager must reconstruct the committed state of the resources it manages. The reconstructed state must reflect all of the effects of committed transactions and none of the effects of aborted transactions. When a resource manager fails, all of its enlisted transactions are aborted except for those that prepared or committed prior to the failure. When the resource manager restarts, it asks the transaction manager about the outcome of the in-doubt transactions in which it enlisted. The transaction manager tells the resource manager the outcome of each in-doubt transaction, and the resource manager commits or aborts these transactions accordingly.
In summary, the two-phase commit protocol and the resource managers combine to make transactions atomic and durable.
The transaction manager is the manager of transaction objects. It creates transaction objects and manages their atomicity and durability. Applications ask the transaction manager to create a transaction object by calling the transaction manager's BeginTransaction method.
When a resource manager first participates in a transaction, it calls the transaction manager to enlist in the transaction. The transaction manager tracks the resource managers that enlist in the transaction. Later, the application commits or aborts the transactions, or the transaction is aborted by a resource manager or a failure.
Commit and Abort are additional transaction methods on transactional objects. When asked to commit a transaction, the transaction manager initiates a two-phase commit protocol. During phase one, it asks all enlisted resource managers to prepare. During phase two, the transaction manager tells the resource managers whether the transaction committed or aborted. The two-phase commit protocol has many optimizations, including the read-only optimization and the transfer-of-commit optimization. MS DTC implements some of these optimizations, but the functionality remains the same: atomicity and durability.
The transaction manager keeps a log in safe storage on disk. The log is a sequential file that records transaction events. The transaction manager records transaction starts, enlistments, and commit decisions in the log. During normal processing, the transaction manager only writes the log. If the transaction manager fails, it reconstructs the transaction's most recent state by reading the log. So, the transaction manager uses the log to make its state durable.
The transaction manager also provides an operations interface to manage transactions. It maintains performance counters that can be displayed by using the system performance monitor. It records important operational events in the system log. These events can be displayed by using the system event viewer. It has a graphical management interface that is integrated with SQL Enterprise Manager. The graphical management interface lets the operator configure the system, view transactions, and abort or commit in-doubt transactions.
The discussion so far has assumed that all the applications and resource managers are on a single computer. MS DTC also supports transactions distributed across two or more Windows systems. Each system has a local transaction manager. All applications and resource managers talk to their local transaction managers. The transaction managers cooperate to manage transactions that span systems.
When a transaction first visits a new system (when the first request tagged with the transaction arrives at that system), the two systems involved in the request establish a relationship. The system making the request informs its local transaction manager that the transaction has an outgoing relationship to the transaction manager on the second system. Similarly, the system receiving the request informs its local transaction manager that the transaction has an incoming relationship with the transaction manager on the first system.
These outgoing-incoming relationships form a tree of transaction manager relationships called the transaction's commit tree. The enlisted resource managers are also members of this commit tree; they have an outgoing-incoming relationship to their local transaction manager.
When a distributed transaction commits or aborts, the prepare, commit and abort messages flow outward on the commit tree. Any node of the tree can unilaterally abort a transaction anytime before it agrees to the prepare request sent at phase one. Once a node has prepared, it remains prepared and in doubt until the commit coordinator tells it to commit or abort the transaction. The root transaction manager of the commit tree is the global commit coordinator. It makes the decision to commit or abort the transaction and is never in doubt.
If a computer fails and then restarts, the transaction manager at that computer will determine the outcome of all in-doubt transactions. The transaction manager reads its log file to determine the outcome of transactions for which it was the commit coordinator. For incoming transactions from other systems, the transaction manager reads the log file to determine whether it was previously notified of the transaction's outcome. For incoming transactions that remain in doubt, the transaction manager queries the incoming transaction manager to learn the transaction's outcome. The transaction manager also responds to queries from other transaction managers regarding in-doubt outgoing transactions sent to them. This is similar to the protocol that transaction managers and resource managers follow at restart. The transaction manager determines the outcome of each in-doubt transaction and, when asked, tells the resource managers the transaction's outcome.
In-doubt transactions are especially troublesome for distributed transactions. System or communication failures can leave transactions in doubt for a long time. While the transaction is in doubt, the resources modified by the transaction remain locked and unavailable to others. MS DTC provides a way for the system operator to resolve transactions that remain in doubt for too long. The operator can use a graphical management interface at the commit coordinator system to determine the transaction's outcome. The operator also can use the graphical management interface at the in-doubt system to force the in-doubt transaction to commit or abort. When the systems reconnect, MS DTC detects these operator actions and flags inconsistent actions.
Transactions are ACID (atomic, consistent, isolated, durable) modules of execution. They complement COM's program module structure. The Microsoft Distributed Transaction Coordinator (MS DTC) provides a transaction manager for each computer that manages transactions. Applications call the transaction manager to begin a transaction. BeginTransaction returns a transaction object. The application includes the transaction object with requests to resource managers. When a resource manager first begins working on a transaction, it enlists in the transaction. When the application has made a consistent transformation of the state, it asks the transaction manager to commit the transaction with the Commit transaction method. If the application cannot complete the transaction, the application program aborts it by using the Abort transaction method. If the application fails or a participating resource manager fails, then MS DTC aborts the transaction.
MS DTC uses a two-phase commit algorithm in which (1) the transaction manager requests each enlisted resource manager to prepare to commit, and (2) if all successfully prepare, the transaction manager broadcasts the commit decision. If any resource manager cannot prepare, the transaction manager broadcasts an abort decision to everyone involved in the transaction. While a resource manager is prepared, it is in doubt about whether the transaction committed or aborted. The transaction manager keeps a sequential log so that its commit or abort decisions will be durable. If a resource manager or transaction manager fails, it reconciles in-doubt transactions when it reconnects.
For distributed transactions, each computer has a local transaction manager. When a transaction works at multiple computers, the transaction managers track incoming and outgoing transactions. Each transaction manager performs all the enlistment, prepare, commit, and abort calls for local resource managers (on that computer). When committing a transaction distributed among several computers, the transaction manager sends prepare, commit, and abort messages to all its outgoing transaction managers. When a transaction manager is in doubt about a distributed transaction, the transaction manager queries the incoming transaction manager. The root transaction manager is never in doubt. If an in-doubt transaction persists for too long, the system operator can force the transaction to commit or abort.