|
|
Microsoft Visual Studio .NET supports debugging assembly-level and high-level language vertex and pixel shaders and effects using the DirectX Extensions for Visual Studio .NET (the shader debugger). Many of the debugger features are listed below. To prepare your machine for shader debugging, see Getting Started Debugging Shaders. If you already have the right tools installed, you can also try the Goal 5 - Shader Debugging Tutorial.
The debugger works with the debug version of the runtime, as well as with certain device types. Any vertex or pixel shader can be debugged when using a reference device. Vertex shaders can be debugged, providing they are running on a hardware abstraction layer (HAL) device created for software vertex processing. No debugging support is provided for hardware vertex processing or "pure" HAL devices. For more information about device types, see D3DDEVTYPE.
To do source-level debugging of a shader, the shader must be built with debug information and it must have an on-disk source file associated with it. You can create such a shader in the following two ways.
For debugging purposes, disable optimizations by supplying the /Od command line flag or the D3DXSHADER_SKIPOPTIMIZATION flag.
If you choose to compile a shader from memory using methods like D3DXAssembleShader or D3DXCompileShader or D3DXCreateEffect, add the #line directive to your shader code to get the debugger to work properly. Here's an example:
#line 2 "currentfile.fx"
It is not necessary to build an application with Visual Studio .NET in order to use the shader debugger. Build your application with any compiler, and then use Visual Studio .NET to debug it by starting your application and using the "Attach To Process" dialog box (which is explained in the next section).
To debug an application, the application must be using the debug version of the Microsoft Direct3D runtime and must have shader debugging enabled. You can configure these options through the Direct3D tab in the Microsoft DirectX control panel applet. The DirectX debug service must also be running on the target computer. This service is installed automatically when you install the Microsoft DirectX extensions for Visual Studio .NET.
There are two techniques for starting the debugger. Both techniques will work on a C++ application; however, a managed code application can only be debugged using the second technique.
Technique 1: Click Debug, Direct3D, Start With Direct3D Debugging (CTRL+ALT+F5) to start the debugger.
Technique 2: You can "attach" to an already running Direct3D process. The easiest way to do this from Visual Studio .NET is as follows:
In the Attach to Process dialog box, select Native to debug your C++ code, and select Direct3D to debug your Direct3D code. You can debug both C++ and Direct3D code simultaneously (see Debugging C++ Code and Direct3D Code Simultaneously near the bottom of this topic).
Visual Studio .NET also supports remote debugging of Direct3D applications. To attach to a Direct3D process on a remote computer, in the Processes dialog box, enter the name of the remote computer in the Name box and then attach to the process the same as for a local process.
To set a breakpoint in a vertex or pixel shader source file, the shader must have debug information present (see Building with Debug Information). The module window (click on Debug, then Windows and finally Modules) can be used to view all shaders that are currently loaded. When debugging native code, right-click the window and choose Show Modules for All Programs to view the shaders that are loaded.
You can set a breakpoint at a particular line in a shader the same way you would set a breakpoint in C++ source code, that is:
Left clicking the margin next to a line in the source code toggles a breakpoint.
Using the Function tab in the Breakpoint Properties dialog box, type the case-sensitive name of the shader function as the name, and set the language to Direct3D Shader. The function name is the name of the shader function stored in the debug information, or the name of the file containing the shader. The module window (open it by clicking on Windows and then select the Windows... menu item) lists all currently loaded shaders and their names. IDirect3DDevice9::BeginScene and IDirect3DDevice9::EndScene breakpoints work as before.
Any of the following types of breakpoints can have a condition, as long as the breakpoint is set to occur when the condition is true (as opposed to when the condition changes):
The expression must evaluate to a single-element Boolean result.
A pixel area breakpoint triggers at the first instruction of any pixel shader that is about to modify a pixel within a given rectangle. Set a pixel area breakpoint by going to the Data tab in the Breakpoint dialog and:
Alternatively, a pixel area breakpoint can also be created by dragging out a rectangle with the mouse on a render target tool window.
Setting a breakpoint on a specific instruction of a shader does not require debug information. You can set an address breakpoint in the following ways.
The format of an address is xxx:yyy where:
To find the values for xxx:yyy, open the Disassembly window, find the instruction on which you want to set a breakpoint, and copy the module number:address number from the previous line. The xxx number starts with a "v" for Vertex Shader or "p" for Pixel Shader. Here's an example of the number: P00000009:00000780 (which could be entered as P9:780). This would set a breakpoint in a pixel shader, that is in module 9, at address 780.
To set a breakpoint when the application you are debugging reaches IDirect3DDevice9::BeginScene or IDirect3DDevice9::EndScene, select the New Breakpoint command (CTRL+B by default), select the Function tab, select Direct3D Shader from the Language combo box, and type either BeginScene or EndScene as the name of the function.
You can step through shader code using the Debug menu's Step Over command (F10 by default). Stepping in Direct3D code will cause the application you are debugging to stop at the next vertex or pixel shader instruction it runs.
When the application you are debugging is stopped in a vertex or pixel shader, or at a BeginScene or EndScene breakpoint, you can use the Watch window to inspect the state of the application you are debugging. You can also quickly view the contents of a register by briefly resting the mouse pointer on the register in the Disassembly window or in your shader source code.
The expression syntax supports register names, swizzles, and the following operators, which all operate per component.
+ - * / == < != <= > >= [] ! || &, &
In addition, the following two intrinsic functions are supported.
All types must match exactly. Floating-point literals must have a decimal point; otherwise, they are treated as integers.
This expression returns TRUE if all of the first three components of r0 are less than the corresponding components of c4.
all( r0.xyz<c4.xyz )
Constant registers are indexed by one plus the x-component of a0.
c[a0.x+1]
To view the contents of a register in the watch window, use the following expression syntax:
registerName[.swizzle][,format][,w]
Where:
To view the current values of render states and texture stage states on the device, enter $DeviceState in the Watch window.
The coordinates of the pixel currently being rendered by a pixel shader are contained in register vPos (see Position Register). Enter vPos in the watch window to see its value.
Viewing the render target - To view the contents of the current render-target surface, open the Render Target window by selecting the Debug menu's Direct3D, Render Target menu options. This window will be updated with the current contents of the render target whenever the application you are debugging enters break mode.
Viewing textures - To view the contents of the texture currently selected into a texture stage, open a Texture window by selecting the appropriate texture stage from the Debug menu's Direct3D, Textures menu. The texture windows will be updated with the current contents of the corresponding texture stage whenever the application you are debugging enters "break" mode.
If the contents of a surface (either the render target or a texture) are not available, the surface's window will show the text as "Unavailable." Possible reasons for a surface being unavailable include:
The Visual Studio .NET debugger supports debugging of multiple program types at the same time. Typical C++ debugging uses the "Native" program type. Direct3D debugging uses the "Direct3D" program type. When the application you are debugging stops in C++ code, the current program will be set to Native, and the call stack, watch window, and other debug windows will reflect the state of the Native program. Similarly, when the application you are debugging stops in shader code, the current program will be set to Direct3D, and the debug windows will reflect the state of the Direct3D program. To switch between program types, use the Program combo box on the Debug Location toolbar. Note that when the application you are debugging is stopped in Native code, most of the state of the Direct3D program will be unavailable. The reverse is not true, however. If you stop at a breakpoint in a shader, for example, you can switch to the Native program and see the C++ call stack that caused your shader to be invoked.
For more information about debugging multiple program types simultaneously, see Debugging Multiple Programs.
If possible, build your shaders with debug information enabled (see Building with Debug Information). In cases where building with debug information is not feasible, such as for shaders that are built up "on the fly," debugging is possible but is much more difficult. Because source code is not available, source code breakpoints won't work. One approach to debugging without debug information is to set a BeginScene breakpoint, and when that breakpoint is reached, step into the shader code of the first shader run by pressing F10. When you're stepping through disassembly code, you can set breakpoints in the Disassembly window. You can also use the Modules window to view the names of all shaders currently loaded and selectively set breakpoints on them as needed.