See Setup API in the Platform SDK as a guide when writing a setup program.
Setup API includes a sample installation program template, documentation that explains how to use the installation template, plus functions including those that control basic user dialog boxes. You may need to write additional custom dialog boxes.
The Microsoft Exchange Server SDK provides a sample setup program called GWSETUP for installing a gateway, and a sample setup program called MBSETUP for installing a mailbox agent.