IIS Diagnostics Toolkit tutorial

Introduction

Having issues with a Microsoft Internet Information Services (IIS) Web server? Want to know what those issues are? You can call an overpriced IIS consultant or troubleshoot it yourself using Microsoft’s free IIS Diagnostics Toolkit. But, using it effectively takes practice.

 In this resource guide, I offer an overview of the following seven utilities in the IIS Diagnostics Toolkit and provide instructions on how to download and use them:

• SSL Diagnostics 1.0diagnoses problems related to Secure Sockets Layer (SSL) issues, such as missing certificate private keys, incorrect IIS metabase bindings, and other problems related to SSL failures. 
• Authentication and Access Control Diagnostics (AuthDiag) 1.0diagnoses, checks and monitors permission problems and security issues for Web and FTP requests.

Exchange Server SMTP Diagnostics 1.0 gathers SMTP server data that helps diagnose problems with DNS or other possible important SMTP failures.

Log Parser 2.2 sifts through thousands of Event Viewer entries, IIS log files, NetMoncaptures and other log types.  

WFetch 1.4 troubleshoots problems that reside in your Web browser. Its GUI allows you to make a request to local or remote Web sites and see the raw HTTPrequest and response to diagnose problems.

 Trace DiagnosticsService Pack 1 for Windows Server 2003 can monitor the requests to an IIS Web server in real time, or follow a request throughout the IIS processing pipeline to find failures.  

Debug Diagnostics 1.0 offers both a user interface and command-line for troubleshooting Web application failures like crashes, hangs and memory leaks.For developers, there is also an extensibility model aimed at reducing the time it takes to resolve complex Web application failures.

How to install the Microsoft IIS Diagnostics Toolkit

To install the IIS Diagnostics Toolkit suite:

1. Download the appropriate platform-specific version for your system environment:
   • IIS Diagnostics Toolkit (x86)
   • IIS Diagnostics Toolkit (amd64)
   • IIS Diagnostics Toolkit (ia64)

    

2. Now download and save the iisdiag.msifile to your workstation or server. (You typically put these directly on the system you’re going to be troubleshooting.) 
3. Double click on iisdiag.msi and then click Next at the IIS Diagnostics Toolkit Wizard. 
4. Accept the terms of the License Agreement (required). 
5. Accept the default username and organization or change and click Next. 
6. Choose the installation type and click Next again. 
7. Click Install.

You also have the option to automate installation of the IIS Diagnostics Toolkit:

1. Download the ssldiag.msifile. 
2. Click Start -> Run. 
3. Enter cmd.exe and click Run. 
4. Change to the download directory where iisdiag.msi is located (i.e., cd iisdiagnostics). 
5. At the command prompt, type: msiexec /i “ssldiag.msi” /qbr-

The toolkit should install with no user interaction. After the installation is complete, IIS Diagnostics tools will appear in the Programs menu under IIS Diagnostics.

How to use SSL Diagnostics 1.0 to identify issues in IIS

The SSL Diagnostics utility helps troubleshoot problems for SSL-enabled Web sites. It is particularly useful for identifying configuration problems in the Internet Information Services (IIS) metabase, certificates, or certificate stores. You can run this tool using the GUI interface or set it up to run silently and just write the information to a log file.

Key features of the SSL Diagnostics tool include:

• Certificate Creator: This feature lets admins replace existing server certificates with self-signed server certificates generated by SSL Diagnostics. The functionality is available with IIS 5.0, IIS 5.1 and IIS 6.0. Certificate Creator does not delete your existing certificates, but temporarily replaces the current certificate with a self-signed certificate. When testing is complete, an administrator can restore the original certificate back into IIS.Certificate Creator can help you determine if your SSL problems are related to your Windows server certificate, as well as detect problems with certificates purchased from third-party certification authorities. If SSL works with the self-signed certificate but did not work with the other certificate, it’s surely a certificate problem. If SSL does not work with the self-signed certificate or the other certificate, it’s not a certificate problem. You can then restore the original certificate, which automatically removes the self-signed one.

   

• SSL handshake: SSL Diagnostics lets admins quickly simulate an SSL connection between a Windows server and Web browser. This is known as an SSL handshake. When implemented, SSL Diagnostics opens a new window that shows the connection information from the client’s point of view, meaning the information the Web browser receives. If there is a problem with the SSL handshake, a warning will appear that describes the problem. This feature helps determine where the connection is breaking down during the SSL handshake process. You can simulate an SSL handshake at the Web-site or Web-page level. 
• Client Certificate Monitor: You can use SSL Diagnostics to monitor the usage of client certificates in real time by attaching to the associated process where the encryption and decryption takes place. As the certificate information is being parsed by the server, Client Certificate Monitor displays both the client certificates that are trying to connect to your Web site and the associated information contained in those certificates. Client Certificate Monitor also shows the error codes associated with the result of the SSL server settings and client certificates. So Client Certificate Monitor displays both valid certificates and the reasons for invalid certificates, including expired, not yet valid, or revoked client certificates.Although useful, Client Certificate Monitor requires some real-time interaction with the server processes. Because of the impact it can have on performance, using it is not recommended on a production server. After using Client Certificate Monitor, you should restart the server.

When you go to Programs -> IIS Diagnostics -> SSL Diagnostics to open the program, the utility will begin a diagnostic scan of the server on which you are running it. In the results section, just highlight the line entry you wish to research (especially those with red exclamation points) and SSL Diagnostics will give you the issue’s explanation and possible fixes to correct the problem.

How to use Authentication and Access Control Diagnostics (AuthDiag) 1.0

Having issues with clients logging into your Internet Information Services (IIS) Web site? If so, don’t be surprised. Authentication and authorization failures are quite common in IIS. That’s why Microsoft created the AuthDiag tool to troubleshoot and determine the cause of these issues.

AuthDiag analyzes IIS metabase configuration and system-wide policies, warns users of possible points of failure, and guides them through problem resolution.

AuthDiag 1.0 also includes a monitoring tool called AuthMon that captures snapshots of problems while as they occur in real time. You can run AuthMon using the GUI interface or set it up to run silently and just write the information to a log file.

Here are a few example issues that this tool can help you correct:

• 401.1 authentication failures 
• 401.3 failed ACL on file or directory 
• Failed authentication because of incorrect privileges in token 
• Failure to access a page based on metabase configuration 
• Kerberos failure when worker processes use custom identities 
• FTP user isolation — file system configured incorrectly 
• Removal of system permissions for process identities causes “Access Denied” error

To use the AuthDiag utility, go to Programs -> IIS Diagnostics -> AuthDiag, select the Task you want to run and the site, and then click Start Diagnostics. You will be presented with a screen that shows you the results of the scan. Just highlight the line entry you wish to research (especially those with red X’s). The program will offer you a link to Microsoft, where you can hopefully find a resolution for that particular issue.

How to use SMTP Diagnostics 1.0 to troubleshoot IIS issues

Written by the Microsoft Exchange Server team, the SMTPDiag utility will benefit anyone with an SMTP-based environment. It’s simple to use and run and has an excellent analysis engine. SMTPDiag is a command-line troubleshooting tool that works directly on a Windows server with IIS/SMTP service enabled or with Exchange Server installed. It utilizes the same APIs as Windows Server and Microsoft Exchange in order to diagnose configuration and connection issues involving SMTP and DNS.

Per Microsoft:

“SMTPDiag issues DNS queries using both User Datagram Protocol (UDP) and Transmission Control Protocol (TCP) to validate that the queries will succeed. If TCP queries fail, mail will not be delivered successfully.

The first thing that SMTPDiag does after verifying syntax is to check the Start of Authority (SOA) record for the remote address domain. The next step is to validate that the local domain MX/A records are resolvable. This test verifies that the sender domain is valid and any bounces can return to the originating server. This test could fail if the domain is not resolvable from inside the firewall. The remote domain MX/A records are then checked also. If this step fails, mail will not route because of issues with DNS.

At this point, the network DNS infrastructure must be investigated. When all the DNS records have been successfully queried, the tool will try to connect to all the MX (mail exchange) records that were published for the remote domain on port 25 and try to do an EHLO, mail from, rcpt to, and quit command.”

Key features

• Makes test requests based on user-inputted information, such as sender and receiver
•  
• Validates that DNS MX records are configured correctly
•  
• Ensures that the correct firewall or proxy ports are open for SMTP to properly work
•  
• Ensures the correct permissions on the relevant SMTP directories such as BadMail, Pickup, etc.

To use the SMTP Diagnostics utility:

1. Go to Start -> Run.>p>
2. Type command and hit Enter.
3.  
4. Once the command window is open, type: cd
5. Now type: cd “Program FilesIIS ResourcesSMTPDiag”

    

6. Finally, enter the following syntax to use the tool:
7. SMTPDIAG “sender address” “recipient address” [-d target DNS] [/v]

Below is a look at the results. (Email servers, email addresses, DNS servers, etc. are blocked out though, of course.)

How to use the IIS Diagnostics Toolkit’s Log Parser 2.2

Need a way of parsing through data, such as Internet Information Services (IIS) log files, the Windows registry, and Active Directory? The Log Parser 2.2 utility lets you query and sift through thousands of files and data sources.

Per Microsoft: “Log parser is a powerful, versatile tool that provides universal query access to text-based data such as log files, XML files and CSV files, as well as key data sources on the Windows operating system such as the Event Log, the Registry, the file system, and Active Directory. You tell Log Parser what information you need and how you want it processed. The results of your query can be custom-formatted in text based output, or they can be persisted to more specialty targets like SQL, SYSLOG, or a chart.”

The Log Parser tool is available as a command-line executable (LogParser.exe) and as a set of scriptable COM objects (LogParser.dll). The two binaries are independent from each other; if you want to use only one, you do not need to install the other file on your computer.

Key features:

 

• Log Parser’s built-in Input Formatscan retrieve data from the following sources:
  • IIS log files (W3C, IIS, NCSA, Centralized Binary Logs, HTTP Error logs, URLScan logs, ODBC logs)
  • Windows Event Log
  • Generic XML, CSV, TSV and W3C – formatted text files (e.g. Exchange Tracking log files, Personal Firewall log files, Windows Media® Services log files, FTP log files, SMTP log files, etc.)
  • Windows registry
  • Active Directory Objects
  • File and Directory information
  • NetMon .cap capture files
  • Extended/Combined NCSA log files
  • ETW traces
  • Custom plug-ins (through a public COM interface)
• A SQL-like engine core processes the records generated by an Input Format, using a dialect of the SQL language that includes common SQL clauses (SELECT, WHERE, GROUP BY, HAVING, ORDER BY), aggregate functions (SUM, COUNT, AVG, MAX, MIN), and a rich set of functions (e.g. SUBSTR, CASE, COALESCE, REVERSEDNS, etc.); the resulting records are then sent to an Output Format.
•  
• Output Formats are generic consumers of records; they can be thought of as SQL tables that receive the results of the data processing. Log Parser’s built-in Output Formats can:
• • Write data to text files in different formats (CSV, TSV, XML, W3C, user-defined, etc.)
  • Send data to a SQL database
  • Send data to a SYSLOG server
  • Create charts and save them in either GIF or JPG image files
  • Display data to the console or to the screen

  Unfortunately, LogParser is so powerful and flexible that I cannot easily show you how to use it. When using the command-line executable, Log Parser works on commands supplied by the user. Each command has five components:

  1. The Input Format to use
  2. Optional parameters for the Input Format
  3. The Output Format to use
  4. Optional parameters for the Output Format
  5. The SQL query that processes the records generated by the Input Format and produces records for the Output Format

  Microsoft provides the following Windows Event Log example in its documentation, but there are MANY potential uses for this tool:

  C:>LogParser “SELECT TimeGenerated, SourceName, EventCategoryName, Message INTO report.txt FROM Security WHERE EventID = 528 AND SID LIKE ‘%TESTUSER%'” -resolveSIDs:ON

  For more information on using the Log Parser utility, check out the following resources:

   

   

• Log Parser examples
•  How Log Parser works

•  Unofficial Log Parser support site

How to use Trace Diagnostics 1.0 to troubleshoot IIS issues

HTTP requests often go into Isapi filters, and either never come back out, or become extremely slow or unresponsive. Using the underlying Enterprise Tracing for Windows (ETW), the Trace Diagnostics utility simplifies HTTP request-tracing. It includes several components, including IISREQMON, IISTRACE for the command-line and IIS Request Viewer (User Interface). The toolset will only install on Service Pack 1 or higher versions of Windows.

 

When an Internet Information Services (IIS) worker process has become unresponsive or slow, Trace Diagnostics is used to understand what requests are executing in that worker process. When an HTTP request is picked up by IIS, it may be handled by multiple components before a response is generated back to the client computer. If a request fails or becomes unresponsive as it makes its way through these components, error-reporting channels like the Windows Event Log and the HTTP Error Log might not provide enough detail to help you locate the source of the problem. That’s when tracing comes in handy.

Key features

• Request Monitoris a command-line helper tool for working with currently executing requests in IIS 6.0 running on Microsoft Windows Server 2003 SP1 or later. It will help you determine what requests are executing in a worker process when it has become slow or unresponsive. When enabled, Request Monitor prompts worker processes to report statistics and details about every request in each worker process. 
• IISTraceis a command-line helper tool for requesting tracing in IIS 6.0 running on Microsoft Windows Server 2003 SP1 or later. IIStrace goes beyond iisreqmon. It allows you to view requests in an apppool and trace the events that occur in order to process those requests. This means you can identify flaws in customer code and provide them with guidance on making corrections. 
• IIS Request Viewer is a GUI that displays all currently running application pools, threads and currently executing requests per application pool. It uses the underlying tracing commands to create, start, refresh and stop traces. It will display the Request ID, Client IP, State of the request and Time that the request has been taking.

These components of the Trace Diagnostics utility can help you determine root causes of HTTP request issues, specify which providers report events to ETW during a specific trace session, customize how much trace data providers report to ETW, determine which URLs to trace, thereby focusing your troubleshooting efforts on a specific application.

From a high level, request-based tracing works like this:

1. The administrator enables a tracing session on the IIS server from a command prompt. 
2. Enterprise Ttracing for Windows (ETW) notifies IIS providers to begin reporting trace events. 
3. A request enters the IIS worker process. 
4. The administrator stops the trace session and reviews the trace log to locate the source of the problem. 
5. Problems are typically identified by error events or a START event that does not contain a corresponding END event.

Since there are multiple tools within the Trace Diagnostics utility, each with their own specific attributes, I will only focus on using the GUI tool, IIS Request Viewer. To trace requests in using the IIS Trace tool:

1. Go to Start-> Programs -> IIS Diagnostics (32bit) -> Trace Diagnostics -> IIS Request Viewer -> reqviewer.exe. 
2. Click File -> Retrieve Requests. 
3. Expand the left-hand navigation tree to view the different application pools running on your system. 
4. View the requests in the right-hand window.

If you get the common error below, set your TEMP system variable to something shorter, like C:Temp.

ERROR: Open TraceLogFile() failed (Win32 error -largeinteger – The path specified is invalid)

For more information, read Trace Diagnostics Known Bugs and How to Fix ‘Em.

How to use Debug Diagnostics to troubleshoot IIS issues

Quoting Microsoft: “The Debug Diagnostics 1.1 tool is designed to help troubleshoot performance issues in any Win32 user-mode process. For example, the tool can help you troubleshoot an application that stops responding (hangs) or crashes, performs slowly, leaks memory or has memory fragmentation issues. The tool also includes debugging scripts for IIS applications, Web data access components and Microsoft COM+ applications.

Before you start Debug Diagnostics 1.1, you must identify the type of issue you are experiencing. For example, determine whether the application stops responding, crashes, performs slowly, or leaks memory. After you know the kind of issue, you can configure the Debug Diagnostics 1.1 tool to gather the correct data. Then, you can use the data to determine and resolve the cause of the problem.”

The Debug Diagnostics 1.1 tool includes three views:

• Rules:Uses a wizard to create control scripts for the debugger host. 
• Advanced analysis:Runs the selected analysis script on one or more memory dump files. 
• Processes: Displays the status of running processes and scripts.

How to use the Debug Diagnostics utility

1. Go to Start -> Run, type the path of the Debug Diagnostics 1.0 tool, and then click OK. (By default, the Debug Diagnostics 1.0 tool is located inC:Program FilesIIS ResourcesDebugDiag.) If the Select Rule Type dialog box appears, click Cancel.

 

• Select the memory dump file that you want to analyze: Go to Start -> Run, type the path of the Debug Diagnostics 1.0 tool, and then click OK. Click the Advanced Analysis tab, and then click Add Data Files. Now select the memory dump file that you want to analyze and click Open.

 

 

• Configure the path for the symbol files: Navigate to Start -> Run, type the path of the Debug Diagnostics 1.0 tool, and then click OK. On the Tools menu, click Options and Settings. Go to the Folders and Search Paths tab, type the following path in the Symbol Search Path for Analysis box, and then click OK: srv*filepath*http://msdl.microsoft.com/download/symbols(Filepath is a placeholder for the folder or for the UNC share where you want to store the downloaded symbol files. By default, the symbol files are stored in the C: Symcache folder. Additionally, you should know that you cannot browse the http://msdl.microsoft.com/download/symbols Web site — only debugging tools can access this Web site.) 
• Start the analysis: Go to Start -> Run, type the path of the Debug Diagnostics 1.0 tool, and then click OK. Navigate to the Advanced Analysis tab -> Available Analysis Scripts, and select the type of analysis that you want. For example, if you created the memory dump file because a process stopped responding, click Crash/Hang Analyzers. If you created the memory dump file to troubleshoot a memory leak issue, click Memory Pressure Analysis. Now, under Data Files, click the memory dump file that you selected in step 3. Click Start Analysis.

 

How to read a Debug Diagnostics report

After completing the steps above, you can review the Debug Diagnostics report that is displayed in Microsoft Internet Explorer. A copy of the report is also stored in the following folder: C: Program Files IIS Resources DebugDiagReports. The report is broken down into the following sections:

• Analysis Summary: In this section, the detected issues are classified as errors, warnings, or information. Each error includes a description. Additionally, the Analysis Summary contains recommendations for how to resolve the issues. The recommendations may include reviewing a Microsoft Knowledge Base article, contacting the application vendor, or contacting Microsoft Product Support Services. Suggestions to the application developer may also be provided.

• Analysis Details: This section provides a detailed analysis of the information in the memory dump file.

 

• Script Summary: This section provides a report on the status of the script (Iisanalysis.asp) that is used to analyze the memory dump file. If an error occurs when the script is running, the Script Summary reports the error code, the source, the destination, and the lines of code that cause the error.

 

For more information on using the Debug Diagnostics utility, I recommend the following resources:

• The Debug Diagnostic 1.1

How to use the Debug Diagnostics 1.0 Tool to analyze a memory dump file

How to use the Debug Diagnostics tool to troubleshoot an IIS process that stops unexpectedly 

How to use the IIS Debug Diagnostics tool to troubleshoot a memory leak in an IIS process 

How to use the Debug Diagnostics Tool to troubleshoot high CPU usage by a process in IIS

How to use the Debug Diagnostics tool to troubleshoot a process that has stopped responding in IIS

 

Best practice for a good Microsoft IIS 7 Security

Microsoft’s Internet Information Services (IIS) Web server has presented enterprises with more than its share of security problems over the years, including the infamous Code Red worm nearly a decade ago. A key security concern with IIS has always been the number of features that are automatically installed and enabled by default, such as scripting and virtual directories, many of which proved vulnerable to exploit and led to major security incidents.

With the release of IIS 6 a few years ago, a “lockdown by default” approach was introduced with several features either not being installed or installed but disabled by default. IIS 7, the newest iteration, goes even further. It’s not even installed on Windows Server 2008 by default, and when it is installed, the Web server is configured to serve only static content with anonymous authentication and local administration, resulting in the simplest of Web servers and the smallest attack surface possible to would-be hackers.

This is possible because IIS 7 is completely modularized. Let’s briefly dig into why that is and how it enables a more secure product. Essentially administrators can select from more than 40 separate feature modules to completely customize their installation. By only installing the feature modules required for a particular website, administrators can greatly reduce the potential attack surface and minimize resource utilization.

Be aware, however, this is true only with a clean install. If you are upgrading your Windows OS and running an earlier version of IIS, all the metabase and IIS state information is gathered and persevered. Consequently, many unnecessary Web server features can be installed during an upgrade. Therefore, it is good practice for an organization to revisit

its application dependencies on IIS functionality after an upgrade and uninstall of any unneeded IIS modules.

Fewer components also means there are fewer settings to manage and problems to patch as it’s only necessary to maintain the subset of modules that are actually being used. This reduces downtime and improves reliability. Also, the IIS Management Console, with all its confusing tabs, has been replaced with a far more intuitive GUI tool, which makes it easier to visualize and understand how security settings are implemented. For example, if the component supporting basic authentication is not installed on your system, the configuration setting for it doesn’t appear and confuse matters.

So what components are likely to be needed to run a secure IIS? The first six listed below will be required by any website running more than just static pages, while seven and eight will be necessary for anyone needing to encrypt data between the server and client, while shared configuration is useful when you have a Web farm and want each Web server in the farm to use the same configuration files and encryption keys:

1. Authentication includes integrated Windows authentication, client certificate authentication and ASP.NET forms-based authentication, which lets you manage client registration and authentication at the application level, instead of relying on Windows accounts. 
2. URL Authorization, which integrates nicely with ASP.NET Membership and Role Management, grants or denies access to URLs within your application based on user names and roles so you can prevent users who are not members of a specific group from accessing restricted content. 
3. IPv4 Address and Domain Name Rules provide content access based on IP Address and Domain Name. The new property “allowUnlisted” makes it a lot easier to prevent access to all IP addresses unless they are listed. 
4. CGI and ISAPI restrictions allow you to enable and disable dynamic content in the form of CGI files (.exe) and ISAPI extensions (.dll). 
5. Request filters incorporate the functionality of the UrlScan tool restricting the types of HTTP requests that IIS 7 will process by rejecting requests containing suspicious data. Like Apache’s mod_rewrite, it can use regular expressions to block attacks or modify requests based on verb, file extension, size, namespace and sequences. 
6. Logging now provides real-time state information about application pools, processes, sites, application domains and running requests as well as the ability to track a request throughout the complete request-and-response process. 
7. Server Certificates 
8. Secure Sockets Layer 
9. Shared Configuration

Other features that enhance the overall security of IIS 7 are new built-in user and group accounts dedicated to the Web server. This enables a common security identifier (SID) to be used across machines, which simplifies access control list management, and application pool sandboxing. Server administrators meanwhile have complete control over what settings are configurable by application administrators, while allowing them to make any configuration changes directly in their application without having administrative access to the server.

IIS 7 is quite a different beast as compared with previous incarnations, and that’s a good thing. It has been designed and built along classic security principles and it gives Windows-based organizations a Web server that can be more securely configured and managed than ever before. There may still not be enough from a security perspective to sway Linux and Apache shops to change to IIS anytime soon, but Microsoft has definitely narrowed the security gap between them. It will take administrators a while to get use to the new modular format and administrative tools and tasks. The training and testing time will be worth it though as it is an OS and framework that administrators are familiar with.