WinDeveloper IMF Tune

WinDeveloper IMF Tune
WinDeveloper IMF Tune

Learning Transport Agent Pipeline Tracing Hands-on (Part 1)

Alexander Zammit

Alexander Zammit Photo

Software Development Consultant. Involved in the development of various Enterprise software solutions. Today focused on Blockchain and DLT technologies.

Cast your Vote
Poor Excellent

Pipeline tracing opens a window into the email transport. It uncovers the email modifications carried out by transport agents. Non-Delivery Reports (NDRs) and other Exchange generated emails can also be exposed in this manner. Here we learn how to use pipeline tracing by example.

Included in Exchange 2007 and 2010, pipeline tracing is one of the features available for troubleshooting email transport issues. This gives us multiple dumps of emails as they flow through the various transport agents running on Edge/Hub Transport servers. In this manner we get a snapshot of the email just before and after each of the agents completes processing.

Although not specific to anti-spam, pipeline tracing is especially useful when analyzing how anti-spam and other email hygiene agents are handling and modifying emails.

Something about Agents

Before we start with the tracing examples let's have a quick look at the configuration side. Since pipeline tracing is intended to troubleshoot transport agents it's a good idea to start from the list of installed agents. From the shell run:
Get-TransportAgent

This is what I had on my Exchange 2010 test Hub Transport server (Note: This article also applies to Exchange 2007):

Get-TransportAgent

To follow this article it is not necessary for you to have exactly the same set of agents. However it would help if you follow the examples on a transport server that has the Exchange anti-spam agents installed. For details on the anti-spam agents check The Exchange 2007 Content Filter Agent. This article is applicable to both Exchange 2007 and 2010.

Configuring Pipeline Tracing

Pipeline tracing is configured and enabled on transport servers using the cmdlet:
Set-TransportServer

There are three settings that can be configured. Firstly we specify a sender address, whose emails are to be traced. Effectively this is a filter that is matched against the SMTP envelope sender address (MAIL FROM). If the filter is matched, the email snapshots are dumped to disk. This filter is critically important since tracing all emails could quickly exhaust the available disk resources.

Here I configure sender@adminstop.com as the pipeline tracing sender address on the HAMRUN transport server:
Set-TransportServer HAMRUN -PipelineTracingSenderAddress sender@adminstop.com

The next setting is the directory where the data is to be dumped. If not set this defaults to:
<Program Files>\Microsoft\Exchange Server\V14\TransportRoles\Logs\PipelineTracing\MessageSnapshots

To change the path to d:\pipeline on the HAMRUN transport server:
Set-TransportServer HAMRUN -PipelineTracingPath "d:\pipeline"

Lastly we need to enable pipeline tracing:
Set-TransportServer HAMRUN -PipelineTracingEnabled $True

To switch it off, rerun the last cmdlet with the flag set to $false.

Test 1 and 2: Tracing Non-Delivery Reports NDRs

In this article I will use telnet to submit test emails. This allows us to directly control the envelope sender address. If you need a refresher on how to submit emails using telnet check Telnet Port 25.

Tracing NDRs and other server generated emails is done by setting the PipelineTracingSenderAddress to an empty sender. Here is how I setup pipeline tracing for the HAMRUN transport server:
Set-TransportServer HAMRUN -PipelineTracingSenderAddress "<>"
Set-TransportServer HAMRUN -PipelineTracingEnabled $True

Configured in this manner, any email with an empty sender will be traced (not just those generated by Exchange). To confirm this submit an email using the command sequence that follows. From the command prompt:

Note: 192.168.0.60 is the IP of the test server HAMRUN. The Exchange email domain is wtest-dom1.local. You will need to replace these with your own values.

telnet 192.168.0.60 25
ehlo test.adminstop.com
MAIL FROM: <>
RCPT TO: user1@wtest-dom1.local
DATA
From: sender@adminstop.com
To: user1@wtest-dom1.local
Subject: Testing Pipeline Tracing

This is a test email
bye
.

Telnet Empty Sender Email

Under the pipeline tracing folder you should find a new directory with the snapshots of this email. But before looking at this directory let's try a more interesting email. I will now send an email to an invalid recipient address so as to generate an NDR.

Note: For this to work make sure you don't have Recipient Filtering | 'Block messages sent to recipients that do not exist in the directory' selected. Otherwise the email would be rejected immediately and no NDR is generated.

Here is the command sequence of the next email:

telnet 192.168.0.60 25
ehlo test.adminstop.com
MAIL FROM: sender@adminstop.com
RCPT TO: WrongAddress@wtest-dom1.local
DATA
From: sender@adminstop.com
To: WrongAddress@wtest-dom1.local
Subject: Sending Email to Wrong Address

Sending Email to Wrong Address
bye
.

Unknown Recipient NDR

Note how the MAIL FROM is no longer an empty sender. On my test server there is no mailbox with the address WrongAddress@wtest-dom1.local. So Exchange will respond to this with an NDR that will be caught by pipeline tracing.

This is what my tracing destination directory looked like:

Pipeline Tracing NDR Snapshots

If you open any of the eml files you will see the NDR itself:

NDR Email

The multiple copies of the same email are what this tracing functionality is all about. It shows how the email change as it starts and finishes processing for each of the transport agents. Very often the differences are minimal. In other cases the changes are more substantial. It all depends on what changes the transport agents perform for that particular email. We will take a closer look at these EML files in the second part of this article.

Final Tips

Today we introduced transport agent pipeline tracing. We saw how this is configured and produced our first traces. In today's examples we captured an NDR showing how this functionality can be used to sniff emails generated by Exchange itself. In the second part we will immediately start with more examples, applying pipeline tracing to agent troubleshooting. We will also look at the snapshots and gain a better understanding of the information contained in the trace output.

References

Learning Transport Agent Pipeline Tracing Hands-on (Part 2)

Enable Pipeline Tracing

Using Pipeline Tracing to Diagnose Transport Agent Problems

Telnet Port 25

The Exchange 2007 Content Filter Agent

User Comments - Page 1 of 1

Charles Derber 15 Jun 2012 09:17
Nice article would look forward for the 2nd :)
Copyright © 2005 - 2024 All rights reserved. ExchangeInbox.com is not affiliated with Microsoft Corporation