How msExchRecipientDisplayType and msExchangeRecipientTypeDetails Relate to Your On-Premises

Updated 22 Jan 2019:

In order to foster open community knowledge and growth, I’ve moved the values to being listed in GitHub, here.

You’re more than welcome to make a pull-request, in order to keep the list up-to-date, should you find any new values in the wild.

O365 and Exchange 2016/Exchange 2013: Understanding the UserPhoto API

NOTE: This post – drafted, composed, written, and published by me – originally appeared on https://blogs.technet.microsoft.com/johnbai and is potentially (c) Microsoft.

We recently had an issue for an Enterprise Cloud customer, in which the photo was not rendering for the user – which was uploaded to AD (and synced over via MMSSPP to the managed environment). It was sussed that the issue was customer-caused, as the customer was modifying the photo via the PowerShell commandlets and had deleted the photo.

Despite the fact that the customer had disabled the OWA functionality to change the user photo in OWA, the customer was using the PowerShell commandlets to modify this object, which is the same interface/commands that OWA utilises behind the scenes.

To explain this behaviour in further detail: When you delete a photo from a mailbox in Exchange (or Exchange Online), it changes the UserPhotoCacheId MAPI property value to ‘0’ and this signifies to the UserPhoto API that the photo has been deleted. When this occurs, we will not fall-back to the ADPhotoHandler to call the photo from AD because the user explicitly deleted the photo. This scenario is by-design of the UserPhoto API.

To rectify this behaviour, clear the MAPI property by running ‘Remove-UserPhoto -ClearMailboxPhotoRecord’.

Keep in mind, as well, that Exchange uses a CachingPhotoHandler and that the photos stored on disk have a TTL of 7 days.

I have written a script to help ascertain the MAPI properties on a given mailbox, obtain the location of the cached photos on disk, and obtain the IPM.UserPhoto item from the user’s mailbox (on-premises):


function Get-UserPhotoDataOnPremises
{
param([string]$User)
Write-Warning -Message "The source of our powers comes from a Cracker Jack‚ĄĘ box, so this will take a minute. Please be patient..."
if([System.String]::IsNullOrEmpty($User))
{
throw [System.NullReferenceException]::new("The user value cannot be null.")
}
else
{
$mbx = Get-Mailbox $User
}

[string]$scriptPath = $env:ExchangeInstallPath + “Scripts\ManagedStoreDiagnosticFunctions.ps1”
# Dot-load script into function
. $scriptPath

[GUID]$guid = $mbx.ExchangeGuid.Guid
[int]$mbxNumber = Get-StoreQuery -Database $mbx.Database -Query “SELECT MailboxNumber FROM Mailbox WHERE MailboxGuid=’$guid'” | Select -ExpandProperty MailboxNumber
[string]$folderId = Get-StoreQuery -Database $mbx.Database -Query “SELECT FolderId FROM Folder WHERE MailboxNumber=’$mbxNumber’ AND DisplayName=’$([System.String]::Empty)'” | Select -ExpandProperty FolderId
$global:item = Get-StoreQuery -Database $mbx.Database -Query “SELECT * FROM Message WHERE MailboxNumber=’$mbxNumber’ AND FolderId=’$folderId’ AND MessageClass=’IPM.UserPhoto'” -Unlimited
[string]$previewPhotoCachedId = (Get-StoreQuery -Database $mbx.Database -Query “SELECT UserPhotoPreviewCacheId FROM Mailbox WHERE MailboxNumber=’$mbxNumber'” -Unlimited).p7C1B0003
[string]$photoCacheId = (Get-StoreQuery -Database $mbx.Database -Query “SELECT UserPhotoCacheId FROM Mailbox WHERE MailboxNumber=’$mbxNumber'” -Unlimited).p7C1A0003

# Obtain files on disk (if any)
$smtp = $mbx.WindowsEmailAddress.Address
$smtpAtIndex = $smtp.IndexOf(“@”)
$smtpAtIndexPlusOne = $smtpAtIndex + 1
$smtpDotIndex = $smtp.LastIndexOf(“.”)
$smtpNewLength = $smtpDotIndex – $smtpAtIndexPlusOne
$subString = $smtp.Substring($smtpAtIndexPlusOne, $smtpNewLength)
$queryString = “_$subString”
$preSubString = $smtp.Substring(0, $smtpAtIndex)
$srvr = Get-MailboxDatabaseCopyStatus $mbx.Database.Name | Where{$_.Status -contains ‘Mounted’} | Select -ExpandProperty MailboxServer
$folderPathUnc = “\\$($srvr)\” + $env:ExchangeInstallPath.Replace(“:”, “$”) + “\ClientAccess\photos”
$obj = Get-ChildItem -Path $folderPathUnc -Filter “*$($queryString)*”
$folderPath = $obj.FullName
$obj2 = Get-ChildItem -Path $folderPath
$fullPaths = @()
foreach($o in $obj2)
{
$picObj = Get-ChildItem -Path $o.FullName -Filter “*$($preSubString)*”
$fullPaths += New-Object PSobject -Property @{
Name=$picObj.Name
Size=$picObj.Length
Location=$picObj.FullName
}
}

if([System.String]::IsNullOrEmpty($item.MessageId) -eq $FALSE)
{
$string = “IPM.UserPhoto item found in user’s mailbox. The object can be found in ” + “$” + “item”
Write-Host $string
}
if($previewPhotoCachedId)
{
Write-Host “UserPhotoPreviewCacheId found: $previewPhotoCachedId”
}
if($photoCacheId)
{
Write-Host “UserPhotoCacheId found: $photoCacheId”
}
Write-Host -ForegroundColor Green “Photo files found on disk on $($srvr) for $($smtp):”
$fullPaths | FL
}

Here’s an example as run from my lab:


[PS] E:\>Get-UserPhotoDataOnPremises -User Administrator
WARNING: The source of our powers comes from a Cracker Jack‚ĄĘ box, so this will take a minute. Please be patient...
IPM.UserPhoto item found in user's mailbox. The object can be found in $item
UserPhotoPreviewCacheId found: -88512737
UserPhotoCacheId found: -88512737
Photo files found on disk on [REDACTED] for [email protected]:

Name : _Administrator-8EEDD78A2D804372C17E9FABD151BCD5.jpg
Location : \\[REDACTED]\E$\exchsrvr\ClientAccess\photos\_contoso.se-BBED6250E228D4F38F5F19BF4F1A6823\HR648x648\_Administrator-8EEDD78A2D804372C17E9FABD151BCD5.jpg
Size : 79838

Name : _Administrator-8EEDD78A2D804372C17E9FABD151BCD5.jpg
Location : \\[REDACTED]\E$\exchsrvr\ClientAccess\photos\_contoso.se-BBED6250E228D4F38F5F19BF4F1A6823\HR96x96\_Administrator-8EEDD78A2D804372C17E9FABD151BCD5.jpg
Size : 3224

I’ve also written a method in C# to test obtain the photo via the EWS GetUserPhoto REST method:


///
/// Creates an EWS request for a user's photo at each standard size.
///

/// Vanity name of your endpoint.
/// Smtp Address of the user you’re targeting.
private static void GetPhotos(string vanityName, string username)
{
int[] sizeInts = new[] { 48, 64, 96, 120, 240, 360, 432, 504, 648 };
Parallel.ForEach(sizeInts, delegate (int i)
{
Uri targetUri = new Uri($”https://{vanityName}/EWS/Exchange.asmx/s/GetUserPhoto?email={username}&size=HR{i}x{i}”);
Console.WriteLine($”Targeting: {targetUri}”);
try
{
HttpWebRequest newWebRequest = (HttpWebRequest)WebRequest.Create(targetUri);
newWebRequest.UserAgent = “Enterprise Cloud UserPhoto EWS Client”;
newWebRequest.Credentials = new NetworkCredential(“[UserName]”, “[PassWord]”);
using (HttpWebResponse newWebResponse = (HttpWebResponse)newWebRequest.GetResponse())
{
if (newWebResponse.StatusCode == HttpStatusCode.OK)
{
Console.WriteLine($”Size {i} photo found.”);
}
else if(newWebResponse.StatusCode == HttpStatusCode.NotFound)
{
Console.WriteLine($”Photo API states that a photo cannot be found for the size {i}x{i}”);
}
else
{
Console.WriteLine($”Unexpected http response received: {newWebResponse.StatusCode}”);
}
}

}
catch (Exception e)
{
Console.WriteLine(e.Message);
}

});
}

If you run into any problems with the script or have any questions or concerns around the UserPhoto API, feel free to let know! ūüôā

Exchange 2013: Understanding the Room Finder function in OWA

NOTE: This post – drafted, composed, written, and published by me – originally appeared on https://blogs.technet.microsoft.com/johnbai and is potentially (c) Microsoft.

In Exchange 2013, when you use OWA to book a room, there can be two methods that get called for this to occur. Both of these methods start workflows against service.svc and you can see this in the client behavior via the following URLs:

https://<vanityDomain>/owa/service.svc?action=GetRoomLists&ID=-<ID>&AC=1
https://<vanityDomain>/owa/service.svc?action=GetRoomsInternal&ID=-<ID>&AC=1

The first method, GetRoomLists, returns all rooms found in room lists (Get-DistributionGroup -Filter {RecipientTypeDetails -eq “RoomList”}) and this is called when you select ‘Add Room’. Once a user selects a room list, that selection is remembered (read: cached) for subsequent uses in OWA to make the fetch of the list faster.

The second method, GetRoomsInternal, returns the first 100 rooms found in the GAL (Get-Recipient -RecipientPreviewFilter $allRoomsAddrList.RecipientFilter -ResultSize Unlimited)* and is called when you use the Scheduling Assistant to find rooms.

*$allRoomsAddrList = Get-AddressList “All Rooms”

Exchange 2013: MailboxTransportMBTDeliveryPercentPermanentFailedMonitor Fires When User is Over Quota

NOTE: This post – drafted, composed, written, and published by me – originally appeared on https://blogs.technet.microsoft.com/johnbai and is potentially (c) Microsoft.

In Exchange 2013, there may be an instance where a user is over-quota and causes the Mailbox Transport responder to escalate. This issue is specific only to Mailbox Delivery and there are some simple steps to prove that this is a ‘by-design’ scenario.

Check the current value of the counter, run command:

Get-Counter “\MSExchange Delivery Store Driver\Percent of Permanent Failed Deliveries within the last 5 minutes”

Perform Message Tracking to find the failure event(s), run command:

 

Get-MessageTrackingLog -Start ([System.DateTime]::Now.AddMinutes(-10)) -End ([System.DateTime]::Now) | Where{$_.Sender -notlike ‘*Health*’ -and $_.Recipients -notlike ‘*Health*’ -and $_.EventID -contains ‘FAIL’} | Sort TimeStamp | FT ‚ÄďAutoSize

If you find failure events in the message tracking log, obtain the MessageID of the failure events, run command:

Get-MessageTrackingLog -Start ([System.DateTime]::Now.AddMinutes(-10)) -End ([System.DateTime]::Now) | Where{$_.Sender -notlike ‘*Health*’ -and $_.Recipients -notlike ‘*Health*’ -and $_.EventID -contains ‘FAIL’} | Sort TimeStamp | FL MessageID


Use the MessageIDs to find the reason for the failure(s), run command:

 

(Get-MessageTrackingLog -MessageId “<MessageID>” | Where{$_.EventID -contains ‘FAIL’}).RecipientStatus

 

If you see the following failure context for the recipient status, this is the ‘by-design’ scenario where a user’s mailbox is over-quota and has been for quite some time.

[{LRT=};{LED=554 5.2.2 mailbox full; STOREDRV.Deliver.Exception:QuotaExceededException.MapiExceptionShutoffQuotaExceeded; Failed to process message due to a permanent exception with message Cannot open mailbox 

Hybrid Configuration Wizard: Exchange server “” was not found. Please make sure you typed the name correctly.

NOTE: This post – drafted, composed, written, and published by me – originally appeared on https://blogs.technet.microsoft.com/johnbai and is potentially (c) Microsoft.

When you decommission an Exchange Server in your Organization and have a Hybrid Configuration, one of the things that does not occur is that the server is not removed from the Hybrid Configuration (msExchCoexistenceRelationship) to reflect this. When you run the Hybrid Configuration Wizard, after the decommission, you’ll see the following error:

The wizard did not complete successfully. Please see the list below for error details.
Exchange server “<Server>” was not found. Please make sure you typed the name correctly.

 

Given the above, you’ll see something similar to the following in Exchange Management Shell for the Hybrid Configuration:

SendingTransportServers   : {UNDERJORDISKE
                            DEL:416ba20c-62c1-4a28-95e5-8dbb6430dc96, LJUSALFER
                            DEL:89bf8a7d-144f-43b8-8aee-b8ac6f98721f}

 

One method to resolve this issue is to do a modification of the object in Active Directory. Using ADSIEdit, browse to the configuration container and either go to the object’s path (Config container > Services > Microsoft Exchange > Organization Name > Hybrid Configuration) or just create a new query (objectClass=msExchCoexistenceRelationship). Once you have the object, look for the property you want to fix (in this case, it was ‘msExchCoexistenceTransportServers’). I removed the two defunct values.

I, then, used Exchange Management Shell to get the DistinguishedName of the current Transport Servers in the Organization:

Get-TransportService | FL DistinguishedName

DistinguishedName : CN=HYLDA,CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=minvan,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=minvan,DC=se
DistinguishedName : CN=STROMKARLEN,CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=minvan,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=minvan,DC=se

I copied these values into the same property I removed the defunct values, previously. After doing so, I re-ran the Hybrid Configuration Wizard.

Exchange 2013: eDiscovery Changes

NOTE: This post – drafted, composed, written, and published by me – originally appeared on https://blogs.technet.microsoft.com/johnbai and is potentially (c) Microsoft.

With the release of Exchange 2013, there are some changes that are relevant to eDiscovery; whether it be for In-Place Holds or Litigation Queries to export to the Discovery Mailbox. Most notably, eDiscovery/Exchange Search does not support AQS ‚Äď it switched to KQL. KQL is supported in the SearchQuery parameter (Keywords box in the¬†Exchange Admin Center). However, Outlook still uses AQS.

Using KQL, we can perform searches that are beneficial to the eDiscovery and will save time/money/resources, without the need to invoke a third-party to process the data for you.

For example, if I perform a query for any messages that only have a word document as an attachment, I get the two messages I expect to find.

If perform the same query but, this time, define a subject or keyword I’m after, the messages are excluded because the primary rule hasn’t been met.

If I perform a third query with words that exist in the document (but not in the document name), these documents will return in my query, as well.

There is a limitation to the number of mailboxes that can be searched and it is 5,000*. Any number beyond this and the specified query will return the following error: An unknown error occurred on the search server. Please contact your administrator for assistance. The message from the search server is ‘The search exceeded the maximum number of mailboxes that can be searched at a time. Please try searching less than 5000 mailboxes.’.

*The maximum number of mailboxes that you can search can be changed in on-premises Exchange 2013. You can use the Set-ThrottlingPolicy command with the DiscoveryMaxMailboxes parameter to do so but this may come at a negative impact to performance.

As Exchange now uses the FAST Search index, we can query for what documents haven’t been processed and why. For example, if I what to query for the error where the document parser encountered a processing error, I would use the following command in Exchange Management Console:

Get-FailedContentIndexDocuments Administrator -ErrorCode 7 | FT -AutoSize

DocID Database                Mailbox       Subject           Description
—– ——–¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† ——-¬†¬†¬†¬†¬†¬† ——-¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† ———–
3462  LAB-NAEX15-01 Store 002 Administrator Binaries Test     The document parser encountered a processing error.
3464  LAB-NAEX15-01 Store 002 Administrator FW: Binaries Test The document parser encountered a processing error.

Using this I can see what, precisely, caused the document to not be indexed:

$errorSevens = Get-FailedContentIndexDocuments Administrator -ErrorCode 7
$errorSevens[0].AdditionalInfo
¬†301002 Error parsing document ‘exchange://localhost/Attachment/34eb02b4-3bc6-4163-a40d-2587faa9e0db/135d5536-d180-4198-9ba8-574b53df8206/e08d777e-e710-4407-a53d-1f57a4a58d79/a654efa1-bb87-426a-aaca-9866be73
3ccd/438086667654.0/System.Data.dll’. Document has an undetectable format and will not be parsed. 301002 Error parsing document ‘exchange://localhost/Attachment/34eb02b4-3bc6-4163-a40d-2587faa9e0db/135d5536-
d180-4198-9ba8-574b53df8206/e08d777e-e710-4407-a53d-1f57a4a58d79/a654efa1-bb87-426a-aaca-9866be733ccd/438086667654.1/mscorlib.dll’. Document has an undetectable format and will not be parsed.

In this case, the documents are binaries attached to the email for testing in regards to another issue. FAST Search cannot reverse-engineer binaries, so it is safe to assume that these files aren’t necessary for my eDiscovery purposes.¬†See here for a list of formats that Exchange FAST Search can index.

The error code enumerations are as follows:

0 – No problems.
1 – An error has occurred.
2 – A timeout has occurred.
3 – The message was not processed in a timely manner.
4 – The mailbox was offline.
5 – The attachment limit was reached.
6 – The item is only partially indexed.
7 – The document parser encountered a processing error.
8 – The document annotations aren’t valid.
9 – The document is suspected of being unable to be processed.
10 – The document processing failed due to a Rights Management error.
11 – The Store Session is not available.
12 – The mailbox is quarantined.
13 – The mailbox is locked.
14 – The operation is not supported.
15 – Search can’t sign in to the mailbox.
16 – Body conversion failed.

O365: Testing Problems With RpcProxy for Outlook Anywhere Migrations

NOTE: This post – drafted, composed, written, and published by me – originally appeared on https://blogs.technet.microsoft.com/johnbai and is potentially (c) Microsoft.

Often times, it may be necessary to test the RpcProxy URL to verify connectivity to your servers. There are various reasons for doing this but the primary is to¬†verify if the¬†connectivity problem isn’t limited to a problem with Exchange in O365.

RpcProxy is a compound URL, meaning it comprises different parts to make the whole. Let’s use this example URL and I’ll break down what I mean:

https://mail.contoso.com/rpc/rpcproxy.dll?MailboxServer.contoso.com:6001

So, the easiest one is the ‘mail.contoso.com’ as it’s¬†the vanity name for external access to the Exchange environment. We follow that with the rpcproxy path in IIS, ‘/rpc/rpcproxy.dll’. We use a delimiter ‘?’ between the vanity name and the server¬†and this is¬†to pass the server we want to hit in the backend (in 2007, this will be your node name). Then, we have the server name ‘MailboxServer.contoso.com’, followed by one of the Outlook Anywhere ports: 6001, 6002, or 6004.

How do we test it? Easy! Use your favorite web browser and browse to the target destination. IIS should require you to authenticate (use your on-premises credentials you use for the migration) and you should get a blank page.

How do we know it fails? Well, the first sign would be a HTTP 400 error. If you have ISA or a TMG, you might get 503 or 500 errors; you may also get “The server denied the specified Uniform Resource Locator (URL). Contact the server administrator.” The key here is: is the error reproducible from other sources and what is the error returned, if any?

Exchange2013: Testing MRS Across A DAG

NOTE: This post – drafted, composed, written, and published by me – originally appeared on https://blogs.technet.microsoft.com/johnbai and is potentially (c) Microsoft.

In Exchange 2013, it may become necessary to test the Mailbox Replication Service (MRS) against all members of a Database Availability Group (DAG). To do this, we can perform the following script:

#We obtain the member servers of the DAG
$Servers = ((Get-DatabaseAvailabilityGroup LAB-NAEX15-01).Servers).Name
#We perform a ‘foreach’ against each member
foreach($server in $Servers)
{

Write-Host -ForegroundColor Green $server.Name
Test-MRSHealth -Identity $server -MonitoringContext:$TRUE

}

When we perform the above script, we should receive output like the following:

PERSEPHONE
RunspaceId  : b14e1547-c215-4dbb-8ee1-a3c4978ca7c3
Check       : ServiceCheck
Passed      : True
Message     : The Mailbox Replication Service is running.
Identity    : PERSEPHONE
IsValid     : True
ObjectState : New

RunspaceId  : b14e1547-c215-4dbb-8ee1-a3c4978ca7c3
Check       : RPCPingCheck
Passed      : True
Message     : The Microsoft Exchange Mailbox Replication service is responding to a RPC ping. Server version: 15.0.712.11 caps:3F.
Identity    : PERSEPHONE
IsValid     : True
ObjectState : New

RunspaceId  : b14e1547-c215-4dbb-8ee1-a3c4978ca7c3
Check       : QueueScanCheck
Passed      : True
Message     : The Microsoft Exchange Mailbox Replication service is scanning mailbox database queues for jobs. Last scan age: 00:14:22.0340000.
Identity    : PERSEPHONE
IsValid     : True
ObjectState : New

RunspaceId          : b14e1547-c215-4dbb-8ee1-a3c4978ca7c3
Events              : {Source: MSExchange Monitoring MRSHealth
                      Id: 1000
                      Type: Success
                      Message: MRSHealth check passed.}
PerformanceCounters : {Object: MSExchange Monitoring MRSHealth
                      Counter: Last Scan Age (secs)
                      Instance: PERSEPHONE
                      Value: 862}

MINERVA
RunspaceId  : b14e1547-c215-4dbb-8ee1-a3c4978ca7c3
Check       : ServiceCheck
Passed      : True
Message     : The Mailbox Replication Service is running.
Identity    : MINERVA
IsValid     : True
ObjectState : New

RunspaceId  : b14e1547-c215-4dbb-8ee1-a3c4978ca7c3
Check       : RPCPingCheck
Passed      : True
Message     : The Microsoft Exchange Mailbox Replication service is responding to a RPC ping. Server version: 15.0.712.11 caps:3F.
Identity    : MINERVA
IsValid     : True
ObjectState : New

RunspaceId  : b14e1547-c215-4dbb-8ee1-a3c4978ca7c3
Check       : QueueScanCheck
Passed      : True
Message     : The Microsoft Exchange Mailbox Replication service is scanning mailbox database queues for jobs. Last scan age: 00:03:35.9990000.
Identity    : MINERVA
IsValid     : True
ObjectState : New

RunspaceId          : b14e1547-c215-4dbb-8ee1-a3c4978ca7c3
Events              : {Source: MSExchange Monitoring MRSHealth
                      Id: 1000
                      Type: Success
                      Message: MRSHealth check passed.}
PerformanceCounters : {Object: MSExchange Monitoring MRSHealth
                      Counter: Last Scan Age (secs)
                      Instance: MINERVA
                      Value: 215}

CHARON
RunspaceId  : b14e1547-c215-4dbb-8ee1-a3c4978ca7c3
Check       : ServiceCheck
Passed      : True
Message     : The Mailbox Replication Service is running.
Identity    : CHARON
IsValid     : True
ObjectState : New

RunspaceId  : b14e1547-c215-4dbb-8ee1-a3c4978ca7c3
Check       : RPCPingCheck
Passed      : True
Message     : The Microsoft Exchange Mailbox Replication service is responding to a RPC ping. Server version: 15.0.516.29 caps:3F.
Identity    : CHARON
IsValid     : True
ObjectState : New

RunspaceId  : b14e1547-c215-4dbb-8ee1-a3c4978ca7c3
Check       : QueueScanCheck
Passed      : True
Message     : The Microsoft Exchange Mailbox Replication service is scanning mailbox database queues for jobs. Last scan age: 00:07:23.1520000.
Identity    : CHARON
IsValid     : True
ObjectState : New

RunspaceId          : b14e1547-c215-4dbb-8ee1-a3c4978ca7c3
Events              : {Source: MSExchange Monitoring MRSHealth
                      Id: 1000
                      Type: Success
                      Message: MRSHealth check passed.}
PerformanceCounters : {Object: MSExchange Monitoring MRSHealth
                      Counter: Last Scan Age (secs)
                      Instance: CHARON
                      Value: 443}

AGAMEMNON
RunspaceId  : b14e1547-c215-4dbb-8ee1-a3c4978ca7c3
Check       : ServiceCheck
Passed      : True
Message     : The Mailbox Replication Service is running.
Identity    : AGAMEMNON
IsValid     : True
ObjectState : New

RunspaceId  : b14e1547-c215-4dbb-8ee1-a3c4978ca7c3
Check       : RPCPingCheck
Passed      : True
Message     : The Microsoft Exchange Mailbox Replication service is responding to a RPC ping. Server version: 15.0.712.11 caps:3F.
Identity    : AGAMEMNON
IsValid     : True
ObjectState : New

RunspaceId  : b14e1547-c215-4dbb-8ee1-a3c4978ca7c3
Check       : QueueScanCheck
Passed      : True
Message     : The Microsoft Exchange Mailbox Replication service is scanning mailbox database queues for jobs. Last scan age: 00:07:51.9760000.
Identity    : AGAMEMNON
IsValid     : True
ObjectState : New

RunspaceId          : b14e1547-c215-4dbb-8ee1-a3c4978ca7c3
Events              : {Source: MSExchange Monitoring MRSHealth
                      Id: 1000
                      Type: Success
                      Message: MRSHealth check passed.}
PerformanceCounters : {Object: MSExchange Monitoring MRSHealth
                      Counter: Last Scan Age (secs)
                      Instance: AGAMEMNON
                      Value: 471}

 

Exchange 2013: ‘Get-AgentLog’ Throws Exception When Run From The Front End

NOTE: This post – drafted, composed, written, and published by me – originally appeared on https://blogs.technet.microsoft.com/johnbai and is potentially (c) Microsoft.

During the investigation of a not-so-recent case, it was discovered that Get-AgentLog Рwhen run from a Front-End server Рwill throw an exception. What occurs is that when you attempt to run the command from EMS, you receive a notice about an exception and that a Watson dump is being generated; however, no Watson dump can be found on the server and there are no events in the Application or Crimson Logs:

Get-AgentLog -TransportService FrontEnd -StartDate ([System.DateTime]::Now).AddDays(-4) -EndDate ([System.DateTime]::Now)
WARNING: An unexpected error has occurred and a Watson dump is being generated: Index (zero based) must be greater than or equal to zero and less than the size of the argument list.
Specified cast is not valid.
    + CategoryInfo          : NotSpecified: (:) [Get-AgentLog], InvalidCastException
    + FullyQualifiedErrorId : System.InvalidCastException,Microsoft.Exchange.Management.AgentLog.GetAgentLog
    + PSComputerName        : <Server>

If you attempt to specify the location, you receive an error that the location cannot be found (note the fully qualified error):

VERBOSE: Connecting to <Server>
VERBOSE: Connected to <Server>.
Get-AgentLog -TransportService FrontEnd -StartDate ([System.DateTime]::Now).AddDays(-4) -EndDate ([System.DateTime]::Now) -Location “C:\exchsrvr\TransportRoles\Logs\FrontEnd\AgentLog\”
The location “C:\exchsrvr\TransportRoles\Logs\FrontEnd\AgentLog” does not exist. Please specify a valid file or directory to look for agent logs using the -Location parameter.
Parameter name: Location
    + CategoryInfo          : InvalidArgument: (:) [Get-AgentLog], ArgumentException
    + FullyQualifiedErrorId : [Server=<Server2>,RequestId=5b8d0c5b-6b0b-4b7d-ad94-008aab18485c,TimeStamp=11/27/2013 11:15:48 PM] 97084C54,Microsoft.Exchange.Management.AgentLog.GetAgentLog
    + PSComputerName        : <Server>

In the stack, you would see that the exception is a CLR exception, due to a conversion:

KERNELBASE!RaiseException+0x39
clr!RaiseTheExceptionInternalOnly+0x28b
clr!IL_Throw+0xe3
System_Management_Automation_ni!System.Management.Automation.LanguagePrimitives.ConvertStringToType(System.Object, System.Type, Boolean, System.Management.Automation.PSObject, System.IFormatProvider, System.Management.Automation.Runspaces.TypeTable)+0x126e67e

_message:00000000054d6638 (System.String) Length=89, String=”Cannot convert the “switchAttribute” value of type “System.String” to type “System.Type”.”
_HResult:0x80004002 (System.Int32)
errorId:0000000002d09190 (System.String) Length=27, String=”InvalidCastFromStringToType”

The cause: In Exchange 2013, commands are proxied from the Frontend Server (CAS) to a Backend Server (Mailbox). This is the reason that Server2 shows up in the Fully Qualified Error, when the command is run from Server.

The work-around is to open Windows PowerShell and add the PSSnapin (this bypasses command proxying). Then, you will receive the necessary output, sans any exceptions occurring.

Get-AgentLog -TransportService FrontEnd -StartDate ([System.DateTime]::Now).AddDays(-30) -EndDate ([System.DateTime]::Now) | FT -AutoSize

Timestamp             SessionId        IPAddress   MessageId P1FromAddress     P2FromAddresses    Recipients    Agent                Event              Action

———¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† ———¬†¬†¬†¬†¬†¬†¬† ———¬†¬† ——— ————-¬†¬†¬†¬† —————¬†¬†¬† ———-¬†¬†¬† —–¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† —–¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† ——

11/21/2013 8:47:25 PM 08D0B544432ACF98 <Redacted>             <Redacted>        <Redacted>        <Redacted>     Inbound Trust Agent   OnEndOfHeaders    ModifyHeaders

11/21/2013 8:47:25 PM 08D0B544432ACF98 <Redacted>             <Redacted>         <Redacted>         <Redacted>     Inbound Trust Agent   OnEndOfHeaders    ModifyHeaders

11/21/2013 8:48:27 PM 08D0B544432ACF9A <Redacted>             <Redacted>         <Redacted>         <Redacted>     Inbound Trust Agent   OnEndOfHeaders    ModifyHeaders

11/21/2013 8:48:27 PM 08D0B544432ACF9A <Redacted>             <Redacted>         <Redacted>         <Redacted>     Inbound Trust Agent   OnEndOfHeaders    ModifyHeaders

You can read more about this at these sites:

http://blogs.msdn.com/b/dvespa/archive/2013/02/21/install-error-transport-agents-exchange-2013-powershell.aspx

http://technet.microsoft.com/en-us/library/bb125175.aspx

Exchange 2013: High Availability – When Maintenance Might Be Necessary

NOTE: This post – drafted, composed, written, and published by me – originally appeared on https://blogs.technet.microsoft.com/johnbai and is potentially (c) Microsoft.

During the course of your on-premises environment, it may become necessary to take a production Exchange server out of rotation and perform maintenance on it (i.e.: replace memory, patching, reconfiguration, etc.). During this time, it will become necessary to prevent RemoteMonitoring from testing against the server you need to repair.

To understand this, we need to cover High Availability (HA) differences in Exchange 2013. Any Health Set that falls under ‘Global Monitoring’ has probes, monitors, and responders that are initialized at run-time on all Exchange 2013 servers in the organization. Any Health Set that falls under ‘Server Monitoring’ has static probes, monitors, and responders that are created locally on the server, only. Remote Monitoring falls under ‘Server Monitoring’ and has probes that test against other Exchange servers within the organization:

Get-ServerHealth -Server E15MBX -HealthSet RemoteMonitoring | FT -AutoSize

Server  State         Name                         TargetResource         HealthSetName    AlertValue ServerComponent
——¬† —–¬†¬†¬†¬†¬†¬†¬†¬† —-¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† ————–¬†¬†¬†¬†¬†¬†¬†¬† ————-¬†¬†¬† ———- —————
E15MBX  NotApplicable HealthManagerObserverMonitor Persephone.contoso.com RemoteMonitoring Healthy    None

If you need to disable this probe while you’re performing maintenance on a server, so an escalation isn’t created, you can create a local server monitoring override for this purpose.

To do so, you’re going to need the Identity of the probe. You can perform the following command to find information regarding the probe:

Get-MonitoringItemIdentity -Server¬†E15MBX¬†-Identity RemoteMonitoring | Where{$_.ItemType -contains ‘Probe’}

RunspaceId     : b475c539-d2ba-4a28-9ad2-5cf0a18024ce
Server         : E15MBX
HealthSetName  : RemoteMonitoring
Name           : HealthManagerObserverProbe
TargetResource : Persephone.contoso.com
ItemType       : Probe
Identity       : RemoteMonitoring\HealthManagerObserverProbe\Persephone.contoso.com
IsValid        : True
ObjectState    : New

Once we’ve found the information regarding the probe, we can commence to setting the necessary override. To do so, we’ll use the following command:

Add-ServerMonitoringOverride -Server E15MBX -Identity ‘RemoteMonitoring\HealthManagerObserverProbe\Persephone.contoso.com‘ -ItemType Probe -PropertyName Enabled -PropertyValue 0 -Duration 60.00:00:00 -Confirm:$FALSE

The ‘PropertyValue’ is, typically, a bitwise flag; meaning that ‘0’ is off and ‘1’ is on. The ‘ItemType’ can be either a Probe, a Monitor, or a Responder. The ‘Duration’ must be 60 days or less.

 

To confirm the override has been set, we can check:

Get-ServerMonitoringOverride -Server E15MBX | FT -AutoSize

Identity                                                                    ItemType PropertyName PropertyValue
——–¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†——– ———— ————-
RemoteMonitoring\HealthManagerObserverProbe\Persephone.contoso.com          Probe    Enabled      0

We can also verify when the expiration time is:

(Get-ServerMonitoringOverride -Server E15MBX | Where{$_.Identity -ilike ‘RemoteMonitoring\HealthManagerObserverProbe\Persephone.contoso.com’}).ExpirationTime
9/15/2013 1:29:59 PM

We can remove the override just as easily as we have set it:

Remove-ServerMonitoringOverride -Server E15MBX -Identity ‘RemoteMonitoring\HealthManagerObserverProbe\Persephone.contoso.com’ -ItemType Probe -PropertyName Enabled -Confirm:$FALSE

And just like that, Exchange 2013 is regularly testing against the remote server, again.

More information about Adding a Server Monitoring Override can be found here.