Monthly Archives: January 2014

interesting things i see on the internet – 27/01/2014

first of all, you should all be planning your SP3 upgrades, if you haven’t started already. MSExchange.org are starting a new series this week on this very topic, so as well as reading my earlier blog post on this, you should read their article as well.

Exchange design:

Here are some nice test lab guides/posters on cross product solutions with exchange, lync and sharepoint, and here’s a brief (very brief) article on setting up an exchange 2013 lab from Steve Goodman.

Exchange troubleshooting:

I’ve seen a wonderful script for troubleshooting unexpected database growth. This script will snapshot a database and compare it to previous snapshots, and then tell which mailbox is growing, by how much and how many items. Like using exmon, but about a million time easier. I heartily recommend that everyone has a good play with this, so that when you come to use it in anger (and you probably will), you know exactly what you’re doing with it.

This looks like it may save some of you some pain in the near future; the right way to create additional receive connectors in Exchange 2013.

An old post, but an interesting one – do you have sleepy NICs? A common cause of databases moving around unexpectedly in a DAG. We’ve got a couple of customers experiencing this, and we’ve checked that this isn’t the case, but it would be great if people would check again. This is one for the best practices document, i think.

The Romanian exchange support engineers have suddenly become active on their blog, after years of very occasional posting. There’s a couple of pretty detailed posts covering some interesting troubleshooting issues up there at the moment. The mailflow troubleshooting guide is good, if brief.

Rhoderick Milne has a great post on mailbox quarantine that’s got some great hands on advice on configuration, which may explain to you why some users cannot reach their mailboxes.

Exchange general:

Here’s an interesting article on the new restrictions on upgrading the database schema in Exchange 2013. Note that’s the DATABASE SCHEMA, not the DIRECTORY SCHEMA. In exchange 2010 the database schema of each dag member upgraded as soon as the service pack/rollup was applied, making it tricky to then move databases around until all the nodes were on the correct version. In 2013 the schema won’t update until all members of the dag are at the correct software level to support the updated version. Why is this good? Because it means we are less likely to get in a situation where nodes get stuck on the wrong version and are unable to support an active database. It does happen. Twice, in my experience – 2007 was particularly prone to it.

As usual Tony Redmond has a batch of interesting posts;  how exchange 2013 measures and monitors server healthten predictions for exchange in 2014, calcualting client access licenses, service packs and cumulative updates, and the reappearance of powershell command logging. yay. There’s a lot of other posts there, all of which are great. You should read them.

Here’s a great way to start reporting message flow statistics – how many messages are generated, NDRs and so on, using the ExLogAnalyzer.

The Redmond Interoperability Plugfest 2013 has a video on MAPIHttp, the replacement for RPC over HTTP which i mentioned in the last mail. There’s also one on exchange 2013 protocols and one on outlook 2013 protocols. They’re not long, which is probably just as well. Soo… sleeepy…

Msexchange.org are starting a new series on monitoring exchange 2013 with scom 2012. Given that we are sooo up on monitoring exchange 2010 with SCOM 2007 r2, we should probably start reading this now, yes?

They’ve also got a new series on transport high availability in exchange 2013, which looks like it may be useful  – shadow redundancy and safety are in there, somewhere.

Here’s a really nice script for automating exchange mailbox audit logging. Remember to keep an eye on your disk space. What does the mailbox audit log contain? Who accessed a mailbox, what was deleted, if mail was sent using a “send as” permission and lots more. Of course you want to keep this information.

Tony Redmond (again?) has published an article on managing activesync partnerships for multiple devices on his personal blog.

Core general:

Perfmon incorrectly calculates disk latency in windows server 2008. If we don’t apply this hotfix, then we can’t trust what perfmon is telling us; given that disk latency is a major cause of poor user experience, you really need to get this installed.

MS have published a complete and updated list on microsoft product virtualization – what is supported and what is not, here.

there is a new MATS tool released for analysing the storage stack on windows 2012 and 2008. What’s MATS?

It’s not great, yet, but it’s highly promising – the solutions node in technet. How-to guides for all things, eventually. In the meantime, single sign on federation in hybrid environments…

Office 365:

Ali Larter demonstrates how office 365 stops her from trashing banks, cop cars, hotel rooms and so on. Save the cheerleader an that.

MS is developing a series of Test Lab Guides on Hybrid solutions – the Office 365 trial subscription guide is discussed here. Is it great? Well no, but it’s just part of the whole hybrid stack of test lab guides – see also the solutions stuff above – if MS manage to pull this off, it’ll be awesome. If they don’t, well, hopefully it will have given you some ideas. Here is the first step on the stack, the windows 2012 configuration test lab for public cloud technologies.

The EHLO blog has some useful, if basic, guided walkthroughs on mailbox and folder sharing scenarios. It’s Nino Bilic – if he thinks it’s worth writing about, it’s probably worth you reading about it.

it’s windows azure jump start week at the Microsoft virtual academy. live video from 8pm til midnight every night from now until Friday. or you can wait a week or so until the recordings get posted.

MS have published a useful collection of KB articles on troubleshooting common Office 365 issues.

and that’s it. my inbox is now clear. time to move to Asana. not.

Oh no, I’ve got a cert that’s about to expire!

cert001The scenario – you know that your cert is about to expire, you’ve bought a new cert, and you’ve installed it correctly, but you’re still getting an error. Event id 12017, to be exact.

I run

 get-exchangecertificate | fl

which lists the certificates I’ve got installed on this CAS:

cert01

There are two certs there – the first listed cert has been issued by an internal certification authority – the “issuer” value is cert.fabrikam.com – and is NOT self signed. I use it for IIS (owa, ecp, outlook anywhere, EAS). The second cert is the original self-signed cert that exchange produces when it’s installed on a server (the issuer is red-cas1, the local server name) and it’s only valid for imap and pop – two services I don’t bother configuring because none of our customers use them.

so lets get all the services onto a certificate that’s not about to expire. I  run

enable-exchangecertificate –thumbprint <thumbprint> -services pop,imap

using the thumbprint of the good certificate. <tip – to copy thumbprints and the like, right click in the powershell window, select mark, then highlight the thumbprint and click enter. Then when you are ready to type the thumbprint, right click anywhere again and hit “paste”>

Now if I run

get-exchangecertificate | fl

again I can see that all the services are now on my new cert, and the old self signed cert is doing nothing:

cert02

So now there are no services on the old cert, I can remove it using remove-exchangecertificate –t <thumbprint>

cert03

Has it gone? Hell yeah.

cert04

Now – here’s a word of warning. It’s perfectly possible to remove a certificate you are using. You get asked are you sure, and if you say “yes”, well… you’re the boss:

cert05

And you’ll need to reimport your cert.

cert06

In this cmdlet I import an certificate from a password protected .pfx file, which I created by exporting a certificate i’d previously requested on another server – this allows me to use the same certificate on a number of servers (for the same thing, obviously).

cert07

I then enable the cert for the required services – the first cmdlet (get-exchangecertificate) gets all the exchange certs on the server, I then run a couple of select cmdlets to create an array of psobjects which only contains the subjects and thumbprints of the original objects where the subject is like mail.fabrikam.com (select thumbprint, subject | ?{$_.subject –like “*mail.fabrikam.com*”}) – I then get the thumbprint of that psobject (select thumbprint)and pass it to the final cmdlet, which enables the chosen cert for iis, imap and pop (enable-exchangecertificate –services iis,imap,pop)

Finally I run get-exchangecertificate | fl  again to make sure that it has taken and the correct services are enabled.

I’d be a pretty sorry feller if I didn’t then run iisreset /restart at some point, to get iis to pick up the new cert.

Best practices around Mobile Devices and Exchange 2010

Introduction

Mobile devices have moved on since they first became popular ten years ago, and almost all access to Exchange mailboxes is via Exchange ActiveSync these days.  This is a protocol that allows mobile devices such as smartphones and tablets to connect to Exchange. Depending on the device software, ActiveSync is capable of mail, calendar and task connectivity, and also varying levels of device management, including “remote wipe” capability. There is a really good primer on ActiveSync here; don’t be put off because it’s for programmers – there’s a wealth of explanation and troubleshooting information further down.

Almost all mobile devices, including the latest Blackberry 10 devices and smartphones running the Jolla operating system now come with an implementation of ActiveSync, or can have a third party ActiveSync client installed such as NitroDesk.

Challenges

This presents a real challenge for Exchange. Different manufacturers implement ActiveSync in different ways, supporting a varying number of features. While ActiveSync is a Microsoft protocol, Microsoft have no control over how a third party manufacturer might choose to implement it, and so troubleshooting ActiveSync problems can become very tricky indeed. Before blaming Exchange for any issues that are experienced with third party devices, it’s always a good idea to have a good read of this article:

Current issues with Microsoft Exchange ActiveSync and third party devices.

I find that this is kept up to date and includes information on the latest versions on phone software. It is worth paying close attention to where the support boundaries lie, as well.

There is a reasonable table of implementations here, which lists which operating systems support which features. Unfortunately it doesn’t cover Exchange 2010 SP3 or Exchange 2013, but I guess we can’t have everything.

I cannot stress enough that new versions of OS software often break exchange – iOS 6.1, issues with iOS 7 and android have all had problems, some of them very serious. If you don’t block novel devices as a matter of course, then you really must keep an eye on the article above. Here it is again, just in case:  Current issues with Microsoft Exchange ActiveSync and third party devices.

Solutions

So, how can we make things better? Three areas require optimisation and management:

Users

By default, all users are enabled for ActiveSync access. this will probably not be suitable for your environment. disable users by running

set-casmailbox <username> -activesyncenabled $false

Check a user’s status by running

get-casmailbox <username>

 You may see reference to running

get-casmailbox | set-casmailbox –activesyncenabled $false

on a regular basis or a schedule to ensure that all users in an organisation are disabled, then running cmdlets to re-enable on a group by group or user basis. Alternatively, by using the cmdlet extension agent (http://www.ehloworld.com/194) and this handy script (http://www.flobee.net/automatically-disable-activesync-for-new-mailboxes-in-exchange-2010) we can provision new users with ActiveSync already disabled. You can fine-tune individual users or groups through the application of policies, below.

Servers

By default, client access servers are enabled for ActiveSync, and SSL and basic authentication are set. It may be necessary to configure other forms of authentication such as RSA two-factor to suitably protect access to the environment. There is more information on this here.

Devices

By default, if a user is enabled for active sync (and by default they are) they can configure any device that supports ActiveSync to synchronise with the exchange server. It is just as easy to connect a cheap Chinese clone phone as it is a top of the range iOS device, although arguably no more harmful. It is therefore essential that mailbox policies are configured to properly control access to the exchange server. As noted above, there are many implementations of the ActiveSync protocol that pay little regard to the performance of the server and the impact the device may have on other users.

There are three main ways to manage these; mailbox policies, throttling and ABQ.

Mailbox Policies

As noted above users and devices require mailbox policies to be set correctly to control access and use of the exchange environment. This is a good article to start from to understand mailbox policies, but remember that not all devices will implement all settings. Mailbox policies allow you to set things like minimum password length and complexity for the device, and whether users are allowed to download attachments, how often they must refresh their passwords and so on.

Like most policy objects in exchange, there is a built in default policy which should be copied rather than altered, and then the copies altered and applied to users according to their need – so you might choose to prevent a group of users downloading attachments, or strengthen the security settings for users who have access to particularly sensitive material.

Another approach is to set policies according to the device, so for example, you can create a policy for Windows Phone 8 devices and a separate policy for iOS devices. You may also choose to differentiate between devices that are owned by the customer, and devices that are owned by the users themselves. The drawback with this approach is that the policy is always applied to the user, not the device, so if the user has multiple devices, or changes devices, then a policy change will be required. Notice that a user can only be assigned to one ActiveSync mailbox policy at a time, and setting a new policy removes any previous policy.

Client throttling

Client throttling is a new feature in exchange 2010 that is intended to prevent single users or groups of users taking down exchange by performing an unwitting denial of service attack. Now, it may be that your execs have been unknowingly co-opted as zombie shock troops by some shadowy criminal cabal, possibly in a secret volcano base, or it may be that they have just installed the latest version of iOS – personally, I find it hard to tell the difference. And exchange can’t differentiate either; in either case, it just stops working. The client throttling policy has sections for each access method – owa, ActiveSync, MAPI etc. – so you can set limits for critical indicators above which exchange will delay or simply deny connections to that user. Again, as with most exchange policies there is a default that is automatically applied to all users, which should not be altered; it should be copied, and then the copies tailored and applied to individual users or groups of users. As above, you can only apply one policy per user. Client throttling was originally “off” in RTM, but was switched on in SP1 by default, and has been further tweaked since then, so your default policy will depend on your service pack level – service pack 3, right? You can track the effect throttling has on your exchange environment using the instructions in this article.

The throttling policy itself can be a little tricky to understand; Elan Shudnow’s blog does a pretty good job of deciphering it – but remember this is for an earlier version of Exchange. I’d thoroughly recommend anyone who was looking to implement non-default policies do some serious reading. The MSExchange.org article and Chapter 10 of “Exchange server 2010 inside out” are also very good.

ABQ

Allow/Block/Quarantine lists are another new feature of Exchange 2010 that are intended to work together with mailbox policies. The policies allow you to control device *features*, whereas ABQ lists allow you to control which devices are allowed access. There is a great post on the Ehlo blog about the lists, and the work that went into designing them, and some generalisations about the way they are expected to be used – whether to block by default and only allow certain types of devices, or whether to allow or quarantine by default, and only block devices known to cause issues – iOS 6.1 take a bow.

Quarantine does not stop the device from connecting to exchange, only downloading content – so items may be added to calendars and other folders while the device is in quarantine – this can catch people out! For a full explanation of how this works, and screenshots, have a look at the msexchange.org pair of articles on mobile device management with Exchange 2010.

All three tools should be used to ensure that servers are appropriately secured, users have a suitable level of functionality and the organisation is protected from novel devices.

The future

There are some great features in Windows InTune , such as the ability to manage android devices with SCCM; the only drawback is that it is a subscription service. In the meantime, ActiveSync it is. If you follow the advice above and use the provided tools to limit your implementation, you should be fine.