The Art and Science of Sizing Exchange 2003 (Part 1)
I’m pretty sure that is, at least, arguable that you can call it art. I don’t know either if it’s deep enough to be called a science. One thing I can assure you is that sizing an Exchange server can be a complex task and it requires not only the knowledge, but also a dose of sensibility and some previous experience with the Microsoft Server family of products. Although there are some pretty good documents from Microsoft about this subject, I’ll try to condense them all and expose the main guidelines along this three part article.
Note: This article is published with permission from www.msexchange.org
The calculation of hardware requirements for a server is not straight math. Although we can use real live data, the truth is that it also requires some dose of estimation for some of the current and future needs.
Let me tell you in advance that, among all components to size, storage is undoubtedly the most critical, since it’s the most common cause of bottlenecks. I’ll cover storage sizing in part 2 of this article. So, what else is there to plan?
A server is a pretty complicated piece of hardware, so in this article I’ll cover the sizing process of only the following components:
The methodology we’ll use is the following:
- Determine user profile – user profile determination can be accomplished by using some live measures from a production server or can be estimated. For either cases be aware that there is always an associated error.
- Size the appropriate hardware – based on user profile we can now calculate the processor, memory, network and storage requirements.
- Validate the final configuration – the most critical component to validate is the storage system, since this is the most common cause of performance bottlenecks. Microsoft provides us some tools we can use to simulate mail load and validate the whole design.
Also I’m going to mention some tools available (from Microsoft and third-party) that can ease your job. Although some of them are quite powerful and can do most of the hard work for you, I strongly advise you to read the rest of this article and all of the literature available (knowledge is never too much) in order to understand what those nice pieces of software are actually doing. And as I said before, sizing Exchange can require some dose of sensibility and human factor that no automated tool could ever provide you with.
Depending of the complexity of the messaging solution you’re planning, there may exist several kinds of server roles, being the most common the mailbox server, also known as a back-end. But there are plenty more as shown on the next table:
Server that holds the users mailboxes
Server that handles internet protocols: HTTP (OWA), POP3 e IMAP.
Inbound SMTP gateways
X.400 / legacy connector servers
Expansion servers route messages that are sent to a distribution list for each of the recipient objects in that group
Server that holds Public Folders
Server responsible for the Offline Address Book (OAB) generation
Since the requirements are different for each kind of role, we cannot use the same calculations for all, so we must adapt the sizing process to the reality of the solution being implemented.
For the purpose of this article I’ll focus mainly the back-end and front-end servers, which are the most common.
To correctly size Exchange hardware, you’ll have to know in advance user profiles, sometimes also called usage patterns. If you already have live data that you can measure (in case you are planning an upgrade or migration), this task will become easier. If you are planning a new Exchange implementation, you’ll have to estimate the profiles of your users, using some standard well accepted measures will see further ahead in this article.
User profile is determined by the following two key metrics:
- Megacycles/mailbox – Megacycles per second, per mailbox tells us the raw processor usage required per mailbox. You can obtain it by measuring a production server for 2 hours, during the peak period.
(megacycles/mailbox) = (average CPU usage) x (speed of processors in megacycles) x (number of processors) ÷ (number of mailboxes)
For example, if a user uses one megacycle/second during peak operation and there are 1,000 users on the server (1,000 megacycles/second), a single 2,000-MHz processor operates at 50 percent CPU usage.
- IOPS/mailbox – Input/output per second, per mailbox. The raw database (DB) disk usage (input/output per second) required per user that is measured during the peak two-hour period on a production server. This metric does not include transaction log input/output (I/O) operations.
(IOPS/mailbox) = (average disk transfers) ÷ (number of mailboxes)
For example, if each mailbox uses 0.5 DB IOPS during peak activity and there are 1,000 users homed on the server, there are 500 DB IOPS. IOPS/mailbox metrics are based on random read/write Exchange database I/O operations.
Note: there are some variations of this 2 metrics. You will probably find some literature using /user instead of /mailbox (megacycles/user, IOPS/user). If you have a close 1:1 relation between active users and mailboxes, that won’t have a significant impact on the calculations. If the mailbox count exceeds the number of users and if some of those mailboxes are not used very often, this is a situation where the use of active users instead of number of mailboxes will be more appropriate.
As you probably guessed by now, these two metrics will become quite useful to calculate processor and storage requirements, as I’ll explain further ahead.
So, by now you’re probably thinking about those cases where you don’t have Exchange installed yet to take some measures. For those cases you can use the information of the next table as a guideline. These profiles represent mailbox access for the “average user” Outlook (or MAPI-based) client.
|Mailbox Profile||IOPS||Megacycles||Message Volume||Mailbox Size|
|Light||0,18||0,75||10 sent / 50 received||< 50 MB|
|Average||0,4||1,9||20 sent / 100 received||50 MB|
|Heavy||0,75||2,5||30 sent / 100 received||100 MB|
Remember that these guidelines are only valid for the time being and for the near future. As technology evolves, user demands will be higher, resulting in different profiles. That would also be the case if you have some particular scenarios, such as Blackberry devices or a massive use of desktop search tools within your organization.
The use of 3rd-party software, such as anti-virus, anti-spam, faxing software, etc. may have a significant impact in usage profile, so make sure you have that in account.
From all the server roles, the more demanding in terms of processor is usually the back-end, which serves the MAPI clients. To correctly size CPU needs for your hardware follow these rules:
- Processor usage should not be more than 60% during peak-time. Remember that 80% or more of consistent CPU utilization is considered a bottleneck.
- One processor for every 1,000 users
- Whenever possible use Xeon/Opteron processors
- Use Hyper-Threading
- Raising the FSB frequency has more impact on performance than raising CPU clock
- For front-end servers, 1-2 processors is usually enough
The following table shows the recommended number of processors versus the number of active users for a back-end server:
|Number of mailboxes||Number of processors|
|500 – 1.000||1-2|
|1.000 – 4.000||2-4|
If you have pre-determined your usage profile (by estimation or by measuring live data), you can use the following formula to more exactly determine your processor needs:
0.80 × (speed of CPU in megacycles) × (number of processors) = (megacycles/mailbox) × (number of mailboxes)
The reason we use the 0.80 factor it’s because the threshold for CPU utilization before it’s considered a bottleneck is 80%. If you want to take into account future growth you should use a smaller factor.
Gone are the days when memory was the main cause of deterioration of performance of an Exchange Server. It’s not that Exchange has dropped its memory hunger, it’s just that nowadays the price of this component is so low that there’s no reason to have a production server without a couple of gigabytes.
The main responsible for memory consumption is the store.exe process, the core of a back-end server. Since it starts it will try to grab as much RAM as it’s possible. This behavior is by design and should not be confused with a memory leak. Exchange can return memory to the operating system using an algorithm known as Dynamic Buffer Allocation. You can also limit the maximum amount of memory that Exchange uses by reducing the ESE Buffer size.
To size your memory needs you’ll have to do some estimation, unless you have previously measured a live server, identical to the one you’re planning.
The main factors that have an impact on memory are:
- Number of users
- User profile (message volume, size of the mailbox, features used, utilization time)
- Third-party software installed
- Number of Storage Groups
- Number of databases
- Size of ESE buffer cache
To correctly size memory requirements we can use the following rule: 1 MByte for each user, meaning that 1,000 users will require 1GB of memory. You can probably live well with less than 1 Mbyte per user, so if you have a budget issue you can try cutting that into half, meaning 512 Kbytes/user.
For front-end servers you won’t normally need more than 2GB of RAM.
As a final note I would like to remind you that Exchange 200x cannot use more than 4GB of physical memory, which corresponds to the 32-bits physical address space. Exchange Server does not support instancing, Physical Address Extension (PAE), or Address Windowing Extensions (AWE). Therefore, 4 GB of RAM is the maximum amount of memory that an Exchange Server computer can efficiently use.
For servers with 1GB or more, additional steps are required in order to optimize memory use.
- Add the switches /3GB and /USERVA=3030 to boot.ini.
- Configure the HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\HeapDeCommitFreeBlockThreshold registry value to 0x00040000.
You can (and should!) also use Exchange Server Best Practices Analyzer (ExBPA) Tool to check your server after it’s installed. The ExBPA Tool will give you all the recommendations regarding efficient use of memory.
Since most of the servers ship today with Gigabit network interface cards (NICs), sizing network becomes a really easy task. You’ll just have to make sure that the rest of your network can handle the messaging traffic you’re predicting.
Typically, 100 Mbit full duplex is sufficient for back-end servers. Consider gigabit only in these situations:
- Front-ends and bridge-heads with a high volume of traffic
- Streaming network backup and restore operations
- Internet SCSI (iSCSI) or network-attached storage
- High concentration of Outlook Web Access, Post Office Protocol (POP3), or Internet Message Access Protocol (IMAP4) users (>5000 users)
If you are using security mechanisms such as IPSec, consider NICs with IPSec offload.
If your network is well documented, carefully analyze it and place your Exchange servers in order not to saturate some physical segments. If necessary, make the required changes before the deployment of your solution.
In this first part I covered the sizing of 3 of the main components of an Exchange server: processor, memory and network. I’ll leave the most critical component, storage, to the next part.
I’d like to go back to the beginning of this article: sizing is indeed a complex task, so if you’re not comfortable doing it, don’t be afraid to ask someone else or looking for help among the technical communities on the internet. There are lots of people willing to help you… for free!
Note: This article is published with permission from www.msexchange.org