Scaling Up, Scaling Out

An important characteristic of a cloud application is the ability to respond to increased demand by scaling the application either by scaling up or scaling out. Scaling up refers to increasing the capacity of the server (such as adding CPU cores or memory) to meet increased demand while scaling out refers to adding new instances to the application to handle the increased demand. While the scaling out option is preferred in a cloud environment because it can be achieved without reconfiguring or interrupting existing instances of the application, both methods are available within Azure.

Note: Scaling options are seriously limited within Azure for free services such as the free Azure Web Sites hosting plan mode. If you change your hosting plan mode to experiment with scaling features, be sure that you return it to free mode to avoid incurring unexpected charges. Many of the scaling options can be explored in the Azure Management Portal by making changes and then selecting the Discard command instead of Save.

Sizing Instances

Azure offers three options for the sizing of VMs hosted with the Basic or Standard web hosting mode (other options are available for VMs which are not exposed through the Web Sites feature). These options include:

• Small instances featuring a single processor core and 1.75GB memory
• Medium instances featuring two processor cores and 3.5GB memory
• Large instances featuring four cores and 7GB memory

As noted earlier, preference should be given to using the smallest appropriate instance size for your application and scaling out when demand warrants it. One potential use case for scaling up rather than out would be if a third-party component is being used that is licensed per instance and adding more instances would create a significant increase in the licensing costs.

To change the instance size for an Azure Web Site hosted using a supported tier, select the site in the Azure Management Portal by clicking on the website name, which will bring up details for the selected website. Use the top navigation to navigate to the “Scale” tab and find the “Instance Size” drop-down menu (shown in Figure 9) and use it to specify the desired instance size. After you’re satisfied with your changes, select Save at the bottom of the window to persist your changes and Azure will initiate resizing of the instance.

9. Scale up

Adding/Removing Instances

Azure Web Sites using the Shared, Basic or Standard pricing tier have the ability to scale out by adding and removing instances of the same size to the resource pool serving the site. The number of available instances for scaling per site varies depending on the specific hosting plan tier:

• Shared: Sites can scale to a maximum of six shared instances (no dedicated Compute resources)
• Basic: Sites can scale to a maximum of three dedicated VMs
• Standard: Sites can scale to a maximum of 10 dedicated VMs

To scale out an Azure Web Site with a supported hosting plan tier, navigate to the Scaling options for the site as described in the previous section and find the Instance Count slider depicted in Figure 10. Either drag the slider to the desired value or type the selected value into the text box and then select Save to persist the configuration. After changes have been saved, the Azure fabric controller will work in the background to provision or decommission instances to reflect the newly selected instance count.

10. Scale out

Auto-scaling

The scaling mechanisms that have been discussed so far have focused around the person who is managing an Azure account, logging into the Azure Management Portal application, and manually either changing instance sizes or adding/removing instances through the UI. But Azure also makes available a powerful feature to help balance cost and performance by automatically adjusting the number of instances that are running based on a few available controlling factors. This feature is called “Auto Scale” and, for Azure Web Sites, the controlling factors can be either based on time or CPU utilization percentage.

Schedule-based scaling

When controlling scaling based on time, the Azure Web Site operator can take advantage of knowing when periods of particularly high or low utilization will occur, and then make sure that resources are added or removed from the solution to respond to these periods.

One example of where schedule-based scaling would be used is the usage pattern of an enrollment website for a health insurance company. Most insurance companies have set periods of time on the calendar that mark open enrollment, the period in which consumers of the insurance product can obtain new insurance policies or make changes to existing coverage. Outside of these periods, consumers must meet very stringent criteria to be eligible for such changes. Let’s assume, for illustrative purposes, that an insurance company with two million subscribers has an open-enrollment period lasting two weeks. During those two weeks, it can be expected that the company’s enrollment website will be actively used by nearly all two million subscribers as they examine options and select the coming year’s benefits. Outside of that two-week period, the website will be lightly used by a few subscribers at a time as they are newly hired or have other events that make them eligible for changes outside the enrollment period. With an Azure Web Site at the Standard hosting tier, an instance count of 10 can be specified for during the enrollment period and 2 for all other times.

Default Schedules

By default, Azure’s auto-scale feature allows you to enable schedules that establish unique periods for day, night, weekday and weekend day. The day/night divide is configurable to provide control over starting and ending time for the day period as well as the time zone that should be used as the point of reference. This feature may be used to ensure that server resources are added during the working day and then deallocated during off hours.

A great example of how the default schedules could save an Azure subscriber money on their bill is a scenario that was described to me when I was first introduced to Azure. The scenario involved a customer who processed trading data from the stock market, which required a lot of processing power during the trading day but literally dropped to needing no processing as soon as the markets closed every day. The auto-scale feature didn’t exist at the time so the subscriber wrote PowerShell scripts which were scheduled on-premises to provision and deallocate Azure Compute instances on a schedule that allowed their Azure costs to be close to zero during the time periods where the processing power was not needed and handled the massive load during the day.

Custom Schedules

In addition to the default schedules, Azure subscribers can configure additional periods by specifying a start date/time and end date/time and providing a name for the custom schedule. This feature would be used to specify the time period during which heavy volume is expected in the open enrollment example earlier.

Configuring Schedule-based Scaling

To scale an Azure Web Site that’s hosted using the Standard web hosting plan tier based on schedule, select the site in the Azure Management Portal by clicking on the site name, which will bring up details for the selected site. Use the top navigation to navigate to the Scale tab and find the scheduling options, which consist of a drop-down menu that allows you to select from existing schedules and a command to create new schedule periods or edit existing ones. These are shown in Figure 11:

11. Scale by schedule options

Assuming that schedule periods have not previously been established, you will first invoke the “Set up schedule times” command, displaying the dialog shown in Figure 12:

12. Set up schedule times dialog

In the dialog, you have the options of enabling the default schedules for day/night and weekday/weekend as well as creating additional named periods that require specification of name, start date, start time, end date, and end time. After you have worked with the schedule periods in the dialog to create periods that are appropriate for the anticipated load profile of your website, accept the changes. You will be returned to the scale configuration page with the new scheduling options now displayed in the schedule drop-down menu as shown in Figure 13:

13. Select schedule

For each schedule presented in the drop-down menu, you can now select the schedule and configure the instance count desired to be active when that schedule is current. The Azure fabric scans for scaling about every five minutes, so scaling may not occur precisely at the configured time.

Scaling by Metric

Many times it is more difficult to predict scaling needs simply by saying, “I need five instances during the day and three at night.” Instead, the definition of when you need more or fewer resources should be driven by some measure that’s tied to how busy your website is. To meet this need, Azure offers the “scale by metric” option in the auto-scale feature. This measure varies by the individual service being measured. For example, some Azure Web Sites are measured strictly on CPU utilization percentage while other services can be measured for auto-scale based on the number of messages waiting to be processed on a queue.

Regardless of the chosen measure, the Azure controller follows an algorithm similar to what is described in the following pseudo-code for each pass of the scheduling loop:

The ability to scale by metric is a powerful tool that allows you to finely tune a balance between saving on your Azure bill and ensuring that your application responds well to user requests.

Configuring Metric-based Scaling

To scale an Azure Web Site hosted using the Standard web hosting plan tier based on schedule, select the website in the Azure Management Portal by clicking on the website name, which will bring up details for the selected website. Use the top navigation bar to navigate to the Scale tab and find the “scale by metric” section which is shown in Figure 14:

14. Scale by metric options

By default, the “None” option is selected within the options but selecting “CPU” (which is currently the only other option for Azure Web Sites) changes the “instance count” slider to accept a range rather than a scalar value, and a “target CPU” slider is presented which allows you to select the range of CPU utilization percentage considered to be appropriate for your application. The default range of 60 to 80 percent is generally a good value to start with, unless you are aware of specific needs of your application that drive selection of other values. The CPU scaling options are shown in Figure 15:

15. Scale by CPU options

Selecting Appropriate Values for Scale by CPU

The selection of appropriate scaling values may seem rather obvious and the initial answer from many people would be to include the following characteristics:

• Minimum instance count of 1 to minimize cost if load allows
• Maximum instance count of either what your plan allows for or some lower limit to meet what you’ve chosen as a maximum cost
• Minimum target CPU close to 100 percent to allow instances to drop off as soon as possible
• Maximum target CPU at 100 percent to ensure additional instances are only paid for when maximizing the utilization of the resources already being paid for

This approach seems pretty solid as a list of bullet points and will make absolute sense to somebody in charge of paying the bills. But the translation from bullet points to the real world reveals some additional variables that should be factored into the decision about what is right for you:

• Minimum instance count: Many website operators will want to ensure at least one additional instance of their website beyond what is expected to be needed to ensure rapid response to an instance dropping unexpectedly without a noticeable hit to website visitors. The adage “Three is two, two is one, one is none” applies here so, for a website with needs that justify a Standard hosting plan mode and configuration of scaling options at all, I would never recommend less than two instances.
• Maximum instance count: There’s nothing I would change about this one. The maximum instance count should either be as many instances allowed to you, or you should work out the maximum amount that you are willing to pay for Azure Web Site compute time, limiting the number of instances to a value that does not potentially have you exceeding this value.
• Minimum target CPU: This value ideally should be low enough to clearly indicate that the high-volume peak has passed on and that dropping an instance won’t immediately throw you back into a CPU utilization that will have it again provisioned on the next pass.
• Maximum target CPU: This value should be set with the understanding that sampling is done at intervals and, if allowed to be too close to 100 percent, you could spike immediately following a sampling interval, leaving your users with a floundering website while waiting for the scheduler to notice that another instance needs to be provisioned and spun up.

Combined Scaling by Schedule and Metric

The factors that should drive selection of appropriate configuration options for scaling by metric are not only more complicated than one would initially think, but can often be impacted by known periods of website activity that differ from each other in anticipated volume. During a period of low volume of traffic, it might be a reasonable, cost-saving measure to have the minimum instance count of the website be a lower value than at other times while, during a period that is anticipated to be very busy, it may be worth the additional cost to have a higher instance count in order to avoid times where the website becomes less responsive while additional instances are provisioned. Azure meets this need by allowing a separate set of Scale by Metric options to be configured for each of the time periods that you have created.

Summary

In this chapter, you learned about scaling, which is one of the most important defining features of a cloud platform. You learned the difference between scaling up and scaling out, and saw Azure features for dynamically adding and removing instances that make scaling out the scaling option of choice on the platform.