A Well-Feathered Nest

Rovio became known in 2009 with a small game that first appeared as a smartphone app. In the game, angry birds cause buildings to collapse with full wing power. Today, the Finnish company mainly generates revenue with T-shirts, cuddly toys, and key rings. Up to a million users a day visit the Angry Birds store in the cloud. At peak times, 5,000 visitors might be online at the same time, and the system processes multiple orders per second.

Rovio relies on Magento, an open source application for e-commerce, and combines it with Amazon Web Services (AWS) to operate the shop economically in quieter times. Rovio's Magento universe consists of the store with front ends for Europe, Asia, and America.

Integration to an external system exists to global warehouse management, shipping, a DRM server for licensing games, an email and gift card service, and the payment service Braintree.

Always Like New

Many online store owners can calculate the load on their systems fairly easily, but the Rovio store must be prepared for a very high volume of visitors when new products appear or a new ad campaign begins.

At the beginning of the project, Rovio, in collaboration with AOE media, decided to run the Angry Birds store in the form of agile software development and to add new releases constantly – a practice known as continuous deployment. Each element is treated as a software project and tested both automatically and manually; after each change, the system rebuilds the website completely from scratch.

Continuous deployment requires a sophisticated system for automatic quality assurance and rollout. It also needs to be capable of migrating new functions, modules, templates, and layout changes from test to production systems within minutes.

Rapid Testing

In the case of Rovio, the developers retrieve the current version on the site from system storage before starting work. Changes to the website flow back into backup storage. The modified site is then merged with the site operators' images and texts, which are waiting in backup storage.

After the first unit tests, the developers install the site on the "Latest System" where further tests follow (Figure 1). The site is then pushed to a staging server in the S3 cloud and, after final integration tests, ends up on the production system. All builds of the website, whether automatic or manually supported, are handled by the free Continuous Integration Server, Jenkins CI [1].

Figure 1: Using an automated test and build system, Rovio changes and tests its store before pushing it into the cloud.

AWS at Your Service

The entire Angry Birds store infrastructure is virtualized on Amazon's Web Services. At the touch of a button, AWS provides databases (MySQL, Oracle, MS SQL Server) in different versions and also offers the option of automatically merging upgrades to minor versions.

The capacity of the database server can be flexibly modified after setup, although this causes downtime – it takes about 10 minutes to get a new database instance up and running. At the press of a button, snapshots can be created, from which the operators can generate new databases as needed.

At the same time, Amazon permanently creates backups of the latest versions. Restore point in time lets you revert to earlier states. The company also guarantees high redundancy and availability, configurability of the Security Group [2] for hardening the database against external access, and a guaranteed number of I/O operations per second (provisioned IOPS). This setup provides a very good basis for a fully virtualized web store. CloudWatch [3], the integrated monitoring tool provides a comprehensive overview of the state of the system at any time.

Read Replicas

In Amazon's cloud, the store relies on the Amazon Relational Database Service (RDS), which greatly simplifies setting up, operating, and scaling relational databases [4]. It can be configured via the AWS Management Console, the interface for AWS. For read-intensive queries – and they occur far more frequently than writes – admins can, for example, call Create Read Replica to create multiple slave database instances (Figure 2). These instances rely on the master-slave MySQL replication model; however, Rovio uses the read replicas exclusively for generating reports.

Figure 2: You can set up read replicas of databases at the press of a button and buffer the load on the master instance.

The read slaves prevent database locks and deadlocks occurring when generating reports by introducing a kind of load balancing. Updates to the database are thus first written to the master instance and are then distributed asynchronously to the read slaves.

Caching

Caching relieves the load on the Magento database. The store achieves high availability, while at the same time minimizing database queries through the use of Varnish Cache [5]. The service is responsible for load balancing: it removes overloaded servers that respond slowly, or not at all, from load balancing and automatically replaces them with new instances. The AOE developers have also made pages statically cacheable in a large part of the Rovio store. Intelligent use of the browser cache and a reverse proxy cache means that clients handle some of the work. Shopping carts, the checkout process, payment functions, and user profiles are dynamically generated by the store.

If 5,000 visitors really do visit the store at the same time, the Varnish cache only queries the back end once per page or cache lifetime (TTL); it otherwise delivers cached content to the visitors. Varnish only allows dynamic requests through to Magento, which boosts speed all told about 400-fold. The application also offers potential for optimization: Reducing the log level, disabling logging functions, and regularly cleaning data (logs, expired cache content) also provide a speed boost.

To relieve the main database, the store's developers initially worked with a separate database until eventually replacing the cache back end with Redis. The NoSQL database stores key-value pairs directly in memory and thus accelerates the requests.

Setting Up Amazon's Databases

Configuring Amazon's database services is a fairly easy process using the Management Console. New instances are easily started, and the database is easily selected (Figure 3).

Figure 3: Setting up a database instance in the AWS Management Console.

Security Groups control access to the instances, thus ensuring the necessary security. Admins can use the Groups parameters where needed to modify individual database configuration parameters. Once AWS is set up, it defines database parameters, such as the host, user, and password specifically for the environment (production or staging) and then enables the database for online access. For Magento, it ultimately does not matter whether its own database runs in a virtual environment such as Amazon RDS or on a dedicated server.

Importing data to the cloud is handled by a MySQL client, as you might expect. The system is limited by the partly static nature of the database. It cannot dynamically field surprising visitor storms, even though the application servers scale automatically. It is thus important to compute your needs correctly on a project-specific basis, because changes in the operation involve some downtime. The available instance sizes offer 5 to 3,370GB of storage with selectable CPU power and RAM.

New to Amazon is an option for guaranteed I/O performance [6], which does cause additional cost but offers significantly improved performance. Standby replicas [7] ensure more reliability and seamless operation without downtime; Amazon creates standby replicas automatically in separate data centers, which it refers to as Availability Zones (AZ). They allow you to handle and compensate for planned maintenance windows for the database. If the main data center fails, the standby database in another zone automatically steps in (hot standby), so that operation continues without intervention.

The Whole Enchilada

Billing of web services is per hour of runtime, and the rates differ depending on the database instance class size [8] you use. The Angry Birds store uses different services worldwide: EC2 (Elastic Cloud 2) boots or shuts down virtualized servers. S3 stores data, such as images, which it keeps in long-term storage and distributes to the various servers. RDS manages databases; CloudFront delivers product images or skins as quickly as possible. Elastic Load Balancing (ELB) distributes the traffic and takes care of SSL termination, while the DNS service, Route 53, provides for cross-server or static IP addresses.

If you assign servers to groups (arrays), you get an overall picture of Rovio's AWS cloud architecture (Figure 4). The developers run two instances of the deploy infrastructure shown in the green area – one with the new version and the other with the previous version of the store. To roll back to an older version, they only need to switch Amazon's Route 53 DNS service [9].

Figure 4: The Angry Birds store is integrated with a complex structure of services that help it survive peak loads.

As a front-end array, only the Magento server (and associated databases) are otherwise used. For safety reasons, servers are redundant – if one fails, there is no need to boot another. The task of the front-end array is to process all dynamic requests (shopping carts, sessions, accounts) and guide visitors to checkout. The Varnish Array consists of two EC2 instances that run caching – the second server is also used as a fail-safe. The back-end array runs on a server and manages Rovio's products in Magento. Finally, a worker array handles all background processes.

Conclusions

AWS lets operators design stores that do not collapse when faced with peak loads  – that is, if you have the budget to build one. The store owner does not need to worry about revenue hits, and the admins have less work with the database. If sales are quiet, resources can be released manually or automatically. And, there is no need to do without the use of free software. At the same time, the combination of AWS and Magento with replicated databases creates a scalable and robust solution. After all, a collapsing building is for the birds.