An aggregation of our Blog Roll, made up of acmqueue authors.   more

All Postings, Werner Vogels:  (240 posts)

Source blog: All Things Distributed

Tue, 13 Nov 2018 08:00:00 UTC

Ciao Milano! ? An AWS Region is coming to Italy!

Posted By Werner Vogels

Today, I am happy to announce our plans to open a new AWS Region in Italy! The AWS Europe (Milan) Region is the 25th AWS Region that we've announced globally. It's the sixth AWS Region in Europe, joining existing regions in France, Germany, Ireland, the UK, and the new Region that we recently announced in Sweden. The AWS Europe (Milan) Region will have three Availability Zones and be ready for customers in early 2020. Currently we have 57 Availability Zones across 19 technology infrastructure Regions. As of this announcement, another five AWS Regions and 15 Availability Zones are coming over the next year in Bahrain, Hong Kong SAR, Italy, South Africa, and Sweden.

Mon, 12 Nov 2018 21:00:00 UTC

Expanding the Cloud ? The Second AWS GovCloud (US) Region, AWS GovCloud (US-East)

Posted By Werner Vogels

Today, I'm happy to announce that the AWS GovCloud (US-East) Region, our 19th global infrastructure Region, is now available for use by customers in the US. With this launch, AWS now provides 57 Availability Zones, with another 12 zones and four Regions in Bahrain, Cape Town, Hong Kong SAR, and Stockholm expected to come online by 2020. The AWS GovCloud (US-East) Region is our second AWS GovCloud (US) Region, joining AWS GovCloud (US-West) to further help US government agencies, the contractors that serve them, and organizations in highly regulated industries move more of their workloads to the AWS Cloud by implementing a number of US government-specific regulatory requirements.

Thu, 25 Oct 2018 17:00:00 UTC

Expanding the AWS Cloud ? An AWS Region is coming to South Africa!

Posted By Werner Vogels

Expanding the AWS Cloud?An AWS Region is coming to South Africa! Today, I am excited to announce our plans to open a new AWS Region in South Africa! AWS is committed to South Africa's transformation. The AWS Africa (Cape Town) Region is another milestone of our growth and part of our long-term investment in and commitment to the country. It is our first Region in Africa, and we're shooting to have it ready in the first half of 2020. The new AWS Africa (Cape Town) Region will have three Availability Zones and provide lower latency to end users across Sub-Saharan Africa.

Wed, 01 Aug 2018 14:00:00 UTC

Make your voice count by simply saying, "Alexa, let's chat."

Posted By Werner Vogels

A while back I wrote about the Alexa Prize, a university competition where participating teams are creating socialbots focused on advancing computer to human interaction. We are now in year two, heading into the final stretch for 2018 and I thought I would give everyone an update. For those who aren't familiar, Alexa Prize teams use customer feedback to advance several areas of conversational AI, with the grand challenge being a socialbotthat can engage coherently for 20 minutes in a fun, high-quality conversation on popular topics such as entertainment, sports, technology, and fashion. Teams are thinking big about how to make strides in areas including knowledge acquisition, natural language understanding and generation, context modeling, common sense reasoning, and dialog planning.

Thu, 21 Jun 2018 14:00:00 UTC

A one size fits all database doesn't fit anyone

Posted By Werner Vogels

A common question that I get is why do we offer so many database products? The answer for me is simple: Developers want their applications to be well architected and scale effectively. To do this, they need to be able to use multiple databases and data models within the same application. Seldom can one database fit the needs of multiple distinct use cases. The days of the one-size-fits-all monolithic database are behind us, and developers are now building highly distributed applications using a multitude of purpose-built databases. Developers are doing what they do best: breaking complex applications into smaller pieces and then picking the best tool to solve each problem.

Mon, 21 May 2018 17:00:00 UTC

The workplace of the future

Posted By Werner Vogels

This article titled "Die Arbeitswelt der Zukunft" appeared in German last week in the "Digitalisierung" column of Wirtschaftwoche. The workplace of the future We already have an idea of how digitalization, and above all new technologies like machine learning, big-data analytics or IoT, will change companies' business models  and are already changing them on a wide scale. So now's the time to examine more closely how different facets of the workplace will look and the role humans will have. In fact, the future is already here  but it's still not evenly distributed. Science fiction author William Gibson said that nearly 20 years ago.

Wed, 11 Apr 2018 15:00:00 UTC

Changing the calculus of containers in the cloud

Posted By Werner Vogels

I wrote to you over two years ago about what happens under the hood of Amazon ECS. Last year at re:Invent, we launched AWS Fargate, and today, I want to explore how Fargate fundamentally changes the landscape of container technology. I spend a lot of time talking to our customers and leaders at Amazon about innovation. One of the things I've noticed is that ideas and technologies which dramatically change the way we do things are rarely new. They're often the combination of an existing concept with an approach, technology, or capability in a particular way that's never been successfully tried before.

Mon, 26 Mar 2018 17:00:00 UTC

Looking back at 10 years of compartmentalization at AWS

Posted By Werner Vogels

At AWS, we don't mark many anniversaries. But every year when March 14th comes around, it's a good reminder that Amazon S3 originally launched on Pi Day, March 14, 2006. The Amazon S3 team still celebrate with homemade pies! March 26, 2008 doesn't have any delicious desserts associated with it, but that's the day when we launched Availability Zones for Amazon EC2. A concept that has changed infrastructure architecture is now at the core of both AWS and customer reliability and operations. Powering the virtual instances and other resources that make up the AWS Cloud are real physical data centers with AWS servers in them.

Mon, 19 Mar 2018 16:00:00 UTC

Infinitely scalable machine learning with Amazon SageMaker

Posted By Werner Vogels

In machine learning, more is usually more. For example, training on more data means more accurate models. At AWS, we continue to strive to enable builders to build cutting-edge technologies faster in a secure, reliable, and scalable fashion. Machine learning is one such transformational technology that is top of mind not only for CIOs and CEOs, but also developers and data scientists. Last re:Invent, to make the problem of authoring, training, and hosting ML models easier, faster, and more reliable, we launched Amazon SageMaker. Now, thousands of customers are trying Amazon SageMaker and building ML models on top of their data lakes in AWS.

Mon, 12 Mar 2018 13:00:00 UTC

Unlocking Enterprise systems using voice

Posted By Werner Vogels

At Amazon, we are heavily invested in machine learning (ML), and are developing new tools to help developers quickly and easily build, train, and deploy ML models. The power of ML is in its ability to unlock a new set of capabilities that create value for consumers and businesses. A great example of this is the way we are using ML to deal with one of the world's biggest and most tangled datasets: human speech. Voice-driven conversation has always been the most natural way for us to communicate. Conversations are personal and they convey context, which helps us to understand each other.

Wed, 20 Dec 2017 17:00:00 UTC

Rethinking the 'production' of data

Posted By Werner Vogels

This article titled "Daten müssen strategischer Teil des Geschäfts werden" appeared in German last week in the "IT und Datenproduktion" column of Wirtschaftwoche. How companies can use ideas from mass production to create business with data Strategically, IT doesn't matter. That was the provocative thesis of a much-talked-about article from 2003 in the Harvard Business Review by the US publicist Nicolas Carr. Back then, companies spent more than half of their entire investment for their IT, in a non-differentiating way. In a world in which tools are equally accessible for every company, they wouldn't offer any competitive advantage  so went the argument.

Tue, 19 Dec 2017 04:00:00 UTC

'Paris s'éveille'! Introducing the AWS EU (Paris) Region

Posted By Werner Vogels

Today, I'm happy to announce that the AWS EU (Paris) Region, our 18th technology infrastructure Region globally, is now generally available for use by customers worldwide. With this launch, AWS now provides 49 Availability Zones, with another 12 Availability Zones and four Regions in Bahrain, Hong Kong, Sweden, and a second AWS GovCloud (US) Region expected to come online by early 2019. In France, you can find one of the most vibrant startup ecosystems in the world, a strong research community, excellent energy, telecom, and transportation infrastructure, a very strong agriculture and food industry, and some of the most influential luxury brands in the world.

Tue, 12 Dec 2017 04:00:00 UTC

Expanding the AWS Cloud: Introducing the AWS China (Ningxia) Region

Posted By Werner Vogels

Today, I am happy to announce the general availability of AWS China (Ningxia) Region, operated by Ningxia Western Cloud Data Technology Co. Ltd. (NWCD). This is our 17th Region globally, and the second in China. To comply with China's legal and regulatory requirements, AWS has formed a strategic technology collaboration with NWCD to operate and provide services from the AWS China (Ningxia) Region. Founded in 2015, NWCD is a licensed data center and cloud services provider, based in Ningxia, China. Coupled with the AWS China (Beijing) Region operated by Sinnet, the AWS China (Ningxia) Region, operated by NWCD, serves as the foundation for new cloud initiatives in China, especially in Western China.

Wed, 29 Nov 2017 18:00:00 UTC

Accelerate Machine Learning with Amazon SageMaker

Posted By Werner Vogels

Applications based on machine learning (ML) can provide tremendous business value. However, many developers find them difficult to build and deploy. As there are few individuals with this expertise, an easier process presents a significant opportunity for companies who want to accelerate their ML usage. Though the AWS Cloud gives you access to the storage and processing power required for ML, the process for building, training, and deploying ML models has unique challenges that often block successful use of this powerful new technology. The challenges begin with collecting, cleaning, and formatting training data. After the dataset is created, you must scale the processing to handle the data, which can often be a blocker.

Tue, 21 Nov 2017 18:00:00 UTC

Scaling Amazon ElastiCache for Redis with Online Cluster Resizing

Posted By Werner Vogels

Amazon ElastiCache embodies much of what makes fast data a reality for customers looking to process high volume data at incredible rates, faster than traditional databases can manage. Developers love the performance, simplicity, and in-memory capabilities of Redis, making it among the most popular NoSQL key-value stores. Redis's microsecond latency has made it a de facto choice for caching. Its support for advanced data structures (for example, lists, sets, and sorted sets) also enables a variety of in-memory use cases such as leaderboards, in-memory analytics, messaging, and more. Four years ago, as part of our AWS fast data journey, we introduced Amazon ElastiCache for Redis, a fully managed, in-memory data store that operates at microsecond latency.

Mon, 02 Oct 2017 18:00:00 UTC

A Decade of Dynamo: Powering the next wave of high-performance, internet-scale applications

Posted By Werner Vogels

Today marks the 10 year anniversary of Amazon's Dynamo whitepaper, a milestone that made me reflect on how much innovation has occurred in the area of databases over the last decade and a good reminder on why taking a customer obsessed approach to solving hard problems can have lasting impact beyond your original expectations. It all started in 2004 when Amazon was running Oracle's enterprise edition with clustering and replication. We had an advanced team of database administrators and access to top experts within Oracle. We were pushing the limits of what was a leading commercial database at the time and were unable to sustain the availability, scalability and performance needs that our growing Amazon business demanded.

Mon, 25 Sep 2017 17:00:00 UTC

As-Salaam-Alaikum: The cloud arrives in the Middle East!

Posted By Werner Vogels

Today, I am excited to announce plans for Amazon Web Services (AWS) to bring an infrastructure Region to the Middle East! This move is another milestone in our global expansion and mission to bring flexible, scalable, and secure cloud computing infrastructure to organizations around the world. Based in Bahrain, this will be the first Region for AWS in the Middle East. The Region will be in the heart of Gulf Cooperation Council (GCC) countries, and we're aiming to have it ready by early 2019. This Region will consist of three Availability Zones at launch, and it will provide even lower latency to users across the Middle East.

Mon, 04 Sep 2017 17:00:00 UTC

AI for everyone - How companies can benefit from the advance of machine learning

Posted By Werner Vogels

This article titled "Wie Unternehmen vom Vormarsch des maschinellen Lernens profitieren können" appeared in German last week in the "Digitaliserung" column of Wirtschaftwoche. When a technology has its breakthrough, can often only be determined in hindsight. In the case of artificial intelligence (AI) and machine learning (ML), this is different. ML is that part of AI that describes rules and recognizes patterns from large amounts of data in order to predict future data. Both concepts are virtually omnipresent and at the top of most buzzword rankings. Personally, I think  and this is clearly linked to the rise of AI and ML  that there has never been a better time than today to develop smart applications and use them.

Fri, 30 Jun 2017 18:00:00 UTC

Improving Customer Service with Amazon Connect and Amazon Lex

Posted By Werner Vogels

Customer service is central to the overall customer experience that all consumers are familiar with when communicating with companies. That experience is often tested when we need to ask for help or have a question to be answered. Unfortunately, we've become accustomed to providing the same information multiple times, waiting on hold, and generally spending a lot more time than we expected to resolve our issue when we call customer service. When you call for customer assistance, you often need to wait for an agent to become available after navigating a set of menus. This means that you're going to wait on hold regardless of whether your issue is simple or complex.

Wed, 28 Jun 2017 17:00:00 UTC

Stop waiting for perfection and learn from your mistakes

Posted By Werner Vogels

This article titled "Wartet nicht auf Perfektion  lernt aus euren Fehlern!" appeared in German last week in the "Digitaliserung" column of Wirtschaftwoche. "Man errs as long as he doth strive." Goethe, the German prince of poets, knew that already more than 200 years ago. His words still ring true today, but with a crucial difference: Striving alone is not enough. You have to strive faster than the rest. And while there's nothing wrong with striving for perfection, in today's digital world you can no longer wait until your products are near perfection before offering them to your customers. If so, you will fall behind in your market.

Wed, 21 Jun 2017 18:00:00 UTC

Amazon DynamoDB Accelerator (DAX): Speed Up DynamoDB Response Times from Milliseconds to Microseconds without Application Rewrite.

Posted By Werner Vogels

Today, I'm excited to announce the general availability of Amazon DynamoDB Accelerator (DAX), a fully managed, highly available, in-memory cache that can speed up DynamoDB response times from milliseconds to microseconds, even at millions of requests per second. You can add DAX to your existing DynamoDB applications with just a few clicks in the AWS Management Console  no application rewrites required. DynamoDB has come a long way in the 5 years since we announced its availability in January 2012. As we said at the time, DynamoDB was a result of 15 years of learning in the area of large scale non-relational databases and cloud services.

Wed, 21 Jun 2017 07:00:00 UTC

Expanding the Cloud  An AWS Region is coming to Hong Kong

Posted By Werner Vogels

Today, I am very excited to announce our plans to open a new AWS Region in Hong Kong! The new region will give Hong Kong-based businesses, government organizations, non-profits, and global companies with customers in Hong Kong, the ability to leverage AWS technologies from data centers in Hong Kong. The new AWS Asia Pacific (Hong Kong) Region will have three Availability Zones and be ready for customers for use in 2018. Over the past decade, we have seen tremendous growth at AWS. As a result, we have opened 43 Availability Zones across 16 AWS Regions worldwide. Last year, we opened new regions in Korea, India, the US, Canada, and the UK.

Wed, 07 Jun 2017 16:00:00 UTC

Unlocking the Value of Device Data with AWS Greengrass.

Posted By Werner Vogels

Unlocking the value of data is a primary goal that AWS helps our customers to pursue. In recent years, an explosion of intelligent devices have created oceans of new data across many industries. We have seen that such devices can benefit greatly from the elastic resources of the cloud. This is because data gets more valuable when it can be processed together with other data. At the same time, it can be valuable to process some data right at the source where it is generated. Some applications  medical equipment, industrial machinery, and building automation are just a few  can't rely exclusively on the cloud for control, and require some form of local storage and execution.

Fri, 19 May 2017 16:00:00 UTC

Weekend Reading: Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases.

Posted By Werner Vogels

In many, high-throughput, OLTP style applications the database plays a crucial role to achieve scale, reliability, high-performance and cost efficiency. For a long time, these requirements were almost exclusively served by commercial, proprietary databases. Soon after the launch of Amazon Relation Dabase Service (RDS) AWS customers were giving us feedback that they would love to migrate to RDS but what they would love even more was if we could also unshackle them from the high-cost, punitive licensing schemes that came with the proprietary databases. They would love to migrate to an open-source style database like MySQL or PostgresSQL if only such a database could meet the enterprise-grade reliability and performance these high-scale applications required.

Wed, 03 May 2017 17:00:00 UTC

Faster, higher, stronger: How the digitalization of industry is redefining value creation

Posted By Werner Vogels

This article titled "Wie die Digitalisierung Wertschöpfung neu definiert" appeared in German last week in the "Größer, höher, weiter (bigger, higher, further)" column of Wirtschaftwoche. Germany's "hidden champions"  family-owned companies, engineering companies, specialists  are unique in the world. They stand for quality, reliability and a high degree of know-how in manufacturing. Hidden champions play a significant role in the German economy; as a result, Germany has become one of the few countries in Western Europe where manufacturing accounts for more than 20% of GDP. By contrast, neighboring countries have seen a continuous decline in their manufacturing base. What's more, digital technologies and business models that are focused on Industry 4.0 (i.e., the term invented in Germany to refer to the digitalization of production) have the potential to reinforce Germany's lead even more.

Wed, 12 Apr 2017 08:00:00 UTC

Coming to STATION F: The first Mentor's Office powered by AWS!

Posted By Werner Vogels

I am excited to announce that AWS is opening its first Mentor's Office at STATION F in Paris! The Mentor's Office is a workplace exclusively dedicated to meetings between AWS experts and the startups. STATION F is the world's biggest startup campus. With this special offer starting at the end of June, at the campus opening, AWS increases the support already available to startup customers in France. All year long, AWS experts will deliver technical and business assistance to startups based on campus. AWS Solutions Architects will meet startup members for face-to-face sessions, to share guidance on how cloud services can be used for their specific use cases, workloads, or applications.

Fri, 07 Apr 2017 16:00:00 UTC

Back-to-Basics Weekend Reading: Twenty years of functional MRI: The science and the stories.

Posted By Werner Vogels

I will be returning this weekend to the US from a very successful AWS Summit in Sydney, so I have ample time to read during travels. This weekend however I would like to take a break from reading historical computer science material, to catch up on another technology I find fascinating, that of functional Magnetic Resonace Imaging, aka fMRI. fMRI is a functional imagine technology, meaning that it just records the state of the brain at one particular point in time, but the changing state over a period of time. The basic technology records brain activity by measuring changes in blood flow through the brain.

Tue, 04 Apr 2017 07:00:00 UTC

Välkommen till Stockholm  An AWS Region is coming to the Nordics

Posted By Werner Vogels

Today, I am very excited to announce our plans to open a new AWS Region in the Nordics! The new region will give Nordic-based businesses, government organisations, non-profits, and global companies with customers in the Nordics, the ability to leverage the AWS technology infrastructure from data centers in Sweden. The new AWS EU (Stockholm) Region will have three Availability Zones and will be ready for customers to use in 2018. Over the past decade, we have seen tremendous growth at AWS. As a result, we have opened 42 Availability Zones across 16 AWS Regions worldwide. Last year, we opened new regions in Canada, India, Korea, the UK, and the US.

Thu, 30 Mar 2017 16:00:00 UTC

Back-to-Basics Weekend Reading: An Implementation of a Log-Structured File System

Posted By Werner Vogels

This weekend I am travelling to Australia for the first AWS Summit of 2017. I find on such a long trip, to keep me from getting distracted, I need an exciting paper that is easy to read. Last week's 'Deep Learning' overview would have not met those requirements. One topic that always gets me excited is how to take computer science research and implement it in production systems. There are often so many obstacles that we do not see much of this work happening. For example when building Dynamo, where we put a collection of different research technologies together in production, we struggled with all the assumptions the researchers had made.

Sat, 25 Mar 2017 04:00:00 UTC

Back-to-Basics Weekend Reading: Deep learning in neural networks

Posted By Werner Vogels

In the past few years, we have seen an explosion in the use of Deep Learning as its software platforms mature and the supporting hardware, especially GPUs with larger memories, become widely available. Even though this is a recent development, Deep Learning has deep historical roots, tracing back all the way to the sixties, or maybe even earlier. By reading up on its history, we get a better understanding of the current state of the art of Deep Learning algorithms and the Neural Networks that you build with them. There is such a broad set of papers to read if we would want to dive deep into the history, that it would take us multiple weekends.

Wed, 15 Mar 2017 17:00:00 UTC

Amazon Makes it Free for Developers to Build and Host Most Alexa Skills Using AWS

Posted By Werner Vogels

Amazon today announced a new program that will make it free for tens of thousands of Alexa developers to build and host most Alexa skills using Amazon Web Services (AWS). Many Alexa skill developers currently take advantage of the AWS Free Tier, which offers one million AWS Lambda requests and up to 750 hours of Amazon Elastic Compute Cloud (Amazon EC2) compute time per month at no charge. However, if developers exceed the AWS Free Tier limits, they may incur AWS usage fees each month. Now, developers with a live Alexa skill can apply to receive a $100 AWS promotional credit and can earn an additional $100 per month in AWS promotional credits if they incur AWS usage charges for their skill  making it free for developers to build and host most Alexa skills.

Mon, 13 Mar 2017 17:00:00 UTC

How companies can become magnets for digital talent

Posted By Werner Vogels

This article titled "Wie Unternehmen digitale Talente anziehen" appeared in German last week in the "Tipps für Arbeitgeber" section of Wirtschaftwoche. The rise in digital business models is a huge challenge for recruiting and talent selection. The sort of skills businesses need today are in short supply. How companies can prepare themselves to attract the best talents for shaping their digital business. Digitalization offers almost endless possibilities to communicate faster, work more efficiently, and be more creative  in real-time. But groundbreaking digital business models need pioneers: creators, forward-looking thinkers and inventors who don't hesitate to leave the beaten path, embody ownership, and who understand how to translate customers' wishes into superb new products, services and solutions that evolve with speed.

Sat, 11 Mar 2017 16:00:00 UTC

Back-to-Basic Weekend Reading: The Foundations of Blockchain

Posted By Werner Vogels

More and more we see stories appearing, like this one in HBR by MIT Media Lab's Joi Ito and crew. It praises the power of blockchain as a disruptive technology, on par with how "the internet" changed everything. I am always surprised to see that these far-reaching predictions are made, without diving into the technology itself. This weekend I would like to read about some of the technologies that predate blockchain, as they are its fundamental building blocks. Blockchain technology first came on the scene in 2008, as a core component of the bitcoin cryptocurrency. Blockchain provides transactional, distributed ledger functionality that can operate without a centralized, trusted authority.

Sat, 04 Mar 2017 17:00:00 UTC

Back-to-Basic Weekend Reading: Why Do Computers Stop and What Can Be Done About It?

Posted By Werner Vogels

"Everything fails, all the time." A humble computer scientist once said. With all the resources we have today, it is easier for us to achieve fault-tolerance than it was many decades ago when computers began playing a role in critical systems such as health care, air traffic control and financial market systems. In the early days, the thinking was to use a hardware approach to achieve fault-tolerance. It was not until the mid-nineties that software fault-tolerance became more acceptable. Tandem Computer was one of the pioneers in building these fault-tolerant, mission-critical systems. They used a shared-nothing multi-cpu approach. This is where each CPU had its own memory- and io-bus, and all were connected through a replicated shared bus, over which the independent OS instances could communication and run in lock step.

Sat, 25 Feb 2017 04:00:00 UTC

Back-to-Basic Weekend Reading: Byzantine Generals

Posted By Werner Vogels

In Reliable Distribution Systems, we need to handle different failure scenarios. Many of those deal with message loss and process failure. However, there is a class of scenarios that deal with malfunctioning processes, which send out conflicting information. The challenge is developing algorithms that can reach an agreement in the presence of these failures. Lamport described that he was frustrated with the attention that Dijkstra had gotten for describing a computer science problem as the story of dining philosophers. He decided the best way to attract attention to a particular distributed systems problem was to present it in terms of a story; hence, the Byzantine Generals.

Fri, 10 Feb 2017 19:00:00 UTC

Back-to-Basic Weekend Reading: Monte-Carlo Methods

Posted By Werner Vogels

I always enjoy looking for solutions to difficult challenges in non-obvious places. That is probably why I like using probabilistic techniques for problems that appear to be hard, or impossible to solve deterministically. The probabilistic approach may not result in the perfect result, but it may get you very close, and much faster than deterministic techniques (which may even be computationally impossible). Some of the earliest approaches using probabilities in physics experiments resulted in the Monte Carlo methods. Their essential idea is using randomness to solve problems that might be deterministic in principle. These are a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results.

Fri, 03 Feb 2017 19:00:00 UTC

Back-to-Basics Weekend Reading - Bloom Filters

Posted By Werner Vogels

Listening to the "Algorithms to Live By" audio on my commute this morning I once again was struck by the beauty of Bloom Filters. So I think it is time to resurrect the Back-to-Basics Weekend Reading series, as I will be re-reading some really fundamental CS papers this weekend. In the past I have done some weekend reading about Counting Bloom Filters, but now I am going even more fundamental. Bloom Filters, conceived by Burton Bloom in 1970, are probabilistic data structures to test whether an item is in a set. False positives are possible but false negatives are not. So if a bit in the filter is not set you can be sure the item is not in the set, if it is set, the mapped item may be in the set.

Sun, 29 Jan 2017 17:00:00 UTC

A survival strategy for the digital transformation

Posted By Werner Vogels

This article titled "Überlebensstrategie für die digitale Transformation" appeared in German last week in the "Die Zukunft beginnt heute (the future starts today)" section of Wirtschaftwoche. Smaller companies have a lot to gain in the digital era  provided they adopt the right mindset. The winners will be those that view their business from the eyes of their customers and understand that fast-paced innovation is the key to long-term growth. With this mindset they can take on even the largest enterprises who are slow to adapt to the fast moving digital reality. The digital era is here. Companies that haven't realized that by now will fall behind.

Wed, 14 Dec 2016 08:00:00 UTC

Expanding the AWS Cloud: Introducing the AWS Europe (London) Region

Posted By Werner Vogels

In November 2015, Amazon Web Services announced that it would launch a new AWS infrastructure region in the United Kingdom. Today, I'm happy to announce that the AWS Europe (London) Region, our 16th technology infrastructure region globally, is now generally available for use by customers worldwide. UK companies are using AWS to innovate across diverse industries, such as energy, manufacturing, medicaments, retail, media, and financial services and the UK is home to some of the world's most forward-thinking businesses. These include startups like Fanduel, JustEat, and Monzo to enterprises such as British Gas, Trainline, Perkins, News UK, the Financial Times. The British Government is also helping to drive innovation and has embraced a cloud-first policy for technology adoption.

Thu, 08 Dec 2016 18:30:00 UTC

Expanding the AWS Cloud: Introducing the AWS Canada (Central) Region

Posted By Werner Vogels

Earlier this year, Amazon Web Services (AWS) announced it would launch a new AWS infrastructure region in Montreal, Quebec. Today, I'm happy to share that the Canada (Central) Region is available for use by customers worldwide. The AWS Cloud now operates in 40 Availability Zones within 15 geographic regions around the world, with seven more Availability Zones and three more regions coming online in China, France, and the U.K. in the coming year. The Canadian opportunity Canada has set forth a bold innovation agenda grounded in entrepreneurship, scientific research, growing small and medium-sized businesses with a focus on environmentally friendly technologies, and the transition to a digital economy.

Thu, 01 Dec 2016 20:00:00 UTC

Transforming Development with AWS

Posted By Werner Vogels

In my keynote at AWS re:Invent today, I announced 13 new features and services (in addition to the 15 we announced yesterday). My favorite parts of James Bond movies is are where 007 gets to visit Q to pick up and learn about new tools of the trade: super-powered tools with special features which that he can use to complete his missions, and, in some cases, get out of some nasty scrapes. Bond always seems to have the perfect tool for every situation that he finds himself in. * At AWS, we want to be the Q for developers, giving them the super-powered tools and services with deep features in the Cloud.

Wed, 30 Nov 2016 18:00:00 UTC

Bringing the Magic of Amazon AI and Alexa to Apps on AWS.

Posted By Werner Vogels

From the early days of Amazon, Machine learning (ML) has played a critical role in the value we bring to our customers. Around 20 years ago, we used machine learning in our recommendation engine to generate personalized recommendations for our customers. Today, there are thousands of machine learning scientists and developers applying machine learning in various places, from recommendations to fraud detection, from inventory levels to book classification to abusive review detection. There are many more application areas where we use ML extensively: search, autonomous drones, robotics in fulfillment centers, text processing and speech recognition (such as in Alexa) etc. Among machine learning algorithms, a class of algorithms called deep learning has come to represent those algorithms that can absorb huge volumes of data and learn elegant and useful patterns within that data: faces inside photos, the meaning of a text, or the intent of a spoken word.After over 20 ...

Tue, 22 Nov 2016 17:00:00 UTC

MXNet - Deep Learning Framework of Choice at AWS

Posted By Werner Vogels

Machine learning is playing an increasingly important role in many areas of our businesses and our lives and is being employed in a range of computing tasks where programming explicit algorithms is infeasible. At Amazon, machine learning has been key to many of our business processes, from recommendations to fraud detection, from inventory levels to book classification to abusive review detection. And there are many more application areas where we use machine learning extensively: search, autonomous drones, robotics in fulfillment centers, text and speech recognitions, etc. Among machine learning algorithms, a class of algorithms called deep learning hascome to represent those algorithms that can absorb huge volumes of data and learn elegant and useful patterns within that data: faces inside photos, the meaning of a text, or the intent of a spoken word.

Tue, 15 Nov 2016 22:00:00 UTC

Spice up your Analytics: Amazon QuickSight Now Generally Available in N. Virginia, Oregon, and Ireland.

Posted By Werner Vogels

Previously, I wrote about Amazon QuickSight, a new service targeted at business users that aims to simplify the process of deriving insights from a wide variety of data sources quickly, easily, and at a low cost. QuickSight is a very fast, cloud-powered, business intelligence service for the 1/10th the cost of old-guard BI solutions. Today, I am very happy to announce that QuickSight is now generally available in the N. Virginia, Oregon, and Ireland regions. When we announced QuickSight last year, we set out to help all customersregardless of their technical skillsmake sense out of their ever-growing data. As I mentioned, we live in a world where massive volumes of data are being generated, every day, from connected devices, websites, mobile apps, and customer applications running on top of AWS infrastructure.

Mon, 14 Nov 2016 17:00:00 UTC

Meet the Teams Competing for the Alexa Prize

Posted By Werner Vogels

On September 29, 2016, Amazon announced the Alexa Prize, a $2.5 million university competition to advance conversational AI through voice. We received applications from leading universities across 22 countries. Each application was carefully reviewed by senior Amazon personnel against a rigorous set of criteria covering scientific contribution, technical merit, novelty, and ability to execute. Teams of scientists, engineers, user experience designers, and product managers read, evaluated, discussed, argued, and finally selected the twelve teams who would be invited to participate in the competition. Today, were excited to announce the 12 teams selected to compete with an Amazon sponsorship. In alphabetical order, they are: Carnegie-Mellon University: CMU Magnus Carnegie-Mellon University: TBD Czech Technical University, Prague: eClub Prague Heriot-Watt University, UK: WattSocialBot Princeton University: Princeton Alexa Rensselaer Polytechnic Institute: BAKAbot University of California, Berkeley: Machine Learning @ Berkeley University of California, Santa Cruz: SlugBots University of Edinburgh, UK: Edina University of ...

Mon, 24 Oct 2016 16:00:00 UTC

Welcoming Adrian Cockcroft to the AWS Team.

Posted By Werner Vogels

I am excited that Adrian Cockcroft will be joining AWS as VP of Cloud Architecture. Adrian has played a crucial role in developing the cloud ecosystem as Cloud Architect at Netflix and later as a Technology Fellow at Battery Ventures. Prior to this, he held positions as Distinguished Engineer at eBay and Sun Microsystems. One theme that has been consistent throughout his career is that Adrian has a gift for seeing the bigger engineering picture. At Netflix, Adrian played a key role in the company's much-discussed migration to a "cloud native" architecture, and the open sourcing of the widely used (and award-winning) NetflixOSS platform.

Mon, 17 Oct 2016 16:00:00 UTC

Expanding the AWS Cloud: Introducing the AWS US East (Ohio) Region

Posted By Werner Vogels

Today I am very happy to announce the opening of the new US East (Ohio) Region. The Ohio Region is the fifth AWS region in the US. It brings the worldwide total of AWS Availability Zones (AZs) to 38, and the number of regions globally to 14. The pace of expansion at AWS is accelerating, and Ohio is our third region launch this year. In the remainder of 2016 and inq 2017, we will launch another four AWS regions in Canada, China, the United Kingdom, and France, adding another nine AZs to our global infrastructure footprint. We strive to place customer feedback first in our considerations for where to open new regions.

Thu, 13 Oct 2016 05:00:00 UTC

Accelerating Data: Faster and More Scalable ElastiCache for Redis

Posted By Werner Vogels

Fast Data is an emerging industry term for information that is arriving at high volume and incredible rates, faster than traditional databases can manage. Three years ago, as part of our AWS Fast Data journey we introduced Amazon ElastiCache for Redis, a fully managed in-memory data store that operates at sub-millisecond latency. Since then weve introduced Amazon Kinesis for real-time streaming data, AWS Lambda for serverless processing, Apache Spark analytics on EMR, and Amazon QuickSight for high performance Business Intelligence. While caching continues to be a dominant use of ElastiCache for Redis, we see customers increasingly use it as an in-memory NoSQL database.

Thu, 29 Sep 2016 17:00:00 UTC

Introducing the Alexa Prize, Its Day One for Voice

Posted By Werner Vogels

In the past voice interfaces were seen as gimmicks, or a nuisance for driving hands-free. The Amazon Echo and Alexa have completely changed that perception. Voice is now seen as potentially the most important interface to interact with the digitally connected world. From home automation to commerce, from news organizations to government agencies, from financial services to healthcare, everyone is working on the best way is to interact with their services if voice is the interface. Especially for the exciting case where voice is the only interface. Voice makes access to digital services far more inclusive than traditional screen-based interaction, for example, an aging population may be much more comfortable interacting with voice-based systems than through tablets or keyboards.

Thu, 29 Sep 2016 07:00:00 UTC

Allez, rendez-vous à Paris  An AWS Region is coming to France!

Posted By Werner Vogels

Today, I am very excited to announce our plans to open a new AWS Region in France! Based in the Paris area, the region will provide even lower latency and will allow users who want to store their content in datacenters in France to easily do so. The new region in France will be ready for customers to use in 2017. Over the past 10 years, we have seen tremendous growth at AWS. As a result, we have opened 35 Availability Zones (AZs), across 13 AWS Regions worldwide. We have announced several additional regions in Canada, China, Ohio, and the United Kingdom  all expected in the coming months.

Tue, 27 Sep 2016 23:00:00 UTC

A Hungry Neighbor is an Angry Neighbor

Posted By Werner Vogels

I am very grateful that I have had the opportunity to meet with President Shimon Peres several times. Especially the first time, which was a 1:1 in his presidential residence, was an unforgettable experience. After I explained in 5 minutes the power of cloud for unlocking digital business building for everyone, he went on a lecture of half an hour how bringing economic prosperity to the region was crucial to achieving a long lasting peace. " A hungry neighbor is an angry neighbor". He strongly believed peace in the Middle East was attainable, and I have no reason to doubt him.

Mon, 27 Jun 2016 12:00:00 UTC

New Ways to Discover and Use Alexa Skills

Posted By Werner Vogels

Introducing New Features That Make It Easier for Customers to Discover and Use Your Alexa Skills Alexa, Amazons cloud-based voice service, powers voice experiences on millions of devices, including Amazon Echo and Echo Dot, Amazon Tap, Amazon Fire TV devices, and devices like Triby that use the Alexa Voice Service. One year ago, Amazon opened up Alexa to developers, enabling you to build Alexa skills with the Alexa Skills Kit and integrate Alexa into your own products with the Alexa Voice Service. Today, tens of thousands of developers are building skills for Alexa, and there are over 1,400 skills for Alexa  including Lyft and Honeywell, which were added today.

Mon, 27 Jun 2016 05:00:00 UTC

Expanding the Cloud: Introducing the AWS Asia Pacific (Mumbai) Region

Posted By Werner Vogels

In June 2015, Amazon Web Services announced that it would launch a new AWS infrastructure region in India. Today, Im happy to announce that the Asia Pacific (Mumbai) Region is generally available for use by customers worldwide. The opportunity to revolutionize A region in India has been highly sought after by companies around the world who want to participate in one of the most significant economic opportunities in the world  India, a rising economy that holds tremendous promise for growth, a thriving technology hub with a rich eco-system of technology talent, and more. Rapid economic growth in India is creating several business opportunities such as distributed ledger technology with blockchains that could drive efficiencies in the real estate market, Fin-Tech innovations such as P2P mobile apps that have the power to change the social economic lives of people through financial inclusion, applying the sharing economy from cabs to other ...

Fri, 10 Jun 2016 14:00:00 UTC

Serverless Reference Architectures with AWS Lambda

Posted By Werner Vogels

Building your applications with only managed components has become very popular, and AWS Lambda plays a crucial role in that. I see a tremendous interest in examples how to build such applications, and articles such as "The Serverless Start-Up - Down With Servers!" about are read eagerly around the globe. If you are looking for more examples there are the Lambda Serverless Reference Architectures that can serve as the blueprint for building your own serverless applications. Mobile Backend Serverless Reference Architecture The Mobile Backend reference architecture demonstrates how to use AWS Lambda along with other services to build a serverless backend for a mobile application.

Tue, 10 May 2016 14:00:00 UTC

Serverless Reference Architecture with AWS Lambda

Posted By Werner Vogels

Building your applications with only managed components has become very popular, and AWS Lambda plays a crucial role in that. I see a tremendous interest in examples how to build such applications, and articles such as "The Serverless Start-Up - Down With Servers!" about are read eagerly around the globe. If you are looking for more examples there are the Lambda Serverless Reference Architectures that can serve as the blueprint for building your own serverless applications. Mobile Backend Serverless Reference Architecture The Mobile Backend reference architecture demonstrates how to use AWS Lambda along with other services to build a serverless backend for a mobile application.

Fri, 11 Mar 2016 14:00:00 UTC

10 Lessons from 10 Years of Amazon Web Services

Posted By Werner Vogels

The epoch of AWS is the launch of Amazon S3 on March 14, 2006, now almost 10 years ago. Looking back over the past 10 years, there are hundreds of lessons that weve learned about building and operating services that need to be secure, reliable, scalable, with predictable performance at the lowest possible cost. Given that AWS is a pioneer in building and operating these services world-wide, these lessons have been of crucial importance to our business. As weve said many times before, There is no compression algorithm for experience. With over a million active customers per month, who in turn may serve hundreds of millions of their own customers, there is no lack of opportunities to gain more experience and perhaps no better environment for continuous improvement in the way we serve our customers.

Wed, 06 Jan 2016 23:00:00 UTC

Expanding the Cloud: Introducing the AWS Asia Pacific (Seoul) Region

Posted By Werner Vogels

In November, Amazon Web Services announced that it would launch a new AWS infrastructure region in South Korea. Today, Im happy to announce that the Asia Pacific (Seoul) Region is now generally available for use by customers worldwide. A region in South Korea has been highly requested by companies around the world who want to take full advantage of Koreas world-leading Internet connectivity and provide their customers with quick, low-latency access to websites, mobile applications, games, SaaS applications, and more. Weve also been hearing many requests from Korean companies, including large enterprises like Samsung and Mirae Asset. For example, Samsung Electronic Printing used AWS to deploy its Printing Apps Center in a way that didnt require them to invest up-front capital and kept total costs quite low.

Fri, 06 Nov 2015 04:00:00 UTC

London Calling! An AWS Region is coming to the UK!

Posted By Werner Vogels

Yesterday, AWS evangelist Jeff Barr wrote that AWS will be opening a region in South Korea in early 2016 that will be our 5th region in Asia Pacific. Customers can choose between 11 regions around the world today and, in addition to Korea, we are adding regions in India, a second region in China, and Ohio in 2016. Today, I am excited to add the United Kingdom to that list! The AWS UK region will be our third in the European Union (EU), and we're shooting to have it ready by the end of 2016 (or early 2017). This region will provide even lower latency and strong data sovereignty to local users.

Wed, 07 Oct 2015 13:00:00 UTC

Expanding the Cloud: Introducing Amazon QuickSight

Posted By Werner Vogels

We live in a world where massive volumes of data are generated from websites, connected devices and mobile apps. In such a data intensive environment, making key business decisions such as running marketing and sales campaigns, logistic planning, financial analysis and ad targeting require deriving insights from these data. However, the data infrastructure to collect, store and process data is geared toward developers (e.g., Amazon Redshift, DynamoDB, Amazon EMR) whereas insights need to be derived by not just developers but also non-technical business users. In AWS quest to enable the best data storage options for engineers, we have built several innovative database solutions like Amazon RDS, Amazon RDS for Aurora, Amazon DynamoDB, and Amazon Redshift.

Mon, 28 Sep 2015 13:00:00 UTC

The Startup Experience at AWS re:Invent

Posted By Werner Vogels

AWS re:Invent is just over one week awayas I prepare to head to Vegas, Im pumped up about the chance to interact with AWS-powered startups from around the world. One of my favorite parts of the week is being able to host three startup-focused sessions Thursday afternoon: The Startup Scene in 2016: a Visionary Panel [Thursday, 2:45PM] In this session, Ill moderate a diverse panel of technology experts wholl discuss emerging trends all startups should be aware of, including how local governments, microeconomic trends, evolving accelerator programs, and the AWS cloud are influencing the global startup scene. This panel will include: Tracy DiNunzio, Founder & CEO, Tradesy Michael DeAngelo, Deputy CIO, State of Washington Ben Whaley, Founder & Principal Consultant, WhaleTech LLC Jason Seats, Managing Director (Austin), & Partner, Techstars CTO-to-CTO Fireside Chat [Thursday, 4:15 PM] This is one of my favorite sessions as I get a chance ...

Tue, 08 Sep 2015 13:00:00 UTC

The AWS Pop-up Lofts are opening in London and Berlin

Posted By Werner Vogels

Amazon Web Services (AWS) has been working closely with the startup community in London, and Europe, since we launched back in 2006. We have grown substantially in that time and today more than two thirds of the UKs startups with valuations of over a billion dollars, including Skyscanner, JustEat, Powa, Fanduel and Shazam, are all leveraging our platform to deliver innovative services to customers around the world. This week I will have the pleasure of meeting up with our startup customers to we celebrate the opening of the first of the AWS Pop-up Lofts to open outside of the US in one of the greatest cities in the World, London.

Thu, 20 Aug 2015 13:00:00 UTC

Titan Graph Database Integration with DynamoDB: World-class Performance, Availability, and Scale for New Workloads

Posted By Werner Vogels

Today, we are releasing a plugin that allows customers to use the Titan graph engine with Amazon DynamoDB as the backend storage layer. It opens up the possibility to enjoy the value that graph databases bring to relationship-centric use cases, without worrying about managing the underlying storage. The importance of relationships Relationships are a fundamental aspect of both the physical and virtual worlds. Modern applications need to quickly navigate connections in the physical world of people, cities, and public transit stations as well as the virtual world of search terms, social posts, and genetic code, for example. Developers need efficient methods to store, traverse, and query these relationships.

Mon, 20 Jul 2015 13:00:00 UTC

Under the Hood of Amazon EC2 Container Service

Posted By Werner Vogels

In my last post about Amazon EC2 Container Service (Amazon ECS), I discussed the two key components of running modern distributed applications on a cluster: reliable state management and flexible scheduling. Amazon ECS makes building and running containerized applications simple, but how that happens is what makes Amazon ECS interesting. Today, I want to explore the Amazon ECS architecture and what this architecture enables. Below is a diagram of the basic components of Amazon ECS: How we coordinate the cluster Lets talk about what Amazon ECS is actually doing. The core of Amazon ECS is the cluster manager, a backend service that handles the tasks of cluster coordination and state management.

Fri, 17 Jul 2015 17:00:00 UTC

Back-to-Basics Weekend Reading - Data Compression

Posted By Werner Vogels

Data compression today is still as important as it was in the early days of computing. Although in those days all computer and storage resources were very limited, the objects in use were much smaller than today. We have seen a shift from generic compression to compression for specific file types, especially those in images, audio and video. In this weekend's back to basic reading we go back in time, 1987 to be specific, when Leweler and Hirschberg wrote a survey paper that covers the 40 years of data compression research. It covers all the areas that we like in a back to basics paper, it does not present the most modern results but it gives you a great understanding of the fundamentals.

Tue, 14 Jul 2015 17:00:00 UTC

Embrace event-driven computing: Amazon expands DynamoDB with streams, cross-region replication, and database triggers

Posted By Werner Vogels

In just three short years, Amazon DynamoDB has emerged as the backbone for many powerful Internet applications such as AdRoll, Druva, DeviceScape, and Battlecamp. Many happy developers are using DynamoDB to handle trillions of requests every day. I am excited to share with you that today we are expanding DynamoDB with streams, cross-region replication, and database triggers. In this blog post, I will explain how these three new capabilities empower you to build applications with distributed systems architecture and create responsive, reliable, and high-performance applications using DynamoDB that work at any scale. DynamoDB Streams enables your application to get real-time notifications of your tables item-level changes.

Thu, 25 Jun 2015 17:00:00 UTC

Amazon announces the Alexa Skills Kit, Enabling Developers to Create New Voice Capabilities

Posted By Werner Vogels

Today, Amazon announced the Alexa Skills Kit (ASK), a collection of self-service APIs and tools that make it fast and easy for developers to create new voice-driven capabilities for Alexa. With a few lines of code, developers can easily integrate existing web services with Alexa or, in just a few hours, they can build entirely new experiences designed around voice. No experience with speech recognition or natural language understanding is requiredAmazon does all the work to hear, understand, and process the customers spoken request so you dont have to. All of the code runs in the cloud  nothing is installed on any user device.

Fri, 12 Jun 2015 17:00:00 UTC

Back-to-Basics Weekend Reading - The Working Set Model for Program Behavior

Posted By Werner Vogels

This weekend we go back in time all the way to the beginning of operating systems research. In the first SOSP conference in 1967 there were several papers that laid the foundation for the development of structured operating systems. There was the of course the lauded paper on the THE operating system by Dijkstra but for this weekend I picked the paper on memory locality by Peter Denning as this work laid the groundwork for the development of virtual memory systems. The Working Set Model for Program Behavior, Peter J.

Fri, 29 May 2015 17:00:00 UTC

Back-to-Basics Weekend Reading - Survey of Local Algorithms

Posted By Werner Vogels

As we know the run time of most algorithms increases when the input set increases in size. There is one noticeable exception: there is a class of distributed algorithms, dubbed local algorithms, that run in constant time, independently of the size of the network. Being highly scalable and fault tolerant, such algorithms are ideal in the operation of large-scale distributed systems. Furthermore, even though the model of local algorithms is very limited, in recent years we have seen many positive results for nontrivial problems. In this weekend's paper Jukka Suomela surveys the state-of-the-art in the field, covering impossibility results, deterministic local algorithms, randomized local algorithms, and local algorithms for geometric graphs.

Thu, 28 May 2015 23:30:00 UTC

Join me at the AWS Summit in Paris, Tel Aviv, Berlin, Amsterdam or New York

Posted By Werner Vogels

An important way of engaging with AWS customers is through the AWS Global Summit Series. All AWS Summits feature a keynote address highlighting the latest announcements from AWS and customer testimonials, technical sessions led by AWS engineers, and hands-on technical training. You will learn best practices for deploying applications on AWS, optimizing performance, monitoring cloud resources, managing security, cutting costs, and more. You will also have opportunities to meet AWS staff and partners to get your technical questions answered. At the Summit we focus on education and helping our customers, there are deep technical developer sessions, broad sessions on architectural principles, sessions for enterprise decision makers and how to best exploit AWS in a public sector or education settings.

Wed, 27 May 2015 20:30:00 UTC

The AWS Pop-up Loft opens in New York City

Posted By Werner Vogels

Over a year ago the AWS team opened a "pop-up loft" in San Francisco at 925 Market Street. The goal of opening the loft was to give developers an opportunity to get in-person support and education on AWS, to network, get some work done, or just hang out with peers. It became a great success; every time when I visit the loft there is a great buzz with people getting advice from our solution architects, getting training or attending talks and demos. It became such a hit among developers that we decided to reopen the loft last year August after its initial run of 4 weeks, making sure everyone would have continued access to this important resource.

Thu, 09 Apr 2015 18:00:00 UTC

Expanding the Cloud: Amazon Machine Learning Service, the Amazon Elastic Filesystem and more

Posted By Werner Vogels

Today was a big day for the Amazon Web Services teams as a whole range of new services and functionality was delivered to our customers. Here is a brief recap of it: The Amazon Machine Learning service As I wrote last week machine learning is becoming an increasingly important tool to build advanced data driven applications. At Amazon we have hundreds of teams using machine learning and by making use of the Machine Learning Service we can significantly speed up the time they use to bring their technologies into production. And you no longer need to be a machine learning expert to be able to use it.

Thu, 09 Apr 2015 17:00:00 UTC

State Management and Scheduling with the Amazon EC2 Container Service

Posted By Werner Vogels

Last November, I had the pleasure of announcing the preview of Amazon EC2 Container Service (ECS) at re:Invent. At the time, I wrote about how containerization makes it easier for customers to decompose their applications into smaller building blocks resulting in increased agility and speed of feature releases. I also talked about some of the challenges our customers were facing as they tried to scale container-based applications including challenges around cluster management. Today, I want to dive deeper into some key design decisions we made while building Amazon ECS to address the core problems our customers are facing. Running modern distributed applications on a cluster requires two key components - reliable state management and flexible scheduling.

Fri, 03 Apr 2015 17:00:00 UTC

Back-to-Basics Weekend Reading - Machine Learning

Posted By Werner Vogels

Machine learning is a scientific discipline that explores the construction and study of algorithms that can learn from data. Such algorithms operate by building a model from example inputs and using that to make predictions or decisions, rather than following strictly static program instructions. Machine Learning is playing an increasing important role in many areas of our businesses and our lives. ML is used for predictive analytics and predictive modeling, e.g. making predictions about the likelihood that a certain event is going to happen (will this customer be interested in this item, is this message spam). At Amazon machine learning has been key to many of our business processes, from recommendations to fraud detection, from inventory levels to book classification to abusive review detection.

Tue, 31 Mar 2015 17:00:00 UTC

European Union Data Protection Authorities Approve Amazon Web Services Data Processing Agreement

Posted By Werner Vogels

As you all know security, privacy, and protection of our customers data is our number one priority and as such we work very closely with regulators to ensure that customers can be assured that they are getting the right protections when processing and storing data in the AWS. I am especially pleased that the group of European Union (EU) data protection authorities known as the Article 29 Working Party has approved the AWS Data Processing Agreement (DPA), assuring customers that it meets the high standards of EU data protection laws. The media alert below that went out today gives the details: European Union Data Protection Authorities Approve Amazon Web Services Data Processing Agreement Customers All Over the World Are Assured that AWS Agreement Meets Rigorous EU Privacy Laws Brussels  March 31, 2015  Amazon Web Services (AWS) today announced that the group of European Union (EU) data protection authorities ...

Tue, 31 Mar 2015 17:00:00 UTC

European Union Data Protection Authorities Approve Amazon Web Services Data Processing Agreement

Posted By Werner Vogels

As you all know security, privacy, and protection of our customers data is our number one priority and as such we work very closely with regulators to ensure that customers can be assured that they are getting the right protections when processing and storing data in the AWS. I am especially pleased that the group of European Union (EU) data protection authorities known as the Article 29 Working Party has approved the AWS Data Processing Agreement (DPA), assuring customers that it meets the high standards of EU data protection laws. The media alert below that went out today gives the details: European Union Data Protection Authorities Approve Amazon Web Services Data Processing Agreement Customers All Over the World Are Assured that AWS Agreement Meets Rigorous EU Privacy Laws Brussels  March 31, 2015  Amazon Web Services (AWS) today announced that the group of European Union (EU) data protection authorities ...

Wed, 11 Mar 2015 17:00:00 UTC

Observations on the Importance of Cloud-based Analytics

Posted By Werner Vogels

Cloud computing is enabling amazing new innovations both in consumer and enterprise products, as it became the new normal for organizations of all sizes. So many exciting new areas are being empowered by cloud that it is fascinating to watch. AWS is enabling innovations in areas such as healthcare, automotive, life sciences, retail, media, energy, robotics that it is mind boggling and humbling. Despite all of the amazing innovations we have already seen, we are still on Day One in the Cloud; at AWS we will continue to use our inventive powers to build new tools and services to enable even more exciting innovations by our customers that will touch every area of our lives.

Tue, 10 Mar 2015 23:30:00 UTC

Join me at SXSW 2015

Posted By Werner Vogels

Every year I enjoy travelling to the South-by-South-West (SXSW) festival as it is ons of the biggest event with many Amazon customers present. Thousand of AWS customers and partners will be in Austin for SXSW Interactive and given the free flowing networking it is a very important feedback opportunity for us. But also many Amazon customers will be there for the Film and the Music festival, and I always enjoy getting feedback from those Amazon consumers and producers that are attending these festivals. The program is always a bit in flux, but here are the events in the beginning of the week that I am taking part in: Sunday 3/15 1-2pm - I will give a talk at Techstars on "The History of Microcroservices at Amazon".

Fri, 06 Mar 2015 23:30:00 UTC

Back-to-Basics Weekend Reading - Experience with Grapevine: The Growth of a Distributed System

Posted By Werner Vogels

Grapevine was one of the first systems designed to be fully distributed. It was built at the famous Xerox PARC (Palo Alto Research Center) Computer Science Laboratory as an exercise in discovering what is needed as the fundamental building blocks of a distributed system; messaging, naming, discovery, location, routing, authentication, encryption, replication, etc. The origins of the system are described in Grapevine: An Exercise in Distributed Computing by researchers who all went on to become grandmasters in distributed computing: Andrew Birrell, Roy Levin, Roger Needham, and Mike Schroeder. For this weekend's reading we will use a followup paper that focusses on the learnings with running Grapevine for several years under substantial load.

Fri, 27 Feb 2015 23:30:00 UTC

Back-to-Basics Weekend Reading - Distributed Snapshots: Determining Global States of a Distributed System

Posted By Werner Vogels

Several problems in Distributed Systems can be seen as the challenge to determine a global state. In the classical "Time, Clocks and the Ordering of Events in a Distributed System" Lamport had laid out the principles and mechanisms to solve such problems, and the Distributed Snapshots algorithm, popularly know as the Chandy-Lamport algorithm, is an application of that work. The fundamental techniques in the Distributed Snapshot paper are the secret sauce in many distributed algorithms for deadlock detection, termination detection, consistent checkpointing for fault tolerance, global predicate detection for debugging and monitoring, and distributed simulation. An interesting anecdote about the algorithm is told by Lamport: "The distributed snapshot algorithm described here came about when I visited Chandy, who was then at the University of Texas in Austin.

Fri, 20 Feb 2015 23:30:00 UTC

Back-to-Basics Weekend Reading - RAID: High-Performance, Reliable Secondary Storage

Posted By Werner Vogels

Disk arrays, which organize multiple, independent disks into a large, high-performance logical disk, were a natural solution to dealing with constraints on performance and reliability of single disk drives. The term "RAID" was invented by David Patterson, Garth A. Gibson, and Randy Katz at the University of California, Berkeley in 1987. In their June 1988 paper "A Case for Redundant Arrays of Inexpensive Disks (RAID)" they argued that the top performing mainframe disk drives of the time could be beaten on performance by an array of the inexpensive drives that had been developed for the growing personal computer market. Although failures would rise in proportion to the number of drives, by configuring for redundancy, the reliability of an array could far exceed that of any large single drive.

Wed, 18 Feb 2015 17:30:00 UTC

AWS Certification for DevOps Engineers

Posted By Werner Vogels

One of our guiding principles at AWS is to listen closely to our customers and the feedback that I am getting about our training and certification program is very positive. Many architects and engineers know the Cloud is the future of development and IT and the are gearing up to be as succesful as possible in this new normal. This is why Im excited to announce the availability of a new Professional level certification from AWS that has been high on the list of our customers. With the growing adoption of cloud computing, we see more of our customers establishing DevOps practices within their IT organizations as a way to increase IT efficiency, improve agility, and in turn innovate faster for their own customers.

Fri, 13 Feb 2015 17:30:00 UTC

Back-to-Basics Weekend Reading - Exploring Complex Networks

Posted By Werner Vogels

After a year of absence I am bringing back the Back to Basic Weekend Reading Series. We'll continue to look at the fundamental works of Computer Science and Engineering, and other interesting technical works. We will start this year with a topic that spans many sciences: that of complex networks. It is relevant to everything from biology, life sciences, social sciences to computer engineering. There is no one better suitable to teach us about the fundamentals of complex networks than Steven Strogatz, the well known author and applied mathematics professor from Cornell University. You can find much of Steven's work on his personal website.

Thu, 13 Nov 2014 17:30:00 UTC

The Easiest Way to Compute in the Cloud  AWS Lambda

Posted By Werner Vogels

When AWS launched, it changed how developers thought about IT services: What used to take weeks or months of purchasing and provisioning turned into minutes with Amazon EC2. Capital-intensive storage solutions became as simple as PUTting and GETting objects in Amazon S3. At AWS we innovate by listening to and learning from our customers, and one of the things we hear from them is that they want it to be even simpler to run code in the cloud and to connect services together easily. Customers want to focus on their unique application logic and business needs  not on the undifferentiated heavy lifting of provisioning and scaling servers, keeping software stacks patched and up to date, handling fleet-wide deployments, or dealing with routine monitoring, logging, and web service front ends.

Thu, 13 Nov 2014 17:00:00 UTC

Expanding The Cloud - Introducing The Amazon EC2 Container Service

Posted By Werner Vogels

Today, I am excited to announce the Preview of the Amazon EC2 Container Service, a highly scalable, high performance container management service. We created EC2 Container Service to help customers run and manage Dockerized distributed applications. Benefits of Containers Customers have been using Linux containers for quite some time on AWS and have increasingly adopted microservice architectures. The microservices approach to developing a single application is to divide the application into a set of small services, each running its own processes, which communicate with each other. Each small service can be scaled independently of the application and can be managed by different teams.

Wed, 12 Nov 2014 16:00:00 UTC

The Story of Apollo - Amazons Deployment Engine

Posted By Werner Vogels

Automated deployments are the backbone of a strong DevOps environment. Without efficient, reliable, and repeatable software updates, engineers need to redirect their focus from developing new features to managing and debugging their deployments. Amazon first faced this challenge many years ago. When making the move to a service-oriented architecture, Amazon refactored its software into small independent services and restructured its organization into small autonomous teams. Each team took on full ownership of the development and operation of a single service, and they worked directly with their customers to improve it. With this clear focus and control, the teams were able to quickly produce new features, but their deployment process soon became a bottleneck.

Thu, 06 Nov 2014 16:00:00 UTC

Don't Miss These Startup Activities at AWS re:Invent!

Posted By Werner Vogels

Im excited to be heading to Las Vegas in less than two weeks for our annual re:Invent conference. One of the highlights for me is being able to host an extensive lineup of startup-focused events which take place at re:Invent on Thursday, November 13. Heres a quick peak at the startup experience this year: Third Annual Startup Launches Im excited to host this event where five AWS-powered startups will make a significant, never-before-shared launch announcement on stage. Included in the announcements are special discounts on the newly-launched productsdiscounts only available to session attendees. And to top it all off, well have a happy hour immediately following the final launch announcement!

Thu, 23 Oct 2014 13:00:00 UTC

Expanding the Cloud  Introducing the AWS EU (Frankfurt) Region

Posted By Werner Vogels

Today, Amazon Web Services is expanding its worldwide coverage with the launch of a new AWS region in Frankfurt, Germany. This is our 11th infrastructure region and was built to support the strong demand we are seeing in Europe and to give our customers the option to run infrastructure located in Germany. The new Frankfurt region provides low millisecond latencies to major cities in continental Europe and is also run with carbon neutral power. With the launch of the new Frankfurt region customers now also have the ability to architect across multiple regions within the European Union. Many prominent German, and European, customers have been using AWS for quite some time already, including start-ups such as 6Wunderkinder, EyeEm, mytaxi, Onefootball, Soundcloud and Wooga, mid-market companies such as Airport Nuremburg, Euroforum, and Kärcher, and Enterprise companies such as Axel Springer, Hubert Burda Media, Kempinski Hotels, RTL, SAP, Software AG, and Talanx.

Wed, 08 Oct 2014 11:00:00 UTC

Document Model Support in DynamoDB: Flexibility, Availability, Performance, and Scale...Together at last

Posted By Werner Vogels

Today, Im thrilled to announce several major features that significantly enhance the development experience on DynamoDB. We are introducing native support for document model like JSON into DynamoDB, the ability to add / remove global secondary indexes, adding more flexible scaling options, and increasing the item size limit to 400KB. These improvements have been sought by many applications developers, and we are happy to be bringing them to you. The best part is that we are also significantly expanding the free tier many of you already enjoy by increasing the storage to 25 GB and throughput to 200 million requests per month.

Fri, 26 Sep 2014 09:00:00 UTC

AWS Pop-up Loft 2.0: Returning to San Francisco on October 1st

Posted By Werner Vogels

Its an exciting time in San Francisco as the return of the AWS Loft is fast approaching. Weve been working round-the-clock, making updates to ensure the experience is more fulfilling and educational than in June. Today were excited to announce that& On Wednesday, October 1st, well be returning to 925 Market Street! The AWS Loft is all about helping you scale and grow your business by offering free AWS technical resources. Youll have access to training including hands-on bootcamps and labs, and 1:1 sessions with AWS Solutions Architects.

Thu, 15 May 2014 17:00:00 UTC

Cloud computing in Europe should put power in the hands of the customer

Posted By Werner Vogels

This is an extended version of an article that appeared in the Guardian today We are rapidly entering into an era where massive computing power, digital storage and global network connections can be deployed by anyone as quickly and easily as turning on the lights. This is the promise  and the reality  of cloud computing which is driving tremendous change in the technology industry and transforming how we do business in Europe and around the world. Cloud computing unlocks innovation within organisations of all types and sizes. No longer do they need to spend valuable human and capital resources on maintaining and procuring expensive technology infrastructure and datacenters, they can focus their most valuable resources on what they do best, building better products and services for their customers.

Wed, 14 May 2014 14:00:00 UTC

The AWS Activate CTO to CTO series on Medium

Posted By Werner Vogels

I'm excited to announce a new blog dedicated to AWS startups. We're launching it on Medium, itself a startup on AWS. I kicked off the blog with a Q&A with the Medium CTO Don Neufeld. I really enjoyed Don's answers to my questions and there are some real gems in here for startup CTOs. Check it out. We'll be keeping this blog fresh with other startup spotlights and good technical content so follow the collection and keep up.

Wed, 23 Apr 2014 13:30:00 UTC

Expanding the Cloud: Docker Containers in Elastic Beanstalk

Posted By Werner Vogels

We launched Elastic Beanstalk in 2011 with support for Java web applications and Tomcat 6 in one region, and we've seen the service grow to 6 container types (Java/Tomcat, PHP, Ruby, Python, .NET, and Node.js) supported in 8 AWS regions around the world. The Elastic Beanstalk team spends a lot of time talking to AWS Developers, and in the last few months they've noticed a common theme in those conversations: developers tell us they're interested in Docker, and ask if we are thinking about making it easy to run and scale Docker workloads in AWS. Several weeks ago we made it simple to yum install Docker on your EC2 Instances running Amazon Linux, and today Elastic Beanstalk introduces the ability to deploy, manage, and scale Docker Containers.

Tue, 04 Feb 2014 07:00:00 UTC

Customer Centricity at Amazon Web Services

Posted By Werner Vogels

In the 2013 Amazon Shareholder letter, Jeff Bezos spent time explaining the decision to pursue a customer-centric way in our business. As regular readers of this letter will know, our energy at Amazon comes from the desire to impress customers rather than the zeal to best competitors. We dont take a view on which of these approaches is more likely to maximize business success. There are pros and cons to both and many examples of highly successful competitor-focused companies. We do work to pay attention to competitors and be inspired by them, but it is a fact that the customer-centric way is at this point a defining element of our culture.

Mon, 23 Dec 2013 20:00:00 UTC

Updated Lampson's Hints for Computer Systems Design

Posted By Werner Vogels

This year I have not been able to publish many back-to-basics readings, so I will not close the year with a recap of those. Instead I have a video of a wonderful presentation by Butler Lampson where he talks about the learnings of the past decades that helped him to update his excellent 1983 "Hints for computer system design". The presentation was part of the Heidelberg Laureate Forum helt in September of this year. At the Forum many of the Abel, Fields and Turing Laureates held presentations. Our most famous computer scientists like Fernando Carbato, Stephen Cook, Edward Feigenbaum, Juris Hartmanis, John Hopcroft, Alan Kay, Vinton Cerf, etc.

Thu, 12 Dec 2013 08:00:00 UTC

Taking DynamoDB beyond Key-Value: Now with Faster, More Flexible, More Powerful Query Capabilities

Posted By Werner Vogels

We launched DynamoDB last year to address the need for a cloud database that provides seamless scalability, irrespective of whether you are doing ten transactions or ten million transactions, while providing rock solid durability and availability. Our vision from the day we conceived DynamoDB was to fulfil this need without limiting the query functionality that people have come to expect from a database. However, we also knew that building a distributed database that has unlimited scale and maintains predictably high performance while providing rich and flexible query capabilities, is one of the hardest problems in database development, and will take a lot of effort and invention from our team of distributed database engineers to solve.

Tue, 26 Nov 2013 22:00:00 UTC

Expanding the Cloud: Enabling Globally Distributed Applications and Disaster Recovery

Posted By Werner Vogels

As I discussed in my re:Invent keynote earlier this month, I am now happy to announce the immediate availability of Amazon RDS Cross Region Read Replicas, which is another important enhancement for our customers using or planning to use multiple AWS Regions to deploy their applications. Cross Region Read Replicas are available for MySQL 5.6 and enable you to maintain a nearly up-to-date copy of your master database in a different AWS Region. In case of a regional disaster, you can simply promote your read replica in a different region to a master and point your application to it to resume operations.

Wed, 13 Nov 2013 15:00:00 UTC

AWS re:Invent 2013

Posted By Werner Vogels

Today we are kicking off AWS re:Invent 2013. Over the course of the next three days, we will host more than 200 sessions, training bootcamps, and hands on labs taught by expert AWS staff as well as dozens of our customers. This years conference kicks off with a keynote address by AWS Senior Vice President Andy Jassy, followed by my keynote on Thursday morning. Tune in to hear the latest from AWS and our customers. If youre not already here in Vegas with us, you can sign up to watch the keynotes on live stream here. Outside of the keynotes, there are an incredible number of sessions offering a tailored experience whether you are a developer, startup, executive, partner, or other.

Wed, 30 Oct 2013 16:00:00 UTC

Simplifying Mobile App Data Management with DynamoDB's Fine-Grained Access Control

Posted By Werner Vogels

Speed of development, scalability, and simplicity of management are among the critical needs of mobile developers. With the proliferation of mobile devices and users, and small agile teams that are tasked with building successful mobile apps that can grow from 100 users to 1 million users in a few days, scalability of the underlying infrastructure and simplicity of management are more important than ever. We created DynamoDB to make it easy to set up and scale databases so that developers can focus on building great apps without worrying about the muck of managing the database infrastructure. As I have mentioned previously, companies like Crittercism and Dropcam have already built exciting mobile businesses leveraging DynamoDB.

Fri, 25 Oct 2013 18:00:00 UTC

Back-to-Basics Weekend Reading - U-Net: A User-Level Network Interface

Posted By Werner Vogels

Many of you know Thorsten von Eicken as the founder of Rightscale, the company that has helped numerous organizations find their way onto AWS. In what seems almost a previous life by now Thorsten was one of the top young professors in Distributed Systems and I had the great pleasure of working with him at Cornell in the early 90's. What set Thorsten aside from so many other system research academics was his desire to build practical, working systems, a path that I followed as well. In the back to basics readings this week I am re-reading a paper from 1995 about the work that I did together with Thorsten on solving the problem of end-to-end low-latency communication on high-speed networks.

Thu, 10 Oct 2013 07:30:00 UTC

AWS Activate  Supporting Startups on AWS

Posted By Werner Vogels

I am very excited to announce AWS Activate, a program designed to provide startups with the resources they need to build applications on AWS. Startups will forever be a very important customer segment of AWS. They were among our first customers and along the way some amazing businesses have been built by these startups, many of which running for 100% on AWS. Startups operate in a world of high uncertainty and limited capital, so an elastic and on-demand infrastructure at low and variable cost aligns very naturally with their needs. By reducing the cost of failure and democratizing access to infrastructure, the cloud has enabled more startups to build, experiment, and scale.

Fri, 04 Oct 2013 21:30:00 UTC

Back-to-Basics Weekend Reading - Tor: The Second-Generation Onion Router

Posted By Werner Vogels

The anonymity routing network Tor is frequently in the news these days, which makes it a good case to read up on the fascinating technologies behind it. Tor stands for The Onion Router as its technology is based on the onion routing principles. These principles were first described by Goldschlag, et al., from the Naval Research Lab, in their 1996 paper on Hiding Routing Information. Almost immediately work started on addressing a number of omissions in the original work in what became known as the second-generation onion router. Tor is the implementation of such a second generation router and has a number of fascinating features.

Wed, 02 Oct 2013 23:00:00 UTC

Fear, Uncertainty and Desperation

Posted By Werner Vogels

Sat, 21 Sep 2013 00:30:00 UTC

Back-to-Basics Weekend Reading - A Decomposition Storage Model

Posted By Werner Vogels

Traditionally records in a database were stored as such: the data in a row was stored together for easy and fast retrieval. Not everybody agreed that the "N-ary Storage Model" (NSM) was the best approach for all workloads but it stayed dominant until hardware constraints, especially on caches, forced the community to revisit some of the alternatives. Combined with the rise of data warehouse workloads, where there is often significant redundancy in the values stored in columns, and database models based on column oriented storage took off. The first practical modern implementation is probably C-Store by Stonebraker, et al. in 2005.

Fri, 06 Sep 2013 07:00:00 UTC

Dutch Enterprises and The Cloud

Posted By Werner Vogels

This spring I travelled through Europe for the AWS Global Summit series. In my many conversations with customers, and with the media, I encountered surprise and excitement about the extent that European enterprises have already been using the Amazon Web Services for some time. Whether it is large telecommunications manufactures like Nokia Siemens Networks running their real-time data analytics for network operators on AWS, or a luxury hotel chain like Kempinski moving their core IT functions to AWS such that they can get out of the IT business, or a major newspaper corporation like News International, who plan to have 75% of their infrastructure running on AWS within 3 years to improve their agility, European enterprises have been moving to the cloud for some time to become more agile and competitive.

Thu, 05 Sep 2013 21:30:00 UTC

DynamoDB for Location Data: Geospatial querying on DynamoDB datasets

Posted By Werner Vogels

Over the past few years, two important trends that have been disrupting the database industry are mobile applications and big data. The explosive growth in mobile devices and mobile apps is generating a huge amount of data, which has fueled the demand for big data services and for high scale databases. Meanwhile, mobile app developers have shown that they care a lot about getting to market quickly, the ability to easily scale their app from 100 users to 1 million users on day 1, and the extreme low latency database performance that is crucial to ensure a great end-user experience. These factors have made DynamoDB a compelling database for mobile developers, who happen to be among the biggest adopters of this technology.

Wed, 04 Sep 2013 01:00:00 UTC

Expanding the Cloud: More memory, more caching and more performance for your data

Posted By Werner Vogels

Today, we added two important choices for customers running high performance apps in the cloud: support for Redis in Amazon ElastiCache and a new high memory database instance (db.cr1.8xlarge) for Amazon RDS. As we prepared to launch these features, I was struck not only by the range of services we provide to enable customers to run fully managed, scalable, high performance database workloads, including Amazon RDS, Amazon DynamoDB, Amazon Redshift and Amazon ElastiCache, but also by the pace at which these services are evolving and improving. Since you now have lots of choices to address your high performance database needs, I decided to write this blog to help you select the most appropriate services for your workload using lessons I have learnt by scaling the infrastructure for

Sat, 31 Aug 2013 00:30:00 UTC

Back-to-Basics Weekend Reading - An Introduction to Spatial Database Systems

Posted By Werner Vogels

Storing and querying datasets that contain objects in a geometric space have always required special treatment. The choice of data structures and query algorithms can easily make the different between a query that runs in seconds or in days. Much of the fundamental work has been done in the late eighties and early nineties, for examples around topological relations (disjoint, meet, equal, overlap, contains, etc.) , direction relations (north, north-east, etc.) and distance relations (far, near), and also with respect to spatial data structures (a great survey by Hanan Samet). With location becoming a more important attribute to many modern datasets a solid understanding of the tradeoffs is important.

Fri, 23 Aug 2013 19:00:00 UTC

Back-to-the-Future Weekend Reading - Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud

Posted By Werner Vogels

The intense travels around the world in the spring have kept me from keeping up on the historical reading that I would like to do, as such there have not been that many suggesting for the back-to-basics reading list. The fall is going be not that much different but I will make an effort to get back into a reading habit. I want to kick off the fall readings not with an historical paper but with two that detail GraphLab, an excellent framework for high performance machine learning that originally has been built by the Carlos Guestrin. GraphLab has been used to build several different data mining and graph processing toolkits and applications.

Tue, 13 Aug 2013 12:30:00 UTC

Making Mobile App Development Easier with Cross Platform Mobile Push

Posted By Werner Vogels

This year as I hosted AWS Summits in 12 different cities around the world, I met thousands of developers who are building powerful new applications for smartphones, tablets and other connected devices, all running mobile cloud backends on AWS. These developers want to engage their users with timely, dynamic content even when the users havent opened their mobile apps. For example, baseball fans want to know as soon as their favorite team player hits a home run, so they can watch a video replay and catch the rest of the game. The rising proliferation of cheap and powerful sensors means not only apps but smart devices want to communicate important information.

Fri, 19 Jul 2013 18:00:00 UTC

Feeling the Customer Love for AWS

Posted By Werner Vogels

We work hard to meet our customers expectations and to innovate continuous on their behalf. This week at the Singapore AWS Summit we were fortunate that our customers Astro Radio from Kuala Lumpur were willing to join us on stage. Jayaram Gopinath Nagaraj and Kavitha Doraimaickam gave truly electrifying presentation about how AWS has transformed their radio stations. They also brought with them a video that showed their appreciation for how we enable them to innovate. It humbling and fun at the same time.

Thu, 18 Jul 2013 00:00:00 UTC

AWS re:Invent 2013

Posted By Werner Vogels

The AWS re:Invent user conference last year in Las Vegas was by many described as the best technology conference they had been to in a long time. We had worked hard to give you great keynote sessions as well as deep technical content by AWS engineers, partners and customers. This year we will again work hard to create a conference that will exceed your expectations of a conference that is unique in its high quality content and engagement.You can choose from 175+ sessions, training bootcamps, hands-on labs, and hackathons to gain deeper skills and knowledge of the AWS Cloud. Bring your entire executive and technical teams and walk away with the skills and knowledge to refine your cloud strategy, improve developer productivity, increase application performance and security, and reduce infrastructure costs.

Mon, 08 Jul 2013 01:30:00 UTC

Exerting Fine Grain Control Over Your Cloud Resources

Posted By Werner Vogels

I am thrilled that now both Amazon EC2 and Amazon RDS support resource-level permissions. As customers move increasing amounts of compute and database workloads over to AWS, they have expressed an increased desire for finer grain control over their underlying resources. You can now use these new features to define the permissions your AWS IAM users (and applications) have to perform actions on specific or groups of Amazon EC2 and Amazon RDS resources. You can apply user-defined tags to your EC2 and RDS resources to help organize resources according to whatever schema is most relevant for a particular organization  be it an application stack, an organization unit, a cost center, or any other schema that might be appropriate.

Sun, 09 Jun 2013 01:00:00 UTC

Back-to-Basics Weekend Reading - Auctions and bidding: A guide for computer scientists

Posted By Werner Vogels

I have just returned from the AWS Summits in New Zealand and Japan, which were both very well attended and, according to the feedback, very successful. While I was in New Zealand I had great discussion with the folks from Trade Me, the auction site which according to some counts for 70% of all NZ internet traffic. This resulted in some deep technical conversations later, over beer, with some colleagues and customers about the principles behinds different auction and bidding styles. I noticed that my basic knowledge there was rather rusty and I have decided to use this weekend to go a bit more in-depth in the various styles and techniques.

Wed, 22 May 2013 15:30:00 UTC

Amazon Redshift and Designing for Security

Posted By Werner Vogels

Its been a few months since I last wrote about Amazon Redshift and I thought Id update you on some of the things we are hearing from customers. Since we launched, weve been adding over a hundred customers a week and are well over a thousand today. Thats pretty stunning. As far as I know, its unprecedented for this space. Weve enabled our customers to save tens of millions of dollars in up front capital expenses by using Amazon Redshift. Its clear that Amazon Redshifts message of price, performance and simplicity has resonated with our customers. Thats no surprise  these are core principles for every AWS service.

Thu, 16 May 2013 01:30:00 UTC

DynamoDB Keeps Getting Better (and cheaper!)

Posted By Werner Vogels

We love getting feedback so we can deliver the improvements and new features that really matter to our customers. You can see from the pace at which we roll out new functionality that teams across AWS take this very seriously. One of the teams thats iterating quickly is DynamoDB. They recently launched Local Secondary Indexes and today they are releasing several new features that will help customers build faster, cheaper, and more flexible applications: Parallel Scans  To be able to increase the throughput of table scans, the team has introduce new functionality that allows you to scan through the table with multiple threads concurrently.

Fri, 10 May 2013 16:30:00 UTC

Back-to-Basics Weekend Reading - Using continuations to implement thread management and communication in operating systems

Posted By Werner Vogels

I have returned from a great series of AWS Summits in NYC and in Europe so it is time to get back to some weekend reading. During the nineties much operating systems research focussed on microkernels, which resulted in a large collection of prototype systems: Mach 3.0, L3/L4, Plan 9, Xenokernel, Minix and others. Not many of those made into production, the version of Mach that rolled into Mac OS X through the XNU integration was an earlier, monolithic version. I believe commercially QNX has been the most successful microkernel. There was a wealth of interesting, fundamental research triggered by the concepts of microkernels: new communication paradigms, memory management structures, schedulers, etc.

Wed, 17 Apr 2013 17:30:00 UTC

Expanding the Cloud: Faster, More Flexible Queries with DynamoDB

Posted By Werner Vogels

Today, Im thrilled to announce that we have expanded the query capabilities of DynamoDB. We call the newest capability Local Secondary Indexes (LSI). While DynamoDB already allows you to perform low-latency queries based on your tables primary key, even at tremendous scale, LSI will now give you the ability to perform fast queries against other attributes (or columns) in your table. This gives you the ability to perform richer queries while still meeting the low-latency demands of responsive, scalable applications. Our customers have been asking us to expand the query capabilities of DynamoDB and were excited to see how they use LSI.

Fri, 12 Apr 2013 11:00:00 UTC

Back-to-Basics Weekend Reading - Join Processing in Relational Databases

Posted By Werner Vogels

Joins are one of the fundamental relational database query operations. It is very hard to implement the join operation efficiently as there any many unknowns in the execution of the operation. In the early days much relation database research was done in understanding the complexity of performing joins, what exactly impacted their performance and which approach performed better under which conditions. In 1992 Priti Mishra and Margaret Eich conducted a survey on what was achieved until then in Join Processing and described in details the algorithms, the implementation complexity and the performance. Which make it a good back-to-basics paper to read this weekend.

Fri, 05 Apr 2013 15:30:00 UTC

Back-to-Basics Weekend Reading - Practical Applications of Triggers and Constraints: Successes and Lingering Issues

Posted By Werner Vogels

At the end of the 80's Ceri and Widom were researching the fundamentals of integrity constraints in databases. In 2000 they were invited by the VLDB conference to review 10 years of work around Constraints and Triggers with an eye on the practical application of both abstractions. The resulting paper gives a good overview of the fundamentals of both concepts. Practical Applications of Triggers and Constraints: Success Stories and Lingering Issues, Stefano Ceri, Roberta Cochrane, and Jennifer. Widom, In 26th Very Large Data Bases Conference Proceedings, Cairo, September 2000, Pages 254-262

Fri, 29 Mar 2013 22:30:00 UTC

Back-to-Basics Weekend Reading - Principles of Transaction-Oriented Database Recovery

Posted By Werner Vogels

I have been reading mainly newer papers in the beginning of this year, but it is time to get back to the basics and start reading some more historical papers again. From the time when researchers and engineers where laying the foundations for our current systems. A good early paper to start again is the Survey that Härder en Reuter did on Database Recovery in 1983. Principles of Transaction-Oriented Database Recovery, Theo Härder and Andreas Reuter, ACM Computing Surveys, Volume 15 Issue 4, December 1983, Pages 287-317

Thu, 21 Mar 2013 00:00:00 UTC

The Netflix OSS Cloud Prize

Posted By Werner Vogels

Netflix has over the years become one of the absolute best engineering powerhouses for building cloud-native applications. At AWS we are very proud to be their infrastructure partner and every day we learn from how they use our cloud services. Many of the observations I talk about in my 21st Century Application Architectures presentation come from seeing Netflix architects at work. Netflix has gone beyond just building great applications; they have made fundamental pieces of their cloud platform available as open source and many in the industry have responded to that with great enthusiasm, evidenced by the packed Netflix House in February where people came to hear more about NetflixOSS.

Mon, 11 Mar 2013 23:00:00 UTC

Beanstalk a la Node

Posted By Werner Vogels

I spent a lot of time talking to AWS developers, many working in the gaming and mobile space, and most of them have been finding Node.js well suited for their web applications. With its asynchronous, event-driven programming model, Node.js allows these developers to handle a large number of concurrent connections with low latencies. These developers typically use EC2 instances combined with one of our database services to create web services used for data retrievals or to create dynamic mobile interfaces. Today, AWS Elastic Beanstalk just added support for Node.js to help developers easily deploy and manage these web applications on AWS.

Fri, 08 Mar 2013 04:00:00 UTC

DynamoDB One Year Later: Bigger, Better, and 85% Cheaper&

Posted By Werner Vogels

Time passes very quickly around here and I hadnt realized until recently that over a year has gone by since we launched DynamoDB. As I sat down with the DynamoDB team to review our progress over the last year, I realized that DynamoDB had surpassed even my own expectations for how easily applications could achieve massive scale and high availability with DynamoDB. Many of our customers have, with the click of a button, created DynamoDB deployments in a matter of minutes that are able to serve trillions of database requests per year. Ive written about it before, but I continue to be impressed by Shazams use of DynamoDB, which is an extreme example of how DynamoDBs fast and easy scalability can be quickly applied to building high scale applications.

Tue, 19 Feb 2013 07:30:00 UTC

Expanding the Cloud - Introducing AWS OpsWorks, a Powerful Application Management Solution

Posted By Werner Vogels

Today Amazon Web Services launched AWS OpsWorks, a flexible application management solution with automation tools that enable you to model and control your applications and their supporting infrastructure. OpsWorks allows you to manage the complete application lifecycle, including resource provisioning, configuration management, application deployment, software updates, monitoring, and access control. As with all the AWS Application Management services AWS OpsWorks is provided at no additional charge. AWS customers only pay for those resources that they have used. Simplified Application Management OpsWorks is designed for IT administrators and ops-minded developers who want an easy way to manage applications of nearly any scale and complexity without sacrificing control.

Fri, 15 Feb 2013 08:01:00 UTC

Amazon Redshift and Designing for Resilience

Posted By Werner Vogels

As you may remember from our announcement at re: Invent in November 2012, Amazon Redshift is a fast and powerful, fully managed, petabyte-scale data warehouse service that delivers fast query performance at less than one tenth the cost of most traditional data warehouse systems. Ive been eagerly waiting for Amazon Redshifts launch since we announced the service preview at re: Invent and Im delighted that its now available for all customers in the US East (N. Virginia) Region, with additional AWS Regions planned for the coming months. To get started with Amazon Redshift, visit: Amazon Redshift and Resilience Previously, Ive written at length about how Amazon Redshift achieves high performance.

Tue, 12 Feb 2013 00:00:00 UTC

Expanding the Cloud: The Amazon Elastic Transcoder

Posted By Werner Vogels

While I was returning from an exciting time in New Orleans watching the Super Bowl, AWS launched a very cool, brand new service: Amazon Elastic Transcoder. Amazon Elastic Transcoder is designed to be very easy to use, scalable and cost-effective video transcoding in the cloud. Jeff Barr did an excellent job running through the service on his blog and you can also check out the detail page. I spent a lot of time talking to AWS customers who were also in New Orleans, many of them who work with media, and all emphasized that transcoding fits naturally with services that we already offer like storage (Amazon S3 and Glacier) and delivery (Amazon CloudFront).

Fri, 25 Jan 2013 17:00:00 UTC

Back-to-Basics Weekend Reading - Epidemics

Posted By Werner Vogels

My paper to read this weekend was the Alan Demers' seminal paper on epidemic techniques for database replication. I realized that in 2004, before my Amazon days, I already wrote a blog post about the fundamental publications in the area of epidemics, so this seems like a good moment to revisit that with updated links, etc. History of Epidemics In the past 6-8 years we have been using various epidemic techniques in building our reliable and scalable distributed systems with great success. Now that industry is starting to deal with issues of scale that can almost only be solved by using epidemic techniques, it becomes important to produce some basic pointers to the origin of the use of epidemics in distributed systems.

Thu, 27 Dec 2012 17:00:00 UTC

My Best Christmas Present  Root Domain Support for Amazon S3 Website Hosting

Posted By Werner Vogels

I have been a big fan of the Amazon S3 Static Website Hosting feature since its launch and this blog happily is being served from it. S3 is not only a highly reliable and available storage service but also one of the most powerful web serving engines that exists today. By storing your website in Amazon S3, you suddenly no longer have to worry about scaling, replication, performance, security, etc. All of that is handled seamlessly by S3. As such I am very happy that the Amazon S3 team has finally knocked off the last piece of dependency on an external infrastructure piece.

Sat, 22 Dec 2012 23:00:00 UTC

An Album for Each Year - 2012 Version

Posted By Werner Vogels

About 5 years ago I joined a challenge to list "a favorite album for every year of your life." The challenge has two restrictions: only one album per year and there can be no repeats of artists. I added for myself the restriction that I should actually own the album, which restricts the set to choose from significantly and also makes for some peculiar choices. My list stopped in 2007, so now that 2012 is almost at its end it is a good moment to add the next 5 years to the list 1958: Jerry Lee Lewis, Great Balls of Fire 1959: Ray Charles, What I'd Say 1960: Miles Davis, Sketches of Spain 1961: Robert Johnson, King of the Delta Blues Singers 1962: Booker T & MG, Green Onions 1963: James Brown, Live at the Apollo 1964: John Coltrane, Love Supreme 1965: Bob Dylan, Highway 61 Revisted 1966: Cream, ...

Wed, 19 Dec 2012 03:00:00 UTC

The Back-to-Basics Readings of 2012

Posted By Werner Vogels

After the AWS re: Invent conference I spent two weeks in Europe for the last customer visits of the year. I have since returned and am now in New York City enjoying a few days of winding down the last activities of the year before spending the holidays here with family. Do not expect too many blog posts or twitter updates. Although there are still a few very exciting AWS news updates to happen this year. I thought this was a good moment to collect all the readings I suggested this year in one summary post. It was not until later in the year that I started to recording the readings here on the blog, so I hope this is indeed the complete list.

Fri, 30 Nov 2012 19:00:00 UTC

Back-to-Basics Weekend Reading - Sparse Partitions

Posted By Werner Vogels

The amazing AWS re: Invent conference completed last night and I am on my way to Europe for a last visit to customers this year. I am carrying with me a more theoretical paper on the principles of distributed computing: Sparse Partitions by Awerbug and Peleg. It deals with the failure of control if networks grow larger and presents several solutions based on locality that have found practical applications. Sparse Partition, Baruch Awerbuch and David Peleg, Proceedings of the 31st Annual Symposium on Foundations of Computer Science (FOCS), 503-513, October 1990.

Wed, 28 Nov 2012 17:00:00 UTC

Expanding the Cloud  Announcing Amazon Redshift, a Petabyte-scale Data Warehouse Service

Posted By Werner Vogels

Today, we are excited to announce the limited preview of Amazon Redshift, a fast and powerful, fully managed, petabyte-scale data warehouse service in the cloud. Amazon Redshift enables customers to obtain dramatically increased query performance when analyzing datasets ranging in size from hundreds of gigabytes to a petabyte or more, using the same SQL-based business intelligence tools they use today. Customers have been asking us for a data warehouse service for some time now and were excited to be able to deliver this to them. Amazon Redshift uses a variety of innovations to enable customers to rapidly analyze datasets ranging in size from several hundred gigabytes to a petabyte and more.

Sun, 18 Nov 2012 07:00:00 UTC

Back-to-Basics Weekend Reading - The Andrew File System

Posted By Werner Vogels

This weekend I am heading to Brussels for meetings with the European Commission, specifically with Vice-president Neelie Kroes who owns the Digital Agenda for the EU, about how to accelerate cloud usage in both business and government in Europe. I am bringing with me a paper with one of first distributed systems that had actually see wide-spread commercial deployment. The Andrew File System (AFS) was developed at CMU and was much more than just a distributed file systems and had a very interesting caching and volume replication architecture. Scale and performance in a distributed file system, John H. Howard, Michael L.

Mon, 12 Nov 2012 13:00:00 UTC

Expanding the Cloud  introducing the Asia Pacific (Sydney) Region

Posted By Werner Vogels

Today, Amazon Web Services is expanding its worldwide coverage with the launch of a new AWS Region in Sydney, Australia. This new Asia Pacific (Sydney) Region has been highly requested by companies worldwide, and it provides low latency access to AWS services for those who target customers in Australia and New Zealand. The Region launches with two Availability Zones to help customers build highly available applications. I have visited Australia at least twice every year for the past four years and I have seen first-hand evidence of the tremendous interest there is in the AWS service. Many young businesses as well as established enterprises are already using AWS, many of them targeting customers globally.

Sat, 10 Nov 2012 01:00:00 UTC

Back-to-Basics Weekend Reading - Using Encryption for Authentication

Posted By Werner Vogels

Now that I am enjoying some time in Seattle with real weekends, I like to remind you that my reading list is called back-to-basics. My goal with reading these papers is that by revisiting the original problems systems researchers were trying to solve you get a much better understanding of the challenges we are often still faced with today. That means that many papers on these papers are "old", as I was recently told, even published before some of you were born :-). That might definitely be the case with this famous Needham - Schroeder paper from 1978. Roger Needham and Mike Schroeder were some of the first researchers to tackle the problem of secure communication over insecure networks; their protocols deal with how to authenticate both parties and how to establish a secure channel between the parties.

Thu, 08 Nov 2012 23:00:00 UTC

Improving the Cloud - More Efficient Queuing with SQS

Posted By Werner Vogels

The Amazon Simple Queue Service (SQS) is a highly scalable, reliable and elastic queuing service that 'just works'. Customers from various verticals (media, social gaming, mobile, news, advertisement) such as Netflix, Shazam and Scopely have used SQS in variety of use-cases requiring loose coupling and high performance. For example, AWS customers use SQS for asynchronous communication pipelines, buffer queues for databases, asynchronous work queues, and moving latency out of highly responsive requests paths. Today, the SQS team is launching two important features  Long Polling and richer client functionality in the SQS SDK  that we believe will extend the reach of SQS to new use cases by reducing the cost of high scale messaging for our customers.

Fri, 02 Nov 2012 18:00:00 UTC

Back-to-Basics Weekend Reading - Weighted Voting for Replicated Data

Posted By Werner Vogels

The last two weeks in Europe and Israel (The image above is from Tel Aviv) were intense so I didn't get to do much reading, hence I didn't post any reading suggestions. This weekend I pick a true back-to-basics paper to read; Dave Gifford's paper on Weighted Voted was the first to describe the "r+w" overlapping quorum approach to reason about the consistency of replicated data. Weighted voting for replicated data, David K. Gifford, Proceedings of the 7th ACM Symposium on Operating Systems Principles, December 10-12, 1979, Pacific Grove, CA USA

Sat, 13 Oct 2012 01:00:00 UTC

Back-to-Basics Weekend Reading - Automatic Reconfiguration in Autonet

Posted By Werner Vogels

I wrote this post last week but didn't get around to publishing it. I am heading to Europe this weekend for, among other things, Structure Europe and the AWS Summit in Tel Aviv. My time in India is almost over and I am heading back to Seattle. I am bringing with me on the plane a relatively unknown paper that I really enjoyed when it was first published. Autonet was a point-to-point network designed at Dec SRC research lab by Mike Schroeder's group. The original paper on Autonet can be found here. I am actually picking a follow-up paper to read this weekend, it deals with fault-tolerance of the network through automatic reconfiguration of its components.

Fri, 28 Sep 2012 16:00:00 UTC

Back-to-Basics Weekend Reading - Counting Bloom Filters

Posted By Werner Vogels

I am in India for the AWS Summits in Mumbai, Chennai and Bangalore (next week). As always in India I have an amazing time, the events are packed, the participants are extremely enthusiastic and eager to learn, the customers very appreciative and the food is just amazing. This weeks reading was triggered by a note from Matt Wood who ran into a great in-depth analysis of the Bloom Filter data structure by Michael Nielsen on his Data Driven Intelligence blog. I love probabilistic data structures and Bloom filters have unique properties of possible false positives, but no false-negatives. They have been used in many network devices, network protocols and distributed applications where a question like "have I possibly seen this before" needs to be able to operate at very large scale.

Tue, 25 Sep 2012 16:00:00 UTC

Expanding the Cloud  Provisioned IOPS for Amazon RDS

Posted By Werner Vogels

Following the huge success of being able to provision a consistent, user-requested I/O rate for DynamoDB and Elastic Block Store (EBS), the AWS Database Services team has now released Provisioned IOPS, a new high performance storage option for the Amazon Relational Database Service (Amazon RDS). Customers can provision up to 10,000 IOPS (input/output operations per second) per database instance to help ensure that their databases can run the most stringent workloads with rock solid, consistent performance. High Performance I/O Amazon RDS Provisioned IOPS is intended for applications that need predictable performance and have database workloads that generate largely random I/O. Amazon RDS Provisioned IOPS is ideal for mission-critical online transaction processing (OLTP) workloads that require a high performance storage option with consistent IOPS, within a narrow band of tolerance.

Sat, 22 Sep 2012 01:00:00 UTC

Back-to-Basics Weekend Reading - Leases

Posted By Werner Vogels

I was in Los Angeles this week for the Digital Media on AWS Summit and to visit many of the studios and production houses that are using AWS for production and post-production work. There is some real jaw dropping work being done around this town and I had the privilege to see some of these highly guarded secrets, all powered by AWS. Of the work that is already public the systems that Uplynk has built for Disney/ABC are impressive. The single source format approach has their customers very enthusiastic about how simple multi device stream is for them with Uplynk handling all the transcoding, dynamic ad-insertion, black out handling by switching from live to vod, etc.

Sat, 15 Sep 2012 00:00:00 UTC

Back-to-the-Future Weekend Reading - CryptDB

Posted By Werner Vogels

This weekend I am not going back in time to fundamentals, but looking forward to the future. Encryption techniques to protect sensitive information such as personal identifiable information are becoming more and more practical, and ubiquitously. Whether in-the-cloud or on-premise there is a shift to a model where individual applications need to protect themselves instead of relying on firewall-like techniques. That goes especially for the interaction between applications and storage engines, and between applications and databases. In last year's SOSP Hari Balakrishnan's group at MIT CSAIL team presented a paper on CryptDB which has a novel SQL-aware encryption approach. "

Wed, 12 Sep 2012 08:00:00 UTC

Expanding Flexibility - Introducing the Reserved Instance Marketplace

Posted By Werner Vogels

Today we launched a new feature that enables you to buy and sell Amazon EC2 Reserved Instances. Reserved Instances are an important pricing option for AWS customers to drive cost down. If you are able to predict the capacity required to run your application, there is likely some combination of Reserved Instance options that will help you drive you costs down significantly (up to 71%) when compared to on-demand pricing. There are three options: heavy-, medium- and low-usage options that allow you to optimize your savings depending on how much you plan to use your Reserved Instance. However, sometimes business and architectures change so that you need to change your mix of Reserved Instances.

Tue, 11 Sep 2012 00:00:00 UTC

A Million Miles Away

Posted By Werner Vogels

I just received a note from United Airlines that I had flown 1 million miles on their airline. I didn't start flying United until I moved to Seattle, so all of these have been miles on Amazon business. The folks I was with immediately started cracking some jokes about a million miles of torture, but that was not at all what came to my mind. I thought about all the wonderful things I got to do on those million miles; all the great engineers and CxO's that I have met, all the cool startups that made time for me and shared their passion, and all the amazing enterprises going through major transformations who opened up about the inner workings of their businesses to me.

Fri, 07 Sep 2012 22:00:00 UTC

Back-to-Basics Weekend Reading - Load Sharing

Posted By Werner Vogels

One of the main reasons for picking some of these older papers as back-to-basics reading is that the first researchers on a topic had to develop the fundamental models and principles from scratch. As such there is much to learn from the descriptions of those first algorithms and models. One area that is tremendously important in todays distributed systems and has a long history is load balancing and load management. In this 1986 paper on load sharing Eager, Lazowska and Zahorjan layout the basics. Even though these early system lack todays scale and complexity the fundamentals are there.

Fri, 31 Aug 2012 16:00:00 UTC

Back-to-Basics Weekend Reading - Granularity of locks

Posted By Werner Vogels

I am at funconf in Ireland. After a ride in a Delorean, a private train ride to Galway and a helicopter flight I am sitting outside a cottage on the island of Inishmore. The attendants are an amazing group of people, mostly engineers, and I am humbled to be invited along. Everybody is equally passionate about "doing the right" in tech, in business and in life. The discussion shift quickly form one to the other. I brought two papers with me on this part of the trip, one was the paper that Ben Black sent out earlier this week on Data-Structures for Geometric Approximation, which is a great in-depth review of that area and it will take me a while to finish it.

Fri, 24 Aug 2012 20:00:00 UTC

Back-to-Basics Weekend Reading - The 5 Minute Rule

Posted By Werner Vogels

I am in the midst of my South America tour in the beautiful but very cold Santiago, Chili. The AWS team launched this week Amazon Glacier, a cold storage archive service at the very low price point of $0.01 per GB/month. Which makes this week a good moment to read up on some of the historical work around the costs of data engineering. For this purpose I have picked work based on two papers by Jim Gray, the brilliant IBM / Tandem / Microsoft researcher, who won a Turing award for his contributions to data and transaction processing. The papers are from 1987, 1997 and 2007.

Tue, 21 Aug 2012 03:01:00 UTC

Expanding the Cloud  Managing Cold Storage with Amazon Glacier

Posted By Werner Vogels

Managing long-term digital archiving is a challenge for almost every company. With the introduction of Amazon Glacier, IT organizations now have a solution that removes the headaches of digital archiving and provides extremely low cost storage. Many organizations have to manage some form of long term archiving. Enterprises have regulatory and business requirements to retain everything from email to customers transactions, hospitals create archives of all digital assets related to patients, research and scientific organizations are creating substantial historical archives of their findings, governments want to provide long-term open data access, media companies are creating huge repositories of digital assets, and libraries and other organizations have been looking to archive everything that takes place in society.

Fri, 17 Aug 2012 22:00:00 UTC

Back-to-Basics Weekend Reading - Staged Event-Driven Architecture

Posted By Werner Vogels

I am in São Paolo, Brazil for the 2012 AWS Latin America Summit and for The Next Web Latin America conference. I will also be visiting Chile and Mexico on this trip and have the great fortune to meet many of our Latin American AWS customers. Staged Event-Driven Architecture. Matt Welsh's thesis work at Berkeley was on building high-performance internet services. In a time when the debates on Threads vs Events were still rampant he came up with a practical and elegant approach of combining both, delivering excellent results. Several of the principles from this paper have made it into systems I have since built.

Thu, 16 Aug 2012 17:00:00 UTC

Total Cost of Ownership and the Return on Agility

Posted By Werner Vogels

In the many meetings with customers in which I have done a deep dive on their architecture and applications to help them create an accurate cost picture, I have observed two common patterns: 1) It is hard for customers to come to an accurate Total Cost of Ownership (TCO) calculation of an on-premise installation and 2) they struggle with how to account for the Return on Agility; the fact that they are now able to pursue business opportunities much faster at much lower costs points than before. Both of these are important as they help customer accurately gauge the economic benefits of running their applications in the cloud.

Sat, 11 Aug 2012 17:00:00 UTC

Back-to-Basics Weekend Reading - On Naming and Binding

Posted By Werner Vogels

I have just returned from two wonderful weeks in the rugged NorthWest wilderness. Every time again I am amazed of the magnificent outdoors at only two hours drive from Seattle. This weeks back to basics is a paper David Richardson reminded me of when we were discussing the merits of Saltzer's end-to-end paper. It is a note by Saltzer that in the most clear and fundamental way lays out the concepts of naming, addressing, routing and binding in distributed systems. It was republished as an IETF RFC given it importance for the design of internet systems: Saltzer, J. H., "On the Naming and Binding of Network Destinations", RFC 1498, August 1993.

Sat, 28 Jul 2012 04:00:00 UTC

Back-to-Basics Weekend Reading - On Systems Simplicity

Posted By Werner Vogels

This weekend's reading departs a bit from the more academic papers of the past weeks. They are two classics that deal with great observations about the tensions between simplicity and complexity in building systems. The first paper is the 1980 Turing Lecture delivered by Tony Hoare, "The Emporers Old Clothes". He had received the Turing Award for his contributions to the fundamentals of programming languages, although for me some of his most influential work, communication sequential processes, still had to happen. In his lecture prof. Hoare, in the superb way of great story tellers, has many observations about programmers, program languages and systems building.

Fri, 20 Jul 2012 19:00:00 UTC

Back-to-Basics Weekend Reading - Virtualizing Operating Systems

Posted By Werner Vogels

This weekend's back-to-basics reading is on operating system virtualization. There are two papers that deserve the "classic" tag as they both form the basis for operating system virtualization that is in production today. Stanford's Disco, the predecessor of VMWare, uses a full hardware virtualization approach, where Cambridge's Xen introduced us to paravirtualization. Disco: Running Commodity Operating Systems on Scalable Multiprocessors by Edouard Bugnion, Scott Devine, Kinshuk Govil, Mendel Rosenblum in the Proceedings of the 16th ACM Symposium on Operating Systems Principles, October 5-8, 1997, St. Malo, France. Xen and the art of virtualization by Paul Barham, Boris Dragovic, Keir Fraser, Steven Hand, Timothy L.

Thu, 19 Jul 2012 07:00:00 UTC

Expanding The Cloud  High Performance I/O Instances for Amazon EC2

Posted By Werner Vogels

AWS customers are bringing their most demanding workloads onto the cloud. These include the likes of high performance computation, for which we introduced the Cluster Compute and Cluster GPU instance types. Customers are also bringing workloads on AWS that require dedicated and high performance IO for which we are now introducing a new Amazon EC2 instance type, the High I/O Quadruple Extra Large (hi1.4xlarge), to meet their needs. The hi1.4xlarge has 8 cores and 60.5GB of memory. Most importantly it has 2 SSDs of 1 TB each and a 10 Gb/s Ethernet NIC that using placement groups can be directly connected to other High I/O instances.

Mon, 16 Jul 2012 16:00:00 UTC

Register for AWS re: Invent

Posted By Werner Vogels

The first annual AWS user and partner conference will be held November 27-29 at The Venetian in Las Vega. It is shaping up to be a great event with many Amazonians, partners and customers presenting in well over 150 sessions. There are sessions in many different categories: Architecture, Big Data, HPC, Computer & Networking, Storage, Databases, Security, Tools & Languages, Media Sharing & Content Delivery, Managing AWS Resources, Enterprise IT, Mobile, Start-up, and more. Starting today the information about the sessions and the registration is live at General registration opens up on July 25, 2012, however active AWS customers will get a chance to jump the line and start registering on July 20.

Fri, 13 Jul 2012 16:00:00 UTC

Back-to-Basics Weekend Reading - Hints for Computer Systems Design

Posted By Werner Vogels

For a while now I have been on a track to read one influential/fundamental Computer Science paper each weekend. I find that going back to the basics of system, network and language design forces a good appreciation for keeping designs simple and focus on those fundamentals that matter most to users. Often I posted the paper on twitter and a number of times I have had requests like "what was the paper you posted three weeks ago about memory management?" . I will now post them here so going back in time will be easy. Last weeks paper was the classic End-To-End Arguments in System Design, by J.

Thu, 21 Jun 2012 16:00:00 UTC

Amazon DynamoDB - From the Super Bowl to WeatherBug

Posted By Werner Vogels

Amazon DynamoDB is the fastest growing new service in the history of AWS. In the five months since it launched in January, DynamoDB, our fast and scalable NoSQL database service, has been setting AWS growth records. Customers have used DynamoDB to support Super Bowl advertising campaigns, drive Facebook applications, collect and analyze data from sensor networks, track gaming information, and more. Customers such as Electronic Arts, News International, SmugMug, Shazam, IMDb, Amazon Cloud Drive, and many others are using DynamoDB to power their applications. The number of items that customers are storing in DynamoDB is more than doubling every couple of months (an item is the basic unit of data stored in DynamoDB and is between 0-64KB).

Mon, 14 May 2012 05:01:00 UTC

Dynamic Content Support in Amazon CloudFront

Posted By Werner Vogels

In the past three and a half years, Amazon CloudFront has changed the content delivery landscape. It has demonstrated that a CDN does not have to be complex to use with expensive contracts, minimum commits, or upfront fees, such that you are forcibly locked into a single vendor for a long time. CloudFront is simple, fast and reliable with the usual pay-as-you-go model. With just one click you can enable content to be distributed to the customer with low latency and high-reliability. Today Amazon CloudFront has taken another major step forward in ease of use. It now supports delivery of entire websites containing both static objects and dynamic content.

Wed, 09 May 2012 21:00:00 UTC

AWS re: Invent

Posted By Werner Vogels

Hundreds of thousands of businesses in over 190 countries are relying on AWS to support some or all of their business and IT needs. From SAP and Oracle production installations to risk management HPC, from Internet banking to protein folding, from social gaming to mobile collaboration tools, from video distribution to hotel reservations systems, supply chain management and medical archiving, the list of what our customers achieve using the AWS cloud is fascinating. Many of our customers will tell you that although the cost savings that AWS brings them are important, more important is that they are able to be more agile, that they are able to move faster in a world with murderous competition and highly compressed time-to-markets.

Tue, 08 May 2012 21:00:00 UTC

Expanding the Cloud for Windows Developers

Posted By Werner Vogels

The software that powers todays world of Internet services has become incredibly diverse. Todays announcement of Amazon RDS for Microsoft SQL Server and .NET support for AWS Elastic Beanstalk marks another important step in our commitment to increase the flexibility for AWS customers to use the choice of operating system, programming language, development tools and database software that meet their application requirements. Using the AWS Toolkit for Visual Studio, you can now deploy your .NET applications to AWS Elastic Beanstalk directly from your Visual Studio environment without changing any code. You can then off load the management and scaling of your database and application stack to Amazon RDS and AWS Elastic Beanstalk, and focus on adding value to your customers.

Thu, 19 Apr 2012 06:00:00 UTC

Expanding the Cloud  Introducing AWS Marketplace

Posted By Werner Vogels

Today Amazon Web Services launched AWS Marketplace, an online store that makes it easy for you to find, buy, and immediately start using software and services that run on the AWS Cloud. You can use AWS Marketplaces 1-Click deployment to quickly launch pre-configured software on your own Amazon EC2 instances and pay only for what you use, by the hour or month. AWS handles billing and payments, and software charges appear on your AWS bill. Marketplace has software listings from well-known vendors including 10gen, CA, Canonical, Couchbase, Check Point Software, IBM, Microsoft, SAP, Zend, and others, as well as many widely used open source offerings including Wordpress, Drupal, and MediaWiki.

Thu, 19 Apr 2012 06:00:00 UTC

Do You Want to Help Build the Next AWS Service?

Posted By Werner Vogels

Over the past several years Ive spent much of my time traveling around the world speaking about distributed systems. From building infinitely scalable data stores, architectures for high performance computing, to the challenges imposed by the CAP theorem, there are wonderful, complex, fascinating problems to be solved in the area of distributed computing. During my travels Ive met thousands of brilliant engineers who are leveraging the cloud to deliver exciting new products and revolutionize IT as we know it. One thing thats become obvious to me is that there are innovative, inspiring developers in every corner of the planet from Australia to Iceland and from Israel to Peru.

Thu, 12 Apr 2012 07:00:00 UTC

Expanding the Cloud  Introducing Amazon CloudSearch

Posted By Werner Vogels

Today Amazon Web Services is introducing Amazon CloudSearch, a new web service that brings the power of the Amazon.coms search technology to every developer. Amazon CloudSearch provides a fully-featured search engine that is easy to manage and scale. It offers full-text search with features like faceting and user-defined rank functions. And like most AWS services, Amazon CloudSearch scales automatically as your data and traffic grow, making it an easy choice for applications small to large. With Amazon CloudSearch, developers just create a Search Domain, upload data, and start querying. Why Search? Search is an essential part of many of today's cloud-centric applications.

Fri, 06 Apr 2012 11:00:00 UTC

Customer Conversations - How Intuit and Edmodo Innovate using Amazon RDS

Posted By Werner Vogels

From tax preparation to safe social networks, Amazon RDS brings new and innovative applications to the cloud Empowering innovation is at the heart of everything we do at Amazon Web Services (AWS). I often get to meet, discuss, and learn from innovators how they are using AWS to deliver transformative applications to their users, customers and partners. Often we think about innovation as doing 'new things' or based on revolutionary new technologies such as DynamoDB, but it is more important to ensure that one can also innovate based on existing paradigms. One of the services that is very successful in driving innovation at our customers in this context is Amazon RDS, the Relational Database Service.

Wed, 21 Mar 2012 04:00:00 UTC

A Thousand Platforms ...

Posted By Werner Vogels

Todays AWS Elastic Beanstalk announcement of PHP and Git support reminded me of the post where I mentioned that we want to let a thousand platforms bloom on AWS. Some might ask why AWS would want a thousand platforms. One of the most important AWS principles is flexibility. Flexibility is in the choice of software and languages running on AWS, in the tools and interfaces available to manipulate resources and applications, and in the ability to leverage services from other providers. One of our customers I met last week was talking about his application and how it runs on AWS; He collects geo-location data, analyzes and crunches this data using Elastic Map Reduce, stores the data for quick access in DynamoDB, runs his user interface on Heroku and his web services layer for mobile devices on Elastic Beanstalk.

Mon, 05 Mar 2012 22:01:00 UTC

Driving Compute Cost Down for AWS Customers

Posted By Werner Vogels

AWS today announced a substantial price drop from March 1, 2012 for many of the Amazon EC2, Amazon RDS, and Amazon ElastiCache instances types around the world. For example, the popular m1.small instance type will see a price drop of 6% for EC2 On-Demand usage and 33% for EC2 Reserved Instance usage. Some of the other instance types have even greater savings: for example, the high memory M2 instances will see a 10% price cut for On Demand and 37% for Reserved instances. Similarly, Amazon RDS will cut its On-Demand prices by up to 10% and Reserved Instance prices by up to 42%.

Wed, 22 Feb 2012 08:01:01 UTC

Expanding the Cloud  The Amazon Simple Workflow Service

Posted By Werner Vogels

Today AWS launched an exciting new service for developers: the Amazon Simple Workflow Service. Amazon SWF is an orchestration service for building scalable distributed applications. Often an application consists of several different tasks to be performed in particular sequence driven by a set of dynamic conditions. Amazon SWF makes it very easy for developers to architect and implement these tasks, run them in the cloud or on premise and coordinate their flow. Amazon SWF manages the execution flow such that the tasks are load balanced across the registered workers, that inter-task dependencies are respected, that concurrency is handled appropriately and that child workflows are executed.

Tue, 07 Feb 2012 05:00:00 UTC

Driving Storage Costs Down for AWS Customers

Posted By Werner Vogels

One of the things that differentiates Amazon Web Services from other technology providers is its commitment to let customers benefits from continuous cost-cutting innovations and from the economies of scale AWS is able to achieve. As we showed last week one of the services that is growing rapidly is the Amazon Simple Storage Service (S3). AWS today announced a substantial price drop per February 1, 2012 for Amazon S3 standard storage to help customers drive their storage cost down. A customer storing 50TB will see on average a 12% drop in cost when they get their Amazon S3 bill for February.

Mon, 23 Jan 2012 08:01:01 UTC

Expanding the Cloud - The AWS Storage Gateway

Posted By Werner Vogels

Today Amazon Web Services has launched the AWS Storage Gateway, making the power of secure and reliable cloud storage accessible from customers on-premises applications. We have been working closely with our customers on their requests to bring the power of the Amazon Web Services cloud closer to their existing on-premises compute infrastructures. The Amazon Virtual Private Cloud extends on-premises compute with all the power of AWS, making it elastic, scalable and highly reliable. AWS Identity and Access Management brings together on-premises and cloud identity management. VM Import allows our customers to move virtual machine images from their datacenters to the Cloud and Amazon Direct Connect makes the network latencies and bandwidth between on-premises and AWS more predictable.

Wed, 18 Jan 2012 15:00:00 UTC

Amazon DynamoDB  a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications

Posted By Werner Vogels

Today is a very exciting day as we release Amazon DynamoDB, a fast, highly reliable and cost-effective NoSQL database service designed for internet scale applications. DynamoDB is the result of 15 years of learning in the areas of large scale non-relational databases and cloud services. Several years ago we published a paper on the details of Amazons Dynamo technology, which was one of the first non-relational databases developed at Amazon. The original Dynamo design was based on a core set of strong distributed systems principles resulting in an ultra-scalable and highly reliable database system. Amazon DynamoDB, which is a new service, continues to build on these principles, and also builds on our years of experience with running non-relational databases and cloud services, such as Amazon SimpleDB and Amazon S3, at scale.

Fri, 13 Jan 2012 16:00:00 UTC

Countdown to What is Next in AWS

Posted By Werner Vogels

Join me at 9AM PST on Wednesday January 18, 2012 to find out what is next in the AWS Cloud. Registration required. Watch live streaming video from AWSCloudEvent at

Thu, 15 Dec 2011 03:00:00 UTC

Expanding the Cloud  Introducing the AWS South America (Sao Paulo) Region

Posted By Werner Vogels

Today, Amazon Web Services is expanding its worldwide coverage with the launch of a new AWS Region in Sao Paulo, Brazil. This new Region has been highly requested by companies worldwide, and it provides low-latency access to AWS services for those who target customers in South America. South America is one of the fastest growing economic regions in the world. In particular, South American IT-oriented companies are seeing very rapid growth. Case in point: over the past 10 years IT has risen to become 7% of the GDP in Brazil. With the launch of the South America (Sao Paolo) Region, AWS now provides companies large and small with infrastructure that allows them to get to market faster while reducing their costs which enables them to focus on delivering value, instead of wasting time on non-differentiating tasks.

Tue, 23 Aug 2011 02:40:58 UTC

Expanding the Cloud - Introducing Amazon ElastiCache

Posted By Werner Vogels

Today AWS has launched Amazon ElastiCache, a new service that makes it easy to add distributed in-memory caching to any application. Amazon ElastiCache handles the complexity of creating, scaling and managing an in-memory cache to free up brainpower for more differentiating activities. There are many success stories about the effectiveness of caching in many different scenarios; next to helping applications achieving fast and predictable performance, it often protects databases from requests bursts and brownouts under overload conditions. Systems that make extensive use of caching almost all report a significant reduction in the cost of their database tier. Given the widespread use of caching in many of the applications in the AWS Cloud, a caching service had been high on the request list of our customers.

Fri, 19 Aug 2011 19:40:58 UTC

Job Openings in AWS - Senior Leader in Database Services

Posted By Werner Vogels

There are some great job openings within Amazon Web Services. I will try to highlight some of those in coming weeks. This week it is an opening for senior leaders with AWS Database Services. AWS Database Services is responsible for setting the database strategy and delivering distributed structured storage services to our AWS customers. This team is constantly rethinking the assumptions behind how traditional databases were built and constantly working on building the right database architectures suited for the Cloud environment. The database services organization is looking for senior leaders who will be able to hire and lead a large software development team that is responsible for designing and running services that are at the cutting edge of distributed database technology that helps our customers to build scalable database-driven applications in the cloud and have a significant bottom-line impact on our business.

Thu, 18 Aug 2011 23:00:50 UTC

Driving down the cost of Big-Data analytics

Posted By Werner Vogels

The Amazon Elastic MapReduce (EMR) team announced today the ability to seamlessly use Amazon EC2 Spot Instances with their service, significantly driving down the cost of data analytics in the cloud. Many of our Big-Data customers already saw a big drop in their AWS bill last month when the cost of incoming bandwidth was dropped to $0.00. Now, given that historically customers using Spot Instances have seen cost saving up to 66% over On-Demand Instance prices, Amazon EMR customers are poised to achieve even greater cost savings. Analyzing vast amounts of data is critical for companies looking to incorporate customer insights into their business, including building recommendation engines or optimizing customer targeting.

Wed, 17 Aug 2011 18:40:58 UTC

No Server Required - Jekyll & Amazon S3

Posted By Werner Vogels

As some of you may remember I was pretty excited when Amazon Simple Storage Service (S3) released its website feature such that I could serve this weblog completely from S3. If you have a largely static site you can rely on the enormous power of S3 to make serving your content highly scalable and storing it extremely durable. Amazon S3 is much more than just storage; the network and distributed systems infrastructure to ensure that content can be served fast and at high rates without customers impacting each other, is amazing. Just dropping your website in an S3 bucket brings all that power to you.

Tue, 16 Aug 2011 20:46:33 UTC

Expanding the Cloud - The AWS GovCloud (US) Region

Posted By Werner Vogels

Today AWS announced the launch of the AWS GovCloud (US) Region. This new region, which is located on the West Coast of the US, helps US government agencies and contractors move more of their workloads to the cloud by implementing a number of US government-specific regulatory requirements. The concept of regions gives AWS customers control over the placement of their resources and services. Next to GovCloud (US) there are five general purpose regions; two in the US (one on the west coast and one on the east coast), one in the EU (in Ireland) and two in APAC (in Singapore and Tokyo).

Mon, 11 Jul 2011 22:22:56 UTC

Spot Instances - Increased Control

Posted By Werner Vogels

Today we announced the launch of an exciting new feature that will significantly increase your control over your Amazon EC2 Spot instances. With this change, we will improve the granularity of pricing information you receive by introducing a Spot Instance price per Availability Zone rather than a Spot Instance price per Region. Spot Instances enable you to bid on unused Amazon EC2 capacity. Customers whose bids exceed the Spot price gain access to the available Spot Instances and run as long as the bid exceeds the Spot Price. Spot Instances are ideal for use cases like web and data crawling, financial analysis, grid computing, media transcoding, scientific research, and batch processing.

Fri, 08 Jul 2011 04:40:58 UTC

Expanding the Cloud - AWS Import/Export Support for Amazon EBS

Posted By Werner Vogels

The AWS Import/Export team has announced today that they have expanded their functionality significantly by adding Import into Amazon EBS. AWS Import/Export transfers data off of storage devices using Amazon's high-speed internal network and bypassing the Internet. With this new functionality AWS Import/Export now supports importing data directly into Amazon EBS snapshots. Once loaded into an Amazon EBS snapshot, The customer can create a volume based on that snapshot and attach it to an Amazon EC2 instance, or they can share that snapshot with others. Amazon Import/Export is an important tool for customers to accelerate moving large amounts of data into the AWS storage systems.

Mon, 04 Jul 2011 06:57:36 UTC

APAC Summer Tour

Posted By Werner Vogels

I have just landed in Tokyo for what will be a month long tour visiting our customers in the Asia Pacific Region. Next to customer visits I will take part in a number of events organized by AWS and by our partners. This week in Japan there are three public events planned: July 4 - AWS HPC Night at Fuji Soft Hall in Akihabara. Next to a presentation by me about HPC on AWS, there is a panel with Japanese HPC experts moderated by Dr Kazuyuki Shudo of Titec. July 5 - Cloud IT Leadership seminar on Business Continuity in partnership with Deloitte.

Thu, 30 Jun 2011 00:55:57 UTC

Driving Bandwidth Cost Down for AWS Customers.

Posted By Werner Vogels

Often we think about innovation as going after new unchartered territories, but it is also important to innovate in those existing dimensions that will remain important for customers. For Amazon retail, some of those dimensions are low pricing, large catalog, fast shipping, and convenience. Every effort we put into improving these drives a flywheel that yields benefits both immediately and for the long-term. For example, when our retail customers contributed to create larger economies of scale for, we used the savings to lower pricing such that our customers could also benefit. In Amazon Web Services there are similar dimensions that are forever important to our customers; scale, reliability, security, performance, ease of use, and of course pricing.

Tue, 24 May 2011 20:26:56 UTC

New Route 53 and ELB features: IPv6, Zone Apex, WRR and more

Posted By Werner Vogels

An important contribution to the success of the Amazon Web Services is the willingness to listen closely to our customers and to use this feedback to drive the feature roadmap of a service. I am excited that today both the Route 53, the highly available and scalable DNS service, and the Elastic Load Balancing teams are releasing new functionality that has been frequently requested by their customers: Route 53 now GA: Route 53 is now Generally Available and will provide an availability SLA of 100%. See the Route 53 detail page for the exact definitions and how SLA violations will be handled.

Wed, 27 Apr 2011 15:51:51 UTC

The 2010 Shareholder Letter Focusses on Technology

Posted By Werner Vogels

In the 2010 Shareholder Letter Jeff Bezos writes about the unique technologies developed at over the years. Given that I have frquently written about many of these technologies on this blog I asked investor relations to be allowed to reprint it here. You can find the original here. To our shareowners: Random forests, naïve Bayesian estimators, RESTful services, gossip protocols, eventual consistency, data sharding, anti-entropy, Byzantine quorum, erasure coding, vector clocks ... walk into certain Amazon meetings, and you may momentarily think you've stumbled into a computer science lecture. Look inside a current textbook on software architecture, and you'll find few patterns that we don't apply at Amazon.

Fri, 01 Apr 2011 00:02:54 UTC

Mashing Up Science - The Mendeley API Binary Battle

Posted By Werner Vogels

Two years ago when I was first introduced to Mendeley I concluded that if they executed right they could "change the face of science". Mendeley is a free reference manager and academic social network that can help you organize your research, collaborate with others online, and discover the latest research. Turn the clock forward two years and Mendeley has now evolved into the world's largest crowdsourced research database, with 70 million documents, usage statistics and reader demographics, social tags, and related research recommendations. This database is available under a creative commons license. To see what innovation can happen when this data is freely available Mendeley has issued a challenge dubbed "The Mendeley API Binary Battle" - build an application that mashes up the Mendeley data and you can win $10,001!

Tue, 29 Mar 2011 05:50:52 UTC

Music to my Ears - Introducing Amazon Cloud Drive

Posted By Werner Vogels

Today announced new solutions to help customers manage their digital music collections. Amazon Cloud Drive and Amazon Cloud Player enable customers to securely and reliably store music in the cloud and play it on any Android phone, tablet, Mac or PC, wherever they are. As a big music fan with well over 100Gb in digital music I am particularly excited that I now have access to all my digital music anywhere I go. Order in the Chaos The number of digital objects in our lives is growing rapidly. What used to be only available in physical formats now often has digital equivalents and this digitalization is driving great new innovations.

Fri, 25 Mar 2011 01:31:18 UTC

Hacking with AWS at The Next Web Hackaton

Posted By Werner Vogels

Over the past years The Next Web Conference has become a premier conference on internet life and its technologies. I have been to the conference almost every year and it is getting better every time. Amsterdam is of course the ideal place for such a conference :-). The TNW team is doing a great job in getting an excellent program together that draws an audience from around the world, not just Europe, and there is an interesting mix of startups, enterprises, investors and media attending. The images from the 2008 TNW Conference have travelled around the world in my Animoto demo: This year TNW is showing that it is not just a conference for talkers but also for builders by organizing a massive Hackaton in the two days running up to the conference.

Wed, 02 Mar 2011 14:00:01 UTC

Expanding the Cloud - Introducing the AWS Asia Pacific (Tokyo) Region

Posted By Werner Vogels

Today Amazon Web Services is expanding its world-wide coverage with the launch of a new AWS Region located in Tokyo, Japan. Japanese companies and consumers have become used to low latency and high-speed networking available between their businesses, residences, and mobile devices. With the launch of the Asia Pacific (Tokyo) Region, companies can now leverage the AWS suite of infrastructure web services directly connected to Japanese networks. The advanced Asia Pacific network infrastructure also makes the AWS Tokyo Region a viable option for customers from South Korea. A well know application area that makes use of the advanced network infrastructure in Japan is social gaming, and it shouldn't be surprising that some of the early customers of the new AWS Tokyo Region are large gaming companies like Gumi and Zynga Japan.

Mon, 28 Feb 2011 07:28:14 UTC

From the Archives - Gapingvoid's Nobody Cares

Posted By Werner Vogels

While cleaning out the digital attic I ran into this drawing that Hugh MacLeod (aka "gapingvoid") made for me in reponse to a storm-in-a-teacup about blogging Amazon. As usual Hugh came straight to the heart of the matter :-) BTW Hugh's new book "Evil Plans, having Fun on the Road to World Domination" was released last week. You can order it here

Fri, 25 Feb 2011 08:00:01 UTC

Simplifying IT - Create Your Application with AWS CloudFormation

Posted By Werner Vogels

With the launch of AWS CloudFormation today another important step has been taken in making it easier for customers to deploy applications to the cloud. Often an application requires several infrastructure resources to be created and AWS CloudFormation helps customers create and manage these collections of AWS resources in a simple and predictable way. Using declarative Templates customers can create Stacks of resources ensuring that all resources have been created, in the right sequence and with the correct confirmation. Earlier this year I met with an ISV partner who transformed his on-premise ERP software into a software-as-a-service offering. They had taken the approach that they would not only be offering their software as a scalable multi-tenant product but also as a single tenant environment for customers that want to have their own isolated environment.

Thu, 24 Feb 2011 01:43:52 UTC

Free at Last - A Fully Self-Sustained Blog Running in Amazon S3

Posted By Werner Vogels

In a follow up to the last blog post I have removed the last two dependencies this blog had on running a server somewhere: comments are now served by Disqus and search is now handled by Bing. I should have moved to Disqus long time again as it is such rich discussion environment. It imported the commented from my Moveable Type server without a hitch. The choice for the search box from Bing was driven by that it was very easy to setup and it was free, where Google Site Search asked for $100/year. I'll evaluate search again in a few weeks, but for the moment this is good enough.

Thu, 17 Feb 2011 23:45:51 UTC

New AWS feature: Run your website from Amazon S3

Posted By Werner Vogels

Since a few days ago this weblog serves 100% of its content directly out of the Amazon Simple Storage Service (S3) without the need for a web server to be involved.  Because my blog is almost completely static content I wanted to run in this very simple configuration since the launch of Amazon S3. It would allow the blog to be powered by the incredible scale and reliability of Amazon S3 with a minimum of effort from my side. I know of several other customers who had asked for this greatly simplifying feature as well. I had held out implementing an alternative to my simple blog server that had been running at a traditional hosting site for many years until this preferred simple solution became available: today marks that day and I couldn't be happier about it.

Fri, 21 Jan 2011 19:51:45 UTC

It is not the critic who counts ...

Posted By Werner Vogels

When Steve Case was asked "how do you turn defeat in to failure" he gracefully quoted Teddy Roosevelt. It is a great speech that will probably inspire people forever. I believe the sentiment of that speech is even more important today. We are enjoying a tremendous rise in entrepreneurial activities and the many startups in technology and in the consumer internet are driving hard to innovate and get their product in the hands of customers at neck breaking speeds. At AWS we are proud to power many of these companies and in my personal interactions I see a whole new generation of technology leaders arising who will have impact for a long time to come.

Wed, 19 Jan 2011 08:00:01 UTC

AWS Elastic Beanstalk: A Quick and Simple Way into the Cloud

Posted By Werner Vogels

Flexibility is one of the key principles of Amazon Web Services - developers can select any programming language and software package, any operating system, any middleware and any database to build systems and applications that meet their requirements. Additionally customers are not restricted to AWS services; they can mix-and-match services from other providers to best meet their needs. A whole range of innovative new services, ranging from media conversion to geo-location-context services have been developed by our customers using this flexibility and are available in the AWS ecosystem. To enable this broad choice, the core of AWS is composed of building blocks which customers and partners can use to build any system or application in the way they see fit.

Wed, 12 Jan 2011 02:20:20 UTC

DROAM - Dreaming about Cheap Data Roaming

Posted By Werner Vogels

I frequently travel outside of the US. Often to Europe and increasingly to the Middle & Far East and Australia. The one thing that I have always struggled with during my travels are the data plans of the cell phone companies. They are complex and ridiculously expensive. For an internet road warrior they are a complete nightmare. One wireless company for example has an international plan that will charge you $25 per month for 50MB after which they will charge you $20 per MB. I frequently do more than 30MB a day, so this plan has no advantage for me. After many investigations I settled for T-Mobile which has an international Blackberry supplement that is $20/month for all BB mail traffic.

Fri, 10 Dec 2010 03:30:00 UTC

Big Just Got Bigger - 5 Terabyte Object Support in Amazon S3

Posted By Werner Vogels

Today, Amazon S3 announced a new breakthrough in supporting customers with large files by increasing the maximum supported object size from 5 gigabytes to 5 terabytes. This allows customers to store and reference a large file as a single object instead of smaller 'chunks'. When combined with the Amazon S3 Multipart Upload release, this dramatically improves how customers upload, store and share large files on Amazon S3. Who has files larger than 5GB? Amazon S3 has always been a scalable, durable and available data repository for almost any customer workload. However, as use of the cloud as grown, so have the file sizes customers want to store in Amazon S3 as objects.

Mon, 06 Dec 2010 06:00:01 UTC

Expanding the Cloud with DNS - Introducing Amazon Route 53

Posted By Werner Vogels

I am very excited that today we have launched Amazon Route 53, a high-performance and highly-available Domain Name System (DNS) service. DNS is one of the fundamental building blocks of internet applications and was high on the wish list of our customers for some time already. Route 53 has the business properties that you have come to expect from an AWS service: fully self-service and programmable, with transparent pay-as-you-go pricing and no minimum usage commitments. Some fundamentals on Naming Naming is one of the fundamental concepts in Distributed Systems. Entities in a system are identified through their name, which is separate from the way that you would choose to access that entity, the address that the access point resides at and what route to take to get to that address.

Fri, 19 Nov 2010 23:51:47 UTC

This week in review: GPUs, Zombies, Biomimicry and Tom Waits

Posted By Werner Vogels

Here are some the links I shared this week on twitter and facebook: Cloud Computing Big news this week was of course the launch of Cluster GPU instances for Amazon EC2. There were blog posts by Jeff Barr The Cluster GPU Instance and James Hamilton HPC in the Cloud with GPGPUs, as well as my background posting: Expanding the Cloud - Adding the Incredible Power of the Amazon EC2 Cluster GPU Instances The Cluster Compute and the Cluster GPU Instances are now supported by Amazon Elastic Map Reduce. There was an excellent first benchmarking report of the Cluster GPU Instances by the folks at Cycle Computing - "A Couple More Nails in the Coffin of the Private Compute Cluster" The Top500 supercomputer list for November 2010 was released and an Amazon EC2 Cluster Compute Instance based cluster came in at #231 The Amazon Web Services received ISO 27001 certification ...

Mon, 15 Nov 2010 08:00:01 UTC

Expanding the Cloud - Adding the Incredible Power of the Amazon EC2 Cluster GPU Instances

Posted By Werner Vogels

Today Amazon Web Services takes another step on the continuous innovation path by announcing a new Amazon EC2 instance type: The Cluster GPU Instance. Based on the Cluster Compute instance type, the Cluster GPU instance adds two NVIDIA Telsa M2050 GPUs offering GPU-based computational power of over one TeraFLOPS per instance. This incredible power is available for anyone to use in the usual pay-as-you-go model, removing the investment barrier that has kept many organizations from adopting GPUs for their workloads even though they knew there would be significant performance benefit. From financial processing and traditional oil & gas exploration HPC applications to integrating complex 3D graphics into online and mobile applications, the applications of GPU processing appear to be limitless.

Thu, 30 Sep 2010 19:27:29 UTC

Around the World in 28 Days

Posted By Werner Vogels

On Monday I will leave Seattle for 4 weeks of meeting existing and future customers of the Amazon Web Services. With existing customers I get a change to dive deep on their AWS usage and understand what works well and where we can do better. Visiting future customers is equally exiting as you get a change to understand their current architecture, if it is a migration, and how they plan to exploit cloud services in their new setup. There is huge variety in exiting architectures and I am often impressed about the ingenuity of the engineers in how to best transform the application if "Lift & Shift" is not an option.

Wed, 29 Sep 2010 22:50:44 UTC


Posted By Werner Vogels

Like many folks who started down the path of using real-time micro-blogging services (read: twitter) the convenience of those platforms has made that sharing happens there instead of on the blogs we (used to) keep. I fully intended to stay committed to using this blog for high-quality, relevant sharing of thoughts but the reality has turned out to be different. I did write for those occasions where I thought additional background on AWS service and feature launches was relevant, but beyond that not much materialized. That was a mistake which I will try to correct in the coming months as I'll get back into blogging mode.

Tue, 13 Jul 2010 08:00:01 UTC

Expanding the Cloud - Cluster Compute Instances for Amazon EC2

Posted By Werner Vogels

Today, Amazon Web Services took very an important step in unlocking the advantages of cloud computing for a very important application area. Cluster Computer Instances for Amazon EC2 are a new instance type specifically designed for High Performance Computing applications. Customers with complex computational workloads such as tightly coupled, parallel processes, or with applications that are very sensitive to network performance, can now achieve the same high compute and networking performance provided by custom-built infrastructure while benefiting from the elasticity, flexibility and cost advantages of Amazon EC2 During my academic career, I spent many years working on HPC technologies such as user-level networking interfaces, large scale high-speed interconnects, HPC software stacks, etc.

Wed, 19 May 2010 07:00:01 UTC

Expanding the Cloud - Amazon S3 Reduced Redundancy Storage

Posted By Werner Vogels

Today a new storage option for Amazon S3 has been launched: Amazon S3 Reduced Redundancy Storage (RRS). This new storage option enables customers to reduce their costs by storing non-critical, reproducible data at lower levels of redundancy. This has been an option that customers have been asking us about for some time so we are really pleased to be able to offer this alternative storage option now. Durability in Amazon S3 Amazon Simple Storage Service (S3) was launched in 2006 as "Storage for the Internet" with the promise to make web-scale computing easier for developers. Four years later it stores over 100 billion objects and routinely performs well over 120,000 storage operations per second.

Thu, 29 Apr 2010 02:00:01 UTC

Expanding the Cloud - Opening the AWS Asia Pacific (Singapore) Region

Posted By Werner Vogels

Today Amazon Web Services has taken another important step in serving customers worldwide: the AWS Asia Pacific (Singapore) Region is now launched. Customers can now store their data and run their applications from our Singapore location in the same way they do from our other U.S. and European Regions. The importance of Regions Quite often "The Cloud" is portrayed as something magically transparent that lives somewhere in the internet. This portrayal can be a desirable and useful abstraction when discussing cloud services at the application and end-user level. However, when speaking about cloud services in terms of Infrastructure-as-a-Service, it is very important to make the geographic locations of services more explicit.

Fri, 23 Apr 2010 08:17:28 UTC

I am looking for new application and platform services

Posted By Werner Vogels

The ecosystem of new application and platform services in the cloud is the future of application development. It will drive rapid innovation and we'll see a wealth of mobile, web and desktop applications arrive that we couldn't dream about a few years ago, and these building blocks are the enablers of that. These services will be delivered not only by new startups but also by enterprises looking to capitalize on their IP. As examples of such services I always use Twillio (voice &sms) and Simplegeo (location), but it is time to start building out my knowledge of all the different services that are in the ecosystem.

Mon, 19 Apr 2010 18:02:41 UTC

Amazon Cloudfront is Streaming Media 2010 Editor's pick

Posted By Werner Vogels

I am excited that Amazon Cloudfront has been selected as one of the 10 Editor's pick of 2010 by Streaming Media. Amazon Cloudfront is the Content Delivery Network (CDN) that is dead simple to use both from a technology and a business point of view. From a technology point it literally takes a single button click in the console or a single API call to have your content CDN enabled. You can find several websites marveling about how simple it is to CDN enable your content with amazon Cloudfront. On the business side Amazon Cloudfront revolutionized the CDN business by providing a true pay-as-you-go service, no longer requiring upfront commitments that are commonplace in the CDN business and which opens up CDN for everyone to use.

Thu, 01 Apr 2010 08:00:01 UTC

AWS Import/Export launches support for Legacy Storage Systems

Posted By Werner Vogels

Today Amazon Web Services takes another big step in making it easier to migrate legacy storage systems to the cloud through AWS Import/Export support for ingesting Punch Cards. AWS Import/Export accelerates moving large amounts of data into and out of AWS using portable storage media for transport. Punch cards are paper-based storage media that represent data using the presence or absence of holes in specific positions. With AWS Import/Export for Punch Cards, enterprises can begin using the service to preserve and unlock the large volumes of data that have accumulated over the last century on what was the first broadly adopted digital storage medium. "

Wed, 24 Feb 2010 23:00:00 UTC

Choosing Consistency

Posted By Werner Vogels

Amazon SimpleDB has launched today with a new set of features giving the customer more control over which consistency and concurrency models to use in their database operations. There are now conditional put and delete operations as well a new "consistent read" option. These new features will make it easier to transition those applications to SimpleDB that are designed with traditional database tools in mind. Revisiting the Consistency Challenges Architecting distributed systems that need to reliably operate at world-wide scale is not a simple task. There are many factors that come into play when you need to meet stringent availability and performance requirements under ultra-scalable conditions.

Mon, 14 Dec 2009 07:00:01 UTC

Expanding the Cloud - Amazon EC2 Spot Instances

Posted By Werner Vogels

Today we launched a new option for acquiring Amazon EC2 Compute resources: Spot Instances. Using this option, customers bid any price they like on unused Amazon EC2 capacity and run those instances for as long their bid exceeds the current "Spot Price." Spot Instances are ideal for tasks that can be flexible as to when they start and stop. This gives our customers an exciting new approach to IT cost management. The central concept in this new option is that of the Spot Price, which we determine based on current supply and demand and will fluctuate periodically. If the maximum price a customer has bid exceeds the current Spot Price then their instances will be run, priced at the current Spot Price.

Thu, 03 Dec 2009 09:00:25 UTC

Powerful New Amazon EC2 Boot Features

Posted By Werner Vogels

Today a powerful new feature is available for our Amazon EC2 customers: the ability to boot their instances from Amazon EBS (Elastic Block Store). Customers like the simplicity of the AMI (Amazon Machine Image) model where they either choose a preconfigured AMI or upload their own AMI into Amazon S3. A wide variety of operating systems and software configurations is available for use. But customers have also asked us for more flexibility and control in the way that Amazon EC2 instances are booted such that they have finer grained control over for example what software configurations and data sets are available to the instance at boot time.

Thu, 03 Dec 2009 09:00:23 UTC

Expanding the Cloud - New AWS Region: US-West (Northern California)

Posted By Werner Vogels

We have expanded the AWS footprint in the US and starting today a new AWS Region is available for use: US-West (Northern California). This new Region consists of multiple Availability Zones and provides low-latency access to the AWS services from for example the Bay Area. In the US, AWS customers now can choose between the US-East (Northern Virginia) Region and the new US-West (Northern California) Region. In addition, the EU (Ireland) Region is available to customers who want local access to services from Europe to address their performance or jurisdiction requirements. As we announced earlier this month a Region with multiple Availability Zones will come online in Singapore in the first half of 2010, with other regions in Asia to follow later in 2010.

Tue, 10 Nov 2009 01:27:33 UTC

82 Billion Objects in Amazon S3

Posted By Werner Vogels

At the end of Q3 2009 we counted over 82 billion objects in Amazon S3. Congrats to the team for providing such a rock solid service! When looking at the graph keep in mind that the first 4 markers are a year apart, but the last one only 6 months.

Tue, 27 Oct 2009 08:00:01 UTC

Expanding the Cloud: The Amazon Relational Database Service (RDS)

Posted By Werner Vogels

Today marks the launch of Amazon RDS - the Amazon Relational Database Service. Amazon RDS is a web service that makes it easy to set up, operate, and scale a relational database in the cloud. Amazon RDS handles all the "muck" of relational database management freeing up its users to focus on their applications and business. Fine Tuning Data Management At Amazon we have a long history of fine tuning our data management solutions to make sure that our systems can be reliable and cost-effective as we continue to scale. Almost from the beginning of operating the Amazon ecommerce platform it was clear that its scalability, reliability, performance, and cost-effectiveness were all dependent on the way that data was managed.

Tue, 15 Sep 2009 18:01:30 UTC

Expanding the Cloud: Amazon Web Services to support the Federal Government

Posted By Werner Vogels

In the past week both Vivek Kundra, the U.S. CIO, and Casey Coleman, the CIO of the GSA, have made very strong statements in supporting the use of cloud computing to power Federal programs. A good example is today's announcement about In conversations with Vivek and Casey, I am struck every time by how much their observations that Federal CIOs are focused too much on infrastructure issues are similar to the observations within Amazon a number of years ago that motivated us to develop the AWS Infrastructure services. At that time, Amazon engineering teams focused more than 70% of their work effort on keeping their infrastructure efficient, scalable and reliable, which were important, but non-differentiating tasks.

Wed, 26 Aug 2009 07:00:10 UTC

Seamlessly Extending the Data Center - Introducing Amazon Virtual Private Cloud

Posted By Werner Vogels

At this 3rd anniversary of the launch of Amazon Elastic Compute Cloud (Amazon EC2), it is amazing to see the impact this service has had on the industry. It is truly disruptive technology and its impact has reached far beyond a pure technology offering as the benefits of the cloud have changed the way we view IT Infrastructure. As one of the CIOs at the ACM Cloud Computing Roundtable summarized it: "IT used to be the blocker in anything we did, but with our shift to the cloud IT is now the enabler." From young businesses and established enterprises to hospitals and governments agencies, all are equally enthusiastic cloud customers for whom IT infrastructure has changed forever.

Thu, 13 Aug 2009 23:43:50 UTC

Feedback for Amazon Web Services

Posted By Werner Vogels

Ingrained in the DNA of Amazon Technologist is a single-minded focus on the needs of our customers. The Amazon development process is even called "Working from the customer backwards". Essential in this process is a good understanding of what the customers need in terms of new services, new features for existing services, or different approaches to things that we are already doing. We collect this feedback continuously from various sources: the AWS forums, the AWS Premium Support Team, Amazonians on the road talking to customers, solution architects helping to define customer architectures, ISV partners building on our services, system integration partners who relay customer needs, advisory boards, and of course the Amazon ecommerce engineers building on the AWS platform.

Tue, 14 Jul 2009 05:20:04 UTC

Amazon is in-world and hiring!

Posted By Werner Vogels

Join recruiters and hiring managers from several of Amazon's global offices on July 14, 2009. We'll be in-world from 6am through midnight (Pacific/Seattle time) for the first ever Amazon Second Life Job Fair. This free event is a unique opportunity for candidates to have direct access to hiring managers and recruiters from around the world! We are looking across all levels of technical and non-technical profiles - from hands-on engineers to program managers and game-changing principal architects. Visit our U.S. career site at for open U.S. positions and links to our global careers pages, then join us in-world at on July 14.

Thu, 21 May 2009 05:00:00 UTC

Expanding the Cloud: Moving large data sets into Amazon S3 with AWS Import/Export.

Posted By Werner Vogels

Before networks were everywhere the easiest way to transport information from one computer in your machine room was to write the data to a floppy disk, run to the computer and load the data there from that floppy. This form of data transport was jokingly called "sneaker net". It was efficient because networks only had limited bandwidth and you wanted to reserve that for essential tasks. In some ways the computing world has changed dramatically; networks have become ubiquitous and the latency and bandwidth capabilities have improved immensely. Next to this growth in network capabilities we have been able to grow something else to even bigger proportions, namely our datasets.

Mon, 18 May 2009 08:01:25 UTC

Automating the management of Amazon EC2 using Amazon CloudWatch, Auto Scaling and Elastic Load Balancing

Posted By Werner Vogels

The Amazon Elastic Compute Cloud (Amazon EC2) embodies much of what makes infrastructure as a service such a powerful technology; it enables our customers to build secure, fault-tolerant applications that can scale up and down with demand, at low cost. Core in achieving these levels of efficiency and fault-tolerance is the ability to acquire and release compute resources in a matter of minutes, and in different Availability Zones. Of course the best way to achieve efficiency and fault-tolerance while maintaining good performance is to fully automate the management of the Amazon EC2 Instances, such that you can optimize the use of the compute resources in different scenarios.

Fri, 10 Apr 2009 21:10:32 UTC

Making A Dramatic Difference

Posted By Werner Vogels

As some of you may know both my daughters are studying Drama in London. Last time when I visited them I met two friends of Kim, Georgia Munnion and Lauren Hopkins. They are all classmates and they are graduating this year. Georgia and Lauren impressed me with a plan they have for spending the two months after their graduation in Nepal providing educational Drama Workshops to Himalayan children. We will use Drama as a basis for a process of individual social development, and to improve individual, group and team-building skills, through a range of individual and ensemble exercises. Using our acquired theatre knowledge and experience, we will provide education and enjoyment to underprivileged children, who are mostly unable to return home during their holidays, due to snow-bound and monsoon-bound trails and the recent outbreak of war.

Wed, 25 Mar 2009 22:58:56 UTC

Good Advice on Keeping Your Database Simple and Fast.

Posted By Werner Vogels

Keeping your database simple and fast is often difficult if you use higher level frameworks such as ActiveRecords in Ruby or Java object persistence technologies such as Hibernate. There is a lot of magic that is happening out of sight that you have no control over. If you then have to scale your application it is often the relational database that these technologies require that becomes the performance and scaling bottleneck. Often requiring complex custom implementations of partitioning and sharding to make it work. The AWS services Amazon S3 and Amazon SimpleDB were designed to handle the dominant storage usage patterns within Amazon and they greatly reduced our need to rely on relational storage for scaling our systems.

Thu, 12 Mar 2009 08:00:01 UTC

Introducing Amazon EC2 Reserved Instances - A way to further reduce IT costs.

Posted By Werner Vogels

Flexibility is a key advantage of using Amazon Web Services; you can obtain resources instantaneously without the headache of owning them. If you no longer need the resource, you release it and only pay for what you have used. This is a very powerful model that has helped many of our customers drive capital expense out of their IT operation. It has helped both enterprises and startups reduce the risk that comes with developing new products and businesses. While this on-demand flexibility is ideal for a whole range of scenarios, some Amazon EC2 customers who have more predictable workloads have asked us for even greater flexibility in the cost model through the ability to reserve capacity.

Tue, 03 Mar 2009 09:16:07 UTC

Expanding the Cloud: Expanding Amazon EC2 for Windows

Posted By Werner Vogels

Today we have some important news for our Amazon EC2 customers who are running Windows Server and Windows SQLServer instances and who have been looking to extend their coverage for fault-tolerance and locality reasons. Starting today Windows instances can be launched in an additional Availability Zone is the US and they can also be launched in two Availability Zones in Europe. This allows developers who use our Windows instances to build solutions that can tolerate various failure and recovery scenarios. It also puts Windows Server into the hands of developers who want low latency for their European customers. We have also integrated these features into the Amazon AWS console, such that you now can use the console to launch instances in any of our regions, regardless whether it is Europe or the US.

Tue, 23 Dec 2008 08:15:07 UTC

Eventually Consistent - Revisited

Posted By Werner Vogels

I wrote a first version of this posting on consistency models about a year ago, but I was never happy with it as it was written in haste and the topic is important enough to receive a more thorough treatment. ACM Queue asked me to revise it for use in their magazine and I took the opportunity to improve the article. This is that new version. Eventually Consistent - Building reliable distributed systems at a worldwide scale demands trade-offs between consistency and availability. At the foundation of Amazon's cloud computing are infrastructure services such as Amazon's S3 (Simple Storage Service), SimpleDB, and EC2 (Elastic Compute Cloud) that provide the resources for constructing Internet-scale computing platforms and a great variety of applications.

Sat, 20 Dec 2008 18:30:17 UTC


Posted By Werner Vogels

A question I get asked frequently is how working in industry is different from working in academia. My answer from the beginning has been that the main difference is teamwork. While in academia there are collaborations among faculty and there are student teams working together, the work is still rather individual, as is the reward structure. In industry you cannot get anything done without teamwork. Products do not get build by individuals but by teams; definition, implementation, delivery and operation are all collaborative processes that have many people from many different disciplines working together. As such the Information Week's Chief of the Year award cannot be my award.

Wed, 10 Dec 2008 06:00:19 UTC

Expanding the Cloud: Amazon EC2 in Europe

Posted By Werner Vogels

Starting today the Amazon Elastic Computing Cloud (EC2) supports the ability to launch instances in multiple geographically distinct regions. The new EU region enables users to launch instances in Europe. This addresses the requests from many our European customers and from companies that want to run instances closer to European customers. Over the past year I have visited with many of our European customers and frequently they remarked "if only we had EC2 in Europe". We heard their requests load and clear and have worked very hard to roll out the European Region. This is a very important milestone on the road to local access to all our services.

Tue, 18 Nov 2008 06:00:05 UTC

Expanding the Cloud: Amazon CloudFront

Posted By Werner Vogels

Today marks the launch of Amazon CloudFront, the new Amazon Web Service for content delivery. It integrates seamlessly with Amazon S3 to provide low-latency distribution of content with high data transfer speeds through a world-wide network of edge locations. It requires no upfront commitments and is a pay-as-you-go service in the same style as the other Amazon Web Services. Amazon CloudFront has been designed to be fast; the service will cache copies of the content in edge locations close to the end-user's location, significantly lowering the access latency to the content. High sustainable data transfer rates can be achieved with the service especially when distributing larger objects.

Thu, 23 Oct 2008 13:31:00 UTC

Using the Cloud to build highly-efficient systems

Posted By Werner Vogels

These are times where many companies are focusing on the basics of their IT operations and are asking themselves how they can operate more efficiently to make sure that every dollar is spent wisely. This is not the first time that we have gone through this cycle, but this time there are tools available to CIOs and CTOs that help them to manage their IT budgets very differently. By using infrastructure as a service, basic IT costs are moved from a capital expense to a variable cost, building clearer relationships between expenditures and revenue generating activities. CFOs are especially excited about the premise of this shift.

Thu, 23 Oct 2008 13:30:00 UTC

Amazon EC2 in Full Production

Posted By Werner Vogels

Congratulations to the Amazon EC2 team for the hard work to get to the point where the beta tag is removed from the service and it is now in full production. Not only that, but there now is an SLA, and Microsoft Windows and SQL Server are available as of today. More details on the Amazon EC2 product page and on the Amazon Web Services weblog.

Wed, 01 Oct 2008 07:30:00 UTC

Expanding the Cloud: Microsoft Windows Server on Amazon EC2

Posted By Werner Vogels

The backend servers that power the world of Internet Services have become increasingly diverse. With today's announcement that Microsoft Windows Server is available on Amazon EC2 we can now run the majority of popular software systems in the cloud. Windows Server ranked very high on the list of requests by customers so we are happy that we will be able to provide this. One particular area that customers have been asking for Amazon EC2 with Windows Server was for Windows Media transcoding and streaming. There is a range of excellent codecs available for Windows Media and there is a large amount of legacy content in those formats.

Mon, 29 Sep 2008 12:42:21 UTC

AWS Startup Challenge 2008

Posted By Werner Vogels

The last week for submitting the applications for the AWS Startup Challenge has started. Looking at the proposals that are being submitted it looks like this will be another very inspiring challenge. These proposals are reviewed by a panel and five finalists will be selected. The finalists will come to Seattle to compete for $50K in cash, $50K in AWS credits, 2 years of Premium Support and more. All finalists will receive Rightscale Premium for 6 months and there will be a number of promotional events that includes all the finalists. Last year there were 900 applications which made for very intense proposal reading sessions.

Thu, 18 Sep 2008 10:00:00 UTC

Expanding the Cloud

Posted By Werner Vogels

For many the "Cloud" in Cloud Computing signifies the notion of location independence; that somewhere in the internet services are provided and that to access them you do not need any specific knowledge of where they are located. Many applications have already been built using cloud services and they indeed achieve this location transparency; their customers do not have to worry about where and how the application is being served. However for developers to do their job properly the cloud cannot be fully transparent. As much as we would like to make it easy and simple for everyone, building high-performance and highly reliable applications in the cloud requires that the developers have more control.

Thu, 21 Aug 2008 06:50:00 UTC

Amazon EBS - Elastic Block Store has launched

Posted By Werner Vogels

Today marks the launch of Amazon EBS (Elastic Block Store), the long awaited persistent storage service for EC2. Details can be found on the EC2 detail page, the press release and Jeff Barr's posting over on the AWS evangelists blog. Also the folks at Rightscale have two detailed postings: why Amazon EBS matters and Amazon EBS explained. With the launch of the Elastic Block Store we complete an important milestone in offering a complete suite of storage solutions as part of the Amazon Infrastructure Services. Back in the days when we made the architectural decision to virtualize the internal Amazon infrastructure one of the first steps we took was a deep analysis of the way that storage was used by the internal Amazon services.

Sat, 26 Jul 2008 01:51:25 UTC

Root Cause

Posted By Werner Vogels

For those of you interested in the details of last Sunday's Amazon S3 Availability issue you should read the detailed explanation posted at the AWS Status Dashboard. Root cause was single bit corruption of internal state messages that are distributed via Gossip techniques.

Fri, 18 Jul 2008 07:28:19 UTC

An Album for Each Year

Posted By Werner Vogels

What better way to pick up posting again than with following a meme. Nick Carr in Albums Going Steady describes the challenge to list "a favorite album for every year of your life." I actually do not have the problem described by Nick and others to really start with my birth year. The challenge has two restrictions: only one album per year and there can be no repeats of artists. I have added for myself the restriction that I should actually own the album, which restricts the set to choose from significantly and also makes for some peculiar choices. Here is my list 1958: Jerry Lee Lewis, Great Balls of Fire 1959: Ray Charles, What I'd Say 1960: Miles Davis, Sketches of Spain 1961: Robert Johnson, King of the Delta Blues Singers 1962: Booker T & MG, Green Onions 1963: James Brown, Live at the Apollo 1964: John Coltrane, ...

Wed, 30 Apr 2008 05:23:11 UTC

The Perfect Laptop - Unboxing the X300

Posted By Werner Vogels

The laptop that appeared on the cover of business week as part of the story “Building the Perfect Laptop” is the Thinkpad X300. It arrived at my doorstep this afternoon. It is everything it promised to be and more; superlight, rugged, SSD, full ports, wifi, lan & cell networks, dvd, replaceable batteries and 13.3" screen with 1440x900 graphics. And all of this weighs in at 1420 grams. It is amazing how light it is for a full featured laptop. See the unboxing pictures on flickr.

Tue, 15 Apr 2008 07:23:52 UTC

Ahead in the Cloud

Posted By Werner Vogels

My opening slide for tomorrow's keynote at the MySQL Conference has this feel of speed and excitement to it that represents the current progress towards Cloud Computing. Persistent Storage for EC2 will be an important part of the presentation, but I'll mainly focus on general non-functional lesson from building large-scale services.

Mon, 14 Apr 2008 05:00:00 UTC

Persistent Storage for Amazon EC2

Posted By Werner Vogels

I would like to introduce to you the newest feature of Amazon EC2: Persistent local storage. This has been very high on the request list of EC2 customers and I believe that combined with the Availability Zones and Elastic IP Address features released earlier this month this makes EC2 the ideal environment for building highly scalable and reliable applications. Significant innovation has gone into this feature: Instead of restricting developers to the use of a particular (distributed) file-system we once again decided to look at what is the most fundamental building block and how we could offer that in the most scalable and reliable manner.

Thu, 27 Mar 2008 08:20:21 UTC

On the Road to Highly Available EC2 Applications

Posted By Werner Vogels

Today Amazon Web Services launched two new features in Amazon EC2 that are essential tools in building highly resilient applications: Elastic IP addresses and Availability Zones. In summary: Elastic IP addresses are associated with a customer account and allow the customer to do its own dynamic mapping of IP address to instance. Using this dynamic mapping applications can remain reachable even in the presence of failures. This is an area where for example DNS reconfiguration is too slow a technique. Availability Zones allow the customer to specify in which location to launch a new EC2 instance. The world is divided up into Regions and a Region can hold multiple Availability Zones.

Tue, 25 Mar 2008 05:42:22 UTC

The Next Web Event

Posted By Werner Vogels

While the past months have been relatively quiet there is now a period coming up with public events that will take me across a few continents. The period already started two weeks with a Distinguished Lecture at the School of Computer Science of CMU. I had a wonderful day meeting many academics to discuss the relevance of particular research subjects for companies such as Amazon. I believe that Peter Lee, the Head of Computer Science, is the only Head/Chair/Dean of a Computer Science department who maintains a weblog. It is worth reading for a different perspective. Last week there was a very different event: A fireside chat at the close of the Under the Radar conference.

Wed, 19 Mar 2008 08:11:00 UTC

Happy Birthday, Amazon S3!

Posted By Werner Vogels

A few days ago, on March 14, Amazon S3 quietly celebrated its 2nd birthday. I think congratulations are in order and I certainly wish the service "many happy returns of this day". That S3 is growing up fast is obvious from the number of objects the customers trust us with.