如果你打开一本现在关于软件设计的书，你会发现几乎没有什么模式我们不在Amazon使用。我们使用high-performance transactions systems（高性能交易系统）, complex rendering and object caching（复杂渲染和对象缓存）, workflow and queuing systems（流程和排队系统）, business intelligence and data analysis（商业智能和数据分析）, matching learning and pattern recognition（机器学习和模式识别）, neural networks and probabilistic decision making（神经网络和基于概率的决策）, 以及其他一大批其他技术。尽管我们非常多的系统都已经是基于最新的研究成果了，但这依然不够，我们的架构师和工程师还不得不在那些学术届还没有成果的领域探索。我们遇到的很多问题还没有标准解法，所以我们会开心的研究新方法。
状态管理对于任何一个会变得非常大的系统都是至关重要的。多年以前，Amazon的需求到达了一个临界点，我们的系统大到当时已经没有任何一家提供技术服务的公司可以搞定了——我们核心的数据服务存储了好几PB的信息（1PB=1000TB），需要每秒处理几百万次请求。要能够满足这种近乎变态的需求，我们开发了几套有持久又具有针对性的替代方案，包括我们自己的Key-value store（译者注：数据库中的专业术语）和single table store。为了做到这些，我们大量利用了分布式系统和数据库研究的核心原理，并且从这些地方开始自己的创新。我们业界领先的存储系统在提供了特别好的延展性（译者注：此处指可以存储海量数据）的同时，还能够保持对性能、稳定性和成本的高度控制。为了达到超强的延展性，我们采取了一个新手段来做数据更新管理：通过放松需要被大量复制传播的更新的同步性需求，让这些系统依然能够在严酷的性能和可供性要求下运行。这些系统是基于最终一致性的概念而设计的。正是这些由亚马逊工程师设计开发的数据管理技术，衍生出了AWS提供的云存储和数据管理服务。比如我们的Simple Storage Service, Elastic Block Store, 和SimpleDB都是从亚马逊的技术中衍生出来的。
Amazon的其他业务也面临着类似的复杂数据处理和决策问题，比如产品数据摄取和分类、需求预测、库存分配和诈骗监测。“书呆子系统（Rule-based systems）”可以用，但是它们很难被维护并且时候久了就会变得脆弱。在很多情况下，先进的机器学习技术提供了更精准的分类，并且能对变化的环境进行自适应。比如，我们的搜索引擎就是用数据挖掘和机器学习的算法，这些算法在后台运行来根据关键词创建模型，然后我们用信息提取算法来识别特征，并从未被结构化的商品介绍中提取相关内容，为用户精准找出符合它们需求的商品。我们为了能够准确预测用户兴趣并最优化搜索结果的排序，考虑了搜索相关性里非常多维度的因素。亚马逊产品的多样性决定我们必须采取现代回归技术，比如经过随机森林算法训练过的决策树（trained random forests of decision trees），来在形成搜索结果排序时，灵活地合并数千条产品。这些幕后算法的工作结果是什么？精准的搜索结果，帮你找到你想要的那件商品！
举个例子，Whispersync功能。这是一个Kindle的服务，它确保你无论在任何地方，任何设备上，你都可以访问你的图书馆以及你所有的高亮、笔记和书签。这个技术的难点在于同时在几百万kindle用户上实现，其中涉及了数亿本书，数百种设备类型，在全球一百多个国家——并且还得保证24×7的可靠性。在Whispersync技术的核心是“最终一致性复制数据存储（an eventually consistent replicated data store）”，其中有应用定义的冲突解决机制，能够处理几个星期以上不同步的设备。当然了，对于一个kindle用户，我们把这些技术过程都隐藏了。所以你只需要知道，当你打开Kindle的时候，所有的阅读进度都同步好了！借用Arthur C. Clarke的一句话，所有足够成熟的技术都和魔术一样。
Jeffrey P. Bezos
To our shareowners:
Random forests, naïve Bayesian estimators, RESTful services, gossip protocols, eventual consistency, data sharding, anti-entropy, Byzantine quorum, erasure coding, vector clocks … walk into certain Amazon meetings, and you may momentarily think you’ve stumbled into a computer science lecture.
Look inside a current textbook on software architecture, and you’ll find few patterns that we don’t apply at Amazon. We use high-performance transactions systems, complex rendering and object caching, workflow and queuing systems, business intelligence and data analytics, machine learning and pattern recognition, neural networks and probabilistic decision making, and a wide variety of other techniques. And while many of our systems are based on the latest in computer science research, this often hasn’t been sufficient: our architects and engineers have had to advance research in directions that no academic had yet taken. Many of the problems we face have no textbook solutions, and so we -- happily -- invent new approaches.
Our technologies are almost exclusively implemented as services: bits of logic that encapsulate the data they operate on and provide hardened interfaces as the only way to access their functionality. This approach reduces side effects and allows services to evolve at their own pace without impacting the other components of the overall system. Service-oriented architecture -- or SOA -- is the fundamental building abstraction for Amazon technologies. Thanks to a thoughtful and far-sighted team of engineers and architects, this approach was applied at Amazon long before SOA became a buzzword in the industry. Our e-commerce platform is composed of a federation of hundreds of software services that work in concert to deliver functionality ranging from recommendations to order fulfillment to inventory tracking. For example, to construct a product detail page for a customer visiting Amazon.com, our software calls on between 200 and 300 services to present a highly personalized experience for that customer.
State management is the heart of any system that needs to grow to very large size. Many years ago, Amazon’s requirements reached a point where many of our systems could no longer be served by any commercial solution: our key data services store many petabytes of data and handle millions of requests per second. To meet these demanding and unusual requirements, we’ve developed several alternative, purpose-built persistence solutions, including our own key-value store and single table store. To do so, we’ve leaned heavily on the core principles from the distributed systems and database research communities and invented from there. The storage systems we’ve pioneered demonstrate extreme scalability while maintaining tight control over performance, availability, and cost. To achieve their ultra-scale properties these systems take a novel approach to data update management: by relaxing the synchronization requirements of updates that need to be disseminated to large numbers of replicas, these systems are able to survive under the harshest performance and availability conditions. These implementations are based on the concept of eventual consistency. The advances in data management developed by Amazon engineers have been the starting point for the architectures underneath the cloud storage and data management services offered by Amazon Web Services (AWS). For example, our Simple Storage Service, Elastic Block Store, and SimpleDB all derive their basic architecture from unique Amazon technologies.
Other areas of Amazon’s business face similarly complex data processing and decision problems, such as product data ingestion and categorization, demand forecasting, inventory allocation, and fraud detection. Rule based systems can be used successfully, but they can be hard to maintain and can become brittle over time. In many cases, advanced machine learning techniques provide more accurate classification and can self-heal to adapt to changing conditions. For example, our search engine employs data mining and machine learning algorithms that run in the background to build topic models, and we apply information extraction algorithms to identify attributes and extract entities from unstructured descriptions, allowing customers to narrow their searches and quickly find the desired product. We consider a large number of factors in search relevance to
predict the probability of a customer’s interest and optimize the ranking of results. The diversity of products demands that we employ modern regression techniques like trained random forests of decision trees to flexibly incorporate thousands of product attributes at rank time. The end result of all this behind-the-scenes software? Fast, accurate search results that help you find what you want.
All the effort we put into technology might not matter that much if we kept technology off to the side in some sort of R&D department, but we don’t take that approach. Technology infuses all of our teams, all of our processes, our decision-making, and our approach to innovation in each of our businesses. It is deeply integrated into everything we do.
One example is Whispersync, our Kindle service designed to ensure that everywhere you go, no matter what devices you have with you, you can access your reading library and all of your highlights, notes, and bookmarks, all in sync across your Kindle devices and mobile apps. The technical challenge is making this a reality for millions of Kindle owners, with hundreds of millions of books, and hundreds of device types, living in over 100 countries around the world—at 24x7 reliability. At the heart of Whispersync is an eventually consistent replicated data store, with application defined conflict resolution that must and can deal with device isolation lasting weeks or longer. As a Kindle customer, of course, we hide all this technology from you. So when you open your Kindle, it’s in sync and on the right page. To paraphrase Arthur C. Clarke, like any sufficiently advanced technology, it’s indistinguishable from magic.
Now, if the eyes of some shareowners dutifully reading this letter are by this point glazing over, I will awaken you by pointing out that, in my opinion, these techniques are not idly pursued – they lead directly to free cash flow.
We live in an era of extraordinary increases in available bandwidth, disk space, and processing power, all of which continue to get cheap fast. We have on our team some of the most sophisticated technologists in the world – helping to solve challenges that are right on the edge of what’s possible today. As I’ve discussed many times before, we have unshakeable conviction that the long-term interests of shareowners are perfectly aligned with the interests of customers.
And we like it that way. Invention is in our DNA and technology is the fundamental tool we wield to evolve and improve every aspect of the experience we provide our customers. We still have a lot to learn, and I expect and hope we’ll continue to have so much fun learning it. I take great pride in being part of this team.
As always, I attach a copy of our original 1997 letter. Our approach remains the same, and it’s still Day 1.
Jeffrey P. Bezos
Founder and Chief Executive Officer