,Chapter 1. The Web As a Platform for Building Distributed
Systems
THE WEB HAS RADICALLY TRANSFORMED THE WAY we produce and share information.
Its international ecosystem of applications and services allows us to search, aggregate, combine,
transform, replicate, cache, and archive the information that underpins today’s digital society.
Successful despite its chaotic growth, it is the largest, least formal integration project ever
attempted—all of this, despite having barely entered its teenage years.
Today’s Web is in large part the human Web: human users are the direct consumers of the services
offered by the majority of today’s web applications. Given its success in managing our digital
needs at such phenomenal scale, we’re now starting to ask how we might apply the Web’s
underlying architectural principles to building other kinds of distributed systems, particularly the
kinds of distributed systems typically implemented by “enterprise application” developers.
Why is the Web such a successful application platform? What are its guiding principles, and how
should we apply them when building distributed systems? What technologies can and should we
use? Why does the Web model feel familiar, but still different from previous platforms?
Conversely, is the Web always the solution to the challenges we face as enterprise application
developers?
These are the questions we’ll answer in the rest of this book. Our goal throughout is to describe
how to build distributed systems based on the Web’s architecture. We show how to implement
systems that use the Web’s predominant application protocol, HyperText Transfer Protocol
(HTTP), and which leverage REST’s architectural tenets. We explain the Web’s fundamental
principles in simple terms and discuss their relevance in developing robust distributed applications.
And we illustrate all this with challenging examples drawn from representative enterprise
scenarios and solutions implemented using Java and .NET.
The remainder of this chapter takes a first, high-level look at the Web’s architecture. Here we
discuss some key building blocks, touch briefly on the REpresentational State Transfer
(REST) architectural style, and explain why the Web can readily be used as a platform for
connecting services at global scale. Subsequent chapters dive deeper into the Web’s principles and
discuss the technologies available for connecting systems in a web-friendly manner.
Architecture of the Web
Tim Berners-Lee designed and built the foundations of the World Wide Web while a research
fellow at CERN in the early 1990s. His motivation was to create an easy-to-use, distributed, loosely
coupled system for sharing documents. Rather than starting from traditional distributed application
middleware stacks, he opted for a small set of technologies and architectural principles. His
approach made it simple to implement applications and author content. At the same time, it enabled
the nascent Web to scale and evolve globally. Within a few years of the Web’s birth, academic
and research websites had emerged all over the Internet. Shortly thereafter, the business world
,started establishing a web presence and extracting web-scale profits from its use. Today the Web
is a heady mix of business, research, government, social, and individual interests.
This diverse constituency makes the Web a chaotic place—the only consistency being the
consistent variety of the interests represented there; the only unifying factor the seemingly never-
ending thread of connections that lead from gaming to commerce, to dating to enterprise
administration, as we see in Figure 1-1.
Despite the emergent chaos at global scale, the Web is remarkably simple to understand and easy
to use at local scale. As documented by the World Wide Web Consortium (W3C) in its
“Architecture of the World Wide Web,” the anarchic architecture of today’s Web is the
culmination of thousands of simple, small-scale interactions between agents and resources that use
the founding technologies of HTTP and the URI.[ 1 ]
, Figure 1-1. The Web
The Web’s architecture, as portrayed in Figure 1-1, shows URIs and resources playing a leading
role, supported by web caches for scalability. Behind the scenes, service boundaries support
isolation and independent evolution of functionality, thereby encouraging loose coupling. In the
enterprise, the same architectural principles and technology can be applied.
Traditionally we’ve used middleware to build distributed systems. Despite the amount of research
and development that has gone into such platforms, none of them has managed to become as
pervasive as the Web is today. Traditional middleware technologies have always focused on the
computer science aspects of distributed systems: components, type systems, objects, remote
procedure calls, and so on.
The Web’s middleware is a set of widely deployed and commoditized servers. From the obvious—
web servers that host resources (and the data and computation that back them)—to the hidden:
proxies, caches, and content delivery networks, which manage traffic flow. Together, these
elements support the deployment of a planetary-scale network of systems without resorting to
intricate object models or complex middleware solutions.
This low-ceremony middleware environment has allowed the Web’s focus to shift to information
and document sharing using hypermedia. While hypermedia itself was not a new idea, its
application at Internet scale took a radical turn with the decision to allow broken links. Although
we’re now nonplussed (though sometimes annoyed) at the classic “404 Page Not Found” error
when we use the Web, this modest status code set a new and radical direction for distributed
computing: it explicitly acknowledged that we can’t be in control of the whole system all the time.
Compared to classic distributed systems thinking, the Web’s seeming ambivalence to dangling
pointers is heresy. But it is precisely this shift toward a web-centric way of building computer
systems that is the focus of this book.
Thinking in Resources
Resources are the fundamental building blocks of web-based systems, to the extent that the Web
is often referred to as being “resource-oriented.” A resource is anything we expose to the Web,
from a document or video clip to a business process or device. From a consumer’s point of view,
a resource is anything with which that consumer interacts while progressing toward some goal.
Many real-world resources might at first appear impossible to project onto the Web. However,
their appearance on the Web is a result of our abstracting out their useful information aspects and
presenting these aspects to the digital world. A flesh-and-blood or bricks-and-mortar resource
becomes a web resource by the simple act of making the information associated with it accessible
on the Web. The generality of the resource concept makes for a heterogeneous community. Almost
anything can be modeled as a resource and then made available for manipulation over the network:
“Roy’s dissertation,” “the movie Star Wars,” “the invoice for the books Jane just bought,” “Paul’s
poker bot,” and “the HR process for dealing with new hires” all happily coexist as resources on
the Web.
Systems
THE WEB HAS RADICALLY TRANSFORMED THE WAY we produce and share information.
Its international ecosystem of applications and services allows us to search, aggregate, combine,
transform, replicate, cache, and archive the information that underpins today’s digital society.
Successful despite its chaotic growth, it is the largest, least formal integration project ever
attempted—all of this, despite having barely entered its teenage years.
Today’s Web is in large part the human Web: human users are the direct consumers of the services
offered by the majority of today’s web applications. Given its success in managing our digital
needs at such phenomenal scale, we’re now starting to ask how we might apply the Web’s
underlying architectural principles to building other kinds of distributed systems, particularly the
kinds of distributed systems typically implemented by “enterprise application” developers.
Why is the Web such a successful application platform? What are its guiding principles, and how
should we apply them when building distributed systems? What technologies can and should we
use? Why does the Web model feel familiar, but still different from previous platforms?
Conversely, is the Web always the solution to the challenges we face as enterprise application
developers?
These are the questions we’ll answer in the rest of this book. Our goal throughout is to describe
how to build distributed systems based on the Web’s architecture. We show how to implement
systems that use the Web’s predominant application protocol, HyperText Transfer Protocol
(HTTP), and which leverage REST’s architectural tenets. We explain the Web’s fundamental
principles in simple terms and discuss their relevance in developing robust distributed applications.
And we illustrate all this with challenging examples drawn from representative enterprise
scenarios and solutions implemented using Java and .NET.
The remainder of this chapter takes a first, high-level look at the Web’s architecture. Here we
discuss some key building blocks, touch briefly on the REpresentational State Transfer
(REST) architectural style, and explain why the Web can readily be used as a platform for
connecting services at global scale. Subsequent chapters dive deeper into the Web’s principles and
discuss the technologies available for connecting systems in a web-friendly manner.
Architecture of the Web
Tim Berners-Lee designed and built the foundations of the World Wide Web while a research
fellow at CERN in the early 1990s. His motivation was to create an easy-to-use, distributed, loosely
coupled system for sharing documents. Rather than starting from traditional distributed application
middleware stacks, he opted for a small set of technologies and architectural principles. His
approach made it simple to implement applications and author content. At the same time, it enabled
the nascent Web to scale and evolve globally. Within a few years of the Web’s birth, academic
and research websites had emerged all over the Internet. Shortly thereafter, the business world
,started establishing a web presence and extracting web-scale profits from its use. Today the Web
is a heady mix of business, research, government, social, and individual interests.
This diverse constituency makes the Web a chaotic place—the only consistency being the
consistent variety of the interests represented there; the only unifying factor the seemingly never-
ending thread of connections that lead from gaming to commerce, to dating to enterprise
administration, as we see in Figure 1-1.
Despite the emergent chaos at global scale, the Web is remarkably simple to understand and easy
to use at local scale. As documented by the World Wide Web Consortium (W3C) in its
“Architecture of the World Wide Web,” the anarchic architecture of today’s Web is the
culmination of thousands of simple, small-scale interactions between agents and resources that use
the founding technologies of HTTP and the URI.[ 1 ]
, Figure 1-1. The Web
The Web’s architecture, as portrayed in Figure 1-1, shows URIs and resources playing a leading
role, supported by web caches for scalability. Behind the scenes, service boundaries support
isolation and independent evolution of functionality, thereby encouraging loose coupling. In the
enterprise, the same architectural principles and technology can be applied.
Traditionally we’ve used middleware to build distributed systems. Despite the amount of research
and development that has gone into such platforms, none of them has managed to become as
pervasive as the Web is today. Traditional middleware technologies have always focused on the
computer science aspects of distributed systems: components, type systems, objects, remote
procedure calls, and so on.
The Web’s middleware is a set of widely deployed and commoditized servers. From the obvious—
web servers that host resources (and the data and computation that back them)—to the hidden:
proxies, caches, and content delivery networks, which manage traffic flow. Together, these
elements support the deployment of a planetary-scale network of systems without resorting to
intricate object models or complex middleware solutions.
This low-ceremony middleware environment has allowed the Web’s focus to shift to information
and document sharing using hypermedia. While hypermedia itself was not a new idea, its
application at Internet scale took a radical turn with the decision to allow broken links. Although
we’re now nonplussed (though sometimes annoyed) at the classic “404 Page Not Found” error
when we use the Web, this modest status code set a new and radical direction for distributed
computing: it explicitly acknowledged that we can’t be in control of the whole system all the time.
Compared to classic distributed systems thinking, the Web’s seeming ambivalence to dangling
pointers is heresy. But it is precisely this shift toward a web-centric way of building computer
systems that is the focus of this book.
Thinking in Resources
Resources are the fundamental building blocks of web-based systems, to the extent that the Web
is often referred to as being “resource-oriented.” A resource is anything we expose to the Web,
from a document or video clip to a business process or device. From a consumer’s point of view,
a resource is anything with which that consumer interacts while progressing toward some goal.
Many real-world resources might at first appear impossible to project onto the Web. However,
their appearance on the Web is a result of our abstracting out their useful information aspects and
presenting these aspects to the digital world. A flesh-and-blood or bricks-and-mortar resource
becomes a web resource by the simple act of making the information associated with it accessible
on the Web. The generality of the resource concept makes for a heterogeneous community. Almost
anything can be modeled as a resource and then made available for manipulation over the network:
“Roy’s dissertation,” “the movie Star Wars,” “the invoice for the books Jane just bought,” “Paul’s
poker bot,” and “the HR process for dealing with new hires” all happily coexist as resources on
the Web.
