🥳 You have completed all topics in the Handbook. Click here to claim your certificate!

Technical Marketing Handbook

2. From request to response

Everything you see and interact with on a web page is the ultimate result of the browser requesting content and a web server responding to this request.
  • 16 min read

In the previous Topic, we discussed the architecture of the web browser on a superficial level. In this Topic, it’s time to dive in.

While it’s important to understand the interplay between different layers and engines and user interface features, the browser wouldn’t be very useful if it didn’t actually fetch content from servers all around the internet.

Servers are machines connected to the internet designed to provide content upon request. The web browser can request all sorts of content from a web server, ranging from HTML source files to stylesheets, from images to script libraries, and from audio to video.

The most fundamental request–response dialog is associated with the navigation action.

When you navigate to a web page, the browser sends a request for the web content to a web server. Which web server it connects with is determined by the domain name associated with the web server.

Don’t miss this fact!

The web experience is built around requests (dispatched by the web browser), responses (returned by the web server), and rendering, where the web browser paints or executes the resource in the server response.

What follows is a walkthrough of this process. It starts when the user types a web address into the address bar of the web browser, and it ends when the user (hopefully) sees the page fully loaded in the viewport of the browser.

The key parts of this exchange are:

  1. The browser is the client that wants to navigate to a URL (web address).
  2. If the URL is legit, it points to an HTML resource on a web server.
  3. The web server sends a response to the browser with the HTML resource in the response body.
  4. The web browser takes this HTML resource and starts rendering it into an interactive, dynamic web page.
  5. Once all the resources referenced in the HTML are loaded, the page load is complete and the process ends.

At almost every step of the way, there are important lessons to be learned for anyone who works in technical marketing.

Step 1: The navigation action

The user opens the web browser, clicks the address bar, and types in the address:

https://www.teamsimmer.com/

This is an absolute URL. It comprises the following parts:

  • https (followed by the separator ://) is the protocol of the request. The s at the end indicates that this request should be transferred using secure, encrypted pathways.
  • www.teamsimmer.com is a three-level domain name.
    • The top-level domain is .com.
    • Under this domain, the name teamsimmer has been reserved and bought by a private company (Simmer, in this case).
    • Finally, www is the subdomain of this privately owned main domain. All three parts are used to identify which web server this domain name points to in the domain name system (DNS).
  • The / at the end is the resource indicator that instructs the web server which resource is being requested. If the path ends in a slash, as this one does, then the request is typically implicitly for a root HTML source file.

Because the user entered the URL in the address bar of the browser, this is a navigation action. There are many different types of resource loads, but a navigation action is almost always a request for an HTML source file on a web server.

HTML is HyperText Markup Language. It’s a programming language that informs the web browser about the structure, design, and functionality of the web page that was loaded. The HTML source file is a set of instructions for how to build, render, and activate the web page the user sees.

Deep Dive

The anatomy of a URL

A typical web URL is constructed with these components:

  • Protocol establishes whether the request happens behind secure HTTPS (HyperText Transfer Protocol – the communication protocol of the web) or insecure HTTP. Luckily, you don’t really see the latter anymore.
  • Hostname locates the web server, and it has to include at least the top-level domain name (e.g. .com) and the naked domain name (e.g. teamsimmer). It can also include a subdomain (e.g. www).
  • Port would come right after the hostname (www.teamsimmer.com:443) to add detail to the address, but this is rarely used in regular navigation.
  • Path is a route to a resource endpoint on the web server.
  • Query string can comprise multiple key-value pairs that add detail to the requested resource.
  • Fragment can optionally be used to encode information that is only available in the browser (the web server cannot read the fragment), for example: #content would instruct the browser to jump to an element flagged with the anchor “content”.

The protocol, hostname, and optional port also combine to create what’s known as the URL origin. The “origin” is often used as a logical boundary for many web-based processes, which want to limit information from being shared across origins.

URLs aren’t used just for navigation – they are also used to load any resources that a web page would instruct the browser to load.

For example, whenever you see an image on the page, it is loaded from a web server using a URL. This URL would point to an image file or a web server resource that returns an image file.

Step 2: The request is dispatched

When the user types this address and presses enter, the request dispatch begins.

One of the first things the browser needs to do is check which web server the domain name www.teamsimmer.com is pointing to. This is done through the Domain Name System, or DNS, which is the “address book” of the internet.

Once the corresponding web server is found, the browser and the server initiate a complex dialog of handshakes and protocols which seek to establish trust between the two parties.

The browser needs to trust that the web server is legitimate, and the server needs to trust that the request comes from a trusted source.

If all goes well, the web server eventually receives the request.

Step 3: The web server responds

What the web server responds with depends on the request itself.

For a navigational request, the web server needs to locate the HTML source file in its file system (for static sites) or generate the HTML dynamically for the response (for dynamic sites).

Static sites typically have the bulk of the resources available in a file system. All the web server needs to do is locate the file the request is asking for and attach it to the response.

Dynamic websites generate the response from templates, using dynamically generated content. A file named index.html, for example, wouldn’t exist in the file system. Instead, there might be a source file like index.php that dynamically initiates a server-side process, where the resource is generated based on the metadata available in the request.

Static sites are often very fast, because there’s no extra computation required from the web server to serve the file. However, they lack the flexibility of dynamic sites, where the contents of a requested resource might change depending on, for example, if the user is logged in or not.

Many modern content distribution systems employ a hybrid approach, where both static and dynamic content are utilized to build the response back to the request source.

In any case, the web server generates the HTML source for the response, assigns a status code to the response, and adds additional metadata to the response headers where necessary.

Once the response is ready, it is dispatched back to the request source (the web browser).

Ready for a quick break?

Now’s a good time to take a small break – walk around for 5 minutes and have a glass of water. 😊

Step 4: The page is rendered

When the web browser receives the HTML source file in the response, the render process starts.

The browser runs through instructions in the HTML file line by line, starting from the very top.

The purpose of rendering is to convert the HTML instructions into a dynamic web page the user can view and interact with.

Many of these HTML instructions detail the layout, text, and styles of the content itself. This HTML is rendered in a tree-like structure, where elements can be nested within each other.

Some of the HTML instructions comprise additional resource loads. For example, for every image on a page, the web browser needs to go through the request-response-render process again. Same for any JavaScript or stylesheet files that need to be loaded in the browser.

While the page becomes interactive early on in the rendering process, it’s important to note that until the entire HTML source has been rendered, the page load is still in progress.

If you have a very resource-heavy site, the page load can take a long time to complete. This might have a direct impact on processes that come at the tail end of the page load, such as analytics measurement, because often visitors are impatient and will move on from the page before it has completely loaded.

Deep Dive

Asynchronous loading

The browser renders the HTML content in a single thread. This means that the browser must render each line of content completely before moving to the next. Compare this with a multi-thread process where operations can run in parallel.

Because of the single thread, if any one of the lines of code takes an inordinate amount of time to render, the page load process will seem very slow and content is revealed in chunks rather than fluidly and seamlessly.

For this reason, it’s important to orchestrate these extra resource loads so that the page render doesn’t have to wait for potentially slowly loading images and scripts.

Asynchronous loading is the answer to this problem. With asynchronous loading, the browser starts to load the image or the script, but it doesn’t wait for the download to complete before moving on with the HTML rendering. Instead, as soon as the download is complete, the browser returns to render the loaded file.

This gives an illusion of seamless rendering, but it’s important to understand that even if content is loaded asynchronously, its rendering still happens line-by-line. So while the actual download of the image doesn’t necessarily interrupt the page render, once the image has been downloaded it still needs to be painted on the screen, which will again interrupt the rendering of the rest of the document.

Don’t miss this fact!

Users are impatient. If they don’t find what they need at a glance, they might navigate away from the page before the page load is complete. If your data collection processes are configured to happen at page load time, it’s possible they will never take place because of this.

Step 5: The page load is complete

Once all the HTML has been rendered from the opening <html> tag to the closing </html> tag, and once all the asynchronously loaded resources have been rendered and executed, the page load is complete.

This means that the spinner next to the site name in the browser tab stops spinning, and the page is fully interactive to the user.

However, this doesn’t mean that loading additional resources is completely done.

Many web pages load content even after this initial page load. For example, an online store might have a list of products that loads additional items as soon as you scroll to the end of the list.

This is called lazy loading, and it’s a smart way to make the initial page load as fast as possible. Only the elements that the user can see are loaded.

Once the user clicks a link to navigate away from the page or closes the browser window, the entire page is unloaded from browser memory to save resources.

This unloading means that every single new page load needs to start the process from scratch. There is no persistent memory in the browser itself to always keep the document alive even when the user isn’t actually browsing it.

However, there are cache mechanisms which do “remember” the content for a period of time to avoid having to load all the content again and again from remote servers.

Don’t miss this fact!

The web is stateless. Information on one page is not persisted to the next page unless specific storage mechanisms (see the next Topic) or caches are in use.

Caches help to ease the workload

The web browser uses caches to avoid having to run through complicated and costly processes over and over again, if the result of the process is expected to be identical for each load.

Think of a cache as your storage cupboard for frequently used household items. Instead of having to go to the store every single time you need a disinfectant, you can simply open your cupboard to use the one you already have. At some point the liquid will expire, at which point you need to revisit the store to get a new one.

The same process applies to caches on the web. Frequently accessed content can be temporarily hosted in different cache repositories, whose sole purpose is to make the fetching and rendering content a faster and less jarring experience.

Example

Your device has a DNS cache, where web addresses that have already been resolved to a web server are stored. That way the browser doesn’t have to query DNS to see which server any given domain name points to – it can just pull this information from the cache.

The browser has a local cache, which stores entire web pages for a short period of time. If you hit the reload button in the browser, or if you press the back button to navigate back to a previous page, it’s likely that the content loads really fast. That’s because it’s loaded from the cache – the browser doesn’t have to make any resource requests to web servers.

Some websites use dedicated cache services to load content from an edge server. For example, instead of loading large images from the image server every single time, they can be loaded from the edge cache instead, which is typically optimized for blazing fast resource delivery.

The biggest problem with caches is that they might prevent the user from seeing new content.

Example

If you run a news website, and you make a critical mistake with an important news item, you need to invalidate all caches when you publish the update. Otherwise, visitors who already visited the content page might only see the cached version and not the important update.

Caches do have a lifetime which determines when they expire, but sometimes it’s imperative to clear the caches immediately.

Key takeaway #1: DNS is the address book of the internet

DNS, or the Domain Name System, maps domain names to server machines connected to the internet. When you type a domain name into the web browser’s address bar, you are actually instructing the browser to resolve the IP address that domain name is mapped to. If the domain name is valid, the IP address should then locate a web server to which the request is dispatched.

Key takeaway #2: A web page is more than its HTML source

Even though the web server returns an HTML source file for navigational requests, this is just the starting point for how the web page is actually rendered in the browser. The HTML is parsed for instructions on how to build the page, but the final product is a dynamic document, built with a combination of HTML, CSS, and JavaScript.

Key takeaway #3: Caches lighten the load

There are a number of different caches that progressively improve page performance as you browse the web. The purpose of a cache is to temporarily store frequently accessed resources. These can be stored locally on the device, in the browser, on the network itself, and on web servers. When the browser needs to request a resource, it first looks into these caches to see if the resource has been cached, in which case it uses this instead of having to do the whole request–response process again.

Quiz: From Request To Response

Ready to test what you've learned? Dive into the quiz below!

1. What is the difference between a static site and a dynamic site?

2. What is lazy loading?

3. What is the typical resource that a navigation action requests?

Your score is

0%

What did you think about this topic?

Thanks for your feedback!
Next up
3. Cookies and other browser storage

Unlock Premium Content

Simmer specializes in self-paced online courses for technical marketers. Take a look at our offering and enroll in one or more of our courses!

Enroll Now
  • Simmer® Exclusive
  • Task-based learning
  • Digital Certificate after completion
  • Helpful Community
  • Lifetime Access
Copyright © 2024 Simmer
Contact Us | Privacy Policy | Consent Settings

Online course

Web Browsers And Tracking Protections

Simmer’s online course teaches you all you need to know about web browsers and their tracking protection mechanisms.

Learn more