pushState() was made for HTMLDecor

When you've finished this article you will be aware of two things:

the article is long
I am not a great writer

What I'm actually hoping to convey is in the Overview, up next.

Overview

A typical use-case for history.pushState() is over-riding the normal browser navigation between pages within the same site. The default navigation is disabled and AJAX handles changing the browser URL and updating the displayed content without refreshing the page. This involves downloading URL-specific content for the next page and merging it with the shared site decor - banners & navigation, layout & stylesheets - that is already in the page.

This is a perfect match for HTMLDecor which expects URLs to only contain specific content, and constructs the end-user view in the browser by merging this content with shared site decor obtained from a decor document.

Thus sites and pages designed to use HTMLDecor can benefit from pushState() without modification. pushState() is already supported in the latest version of HTMLDecor.

Background

I caught up with an imaginary friend the other day and he started asking me about the latest version of HTMLDecor, pushState and Twitter.

The conversation went something like this...

Intro

I see that version 1.1 of HTMLDecor is available. What has changed?

Well, the only goal for this version was adding support for history.pushState() on browsers that implement it. This required some internal refactoring, but that's all.

So, if I've been using HTMLDecor on my site, what do I have to change in order to benefit from this pushState support?

Nothing, except for upgrading to the latest version of the HTMLDecor.js script.

I don't need to make any change within a page?

No. HTMLDecor and pushState were made for eachother.

Actually, I haven't really considered how, or even why, I would use pushState on my site. But I did hear that Twitter is switching from hashbangs to pushState. Why are they doing that?

Quoting more-or-less from the horse's mouth, it's because

URLs are important
URLs are forever (and once you hashbang you can't just stop)
Cool URLs don't change
hashbangs are less search-engine friendly, and
hashbangs don't gracefully degrade

But those are all reasons not to use hashbangs in the first place. You know, like the way the web used to work. Why should I start using pushState?

Actually, a good introduction to why and how to use pushState is Manipulating History for Fun & Profit from the Dive into HTML5 book by Mark Pilgrim. Why don't you take a few minutes to read that?

...

Okay, I'm done.

Did you read it?

I skimmed it. I'll read it properly tomorrow.

What a coincidence - that's just what I would do.

Anyway, as long as you're here...

Why should I use pushState?

Most sites have pages that consist of page-specific content together with (often considerable) shared content - banners, navigation, layout divs & stylesheets.

And you call this shared content "site decor"?

Yes, decor. Not that banners, navigation and layout are purely decorative but they aren't specific to the page and you may well make changes to the decor without considering that the essential content of the page has changed.

And the page-specific content... you refer to that as unique or essential content?

Actually I prefer the term raw, as in unprocessed or - sticking with the idea of decor - undecorated.

The "Dive into HTML5" article complained that downloading and rendering this shared content, again-and-again for each page that is viewed, seems wasteful.

Wow, great skim-reading.

Imagine the browser could be instructed to only download the raw content of pages (once the shared decor is already loaded). The network load would be reduced, directly resulting in quicker loading times for new pages.

Additionally, if the decor can remain in place between page navigations then it (hopefully) doesn't need a relayout or repaint, which should also slightly improve the laoding time. More importantly, this enables a better user-experience as it is clear that those parts of the page haven't changed.

But you can already do this with AJAX.

Yes, with the minor short-coming that the URL in the address bar will not be changed to match the updated content of the page. Which means that you can't bookmark the URL. Oh, and the back button won't work as you expect. Little things like that.

Well, how about faking it using hashes... oh, nevermind.

Anyway, pushState is the simple HTML5 feature that allows your AJAX to also update the URL in the address bar, which resolves these problems.

Just like that, huh? You'd better explain a bit more, like...

How do I use AJAX and pushState?

When someone clicks on a hyperlink inside your page, what is the process of handling that? And what design issues need to be resolved?

Yes.

There are three basic steps:

check that the next URL is compatible
get the raw content for the next URL
update the content being displayed in the browser

You also have to make the pushState call at some stage. Remind me later.

Check that the next URL is compatible

Not every hyperlink in a page will be to pages that share a common decor. In those instances you will want to defer to the browser (or a different handler). Some links should be ignored.

links to pages on other sites
links with a different protocol, e.g. javascript:..., ftp:
links that target a different window or iframe, e.g.
```
<a href="some_page.html" target="_blank">...</a>
```
anchor links - <a href="#skip">.
(Although you might want to handle these in a different way)

That leaves us with links to other pages within your site. But not all of these will necessarily have the same decor.

What is a good policy for deciding if the decor for the next URL will match that of the current URL?

Here are a few ideas:

use a class-name or data-attribute on the hyperlink element, e.g.
```
<a href="some_page.html" class="pushstate-safe">...</a>
```

same file, different query

/blog/index.php?id=1
/blog/index.php?id=2

same directory
```
/blog/story1.html
/blog/story2.html
```

same parent directory

/blog/story1/index.html
/blog/story2/index.html

I suspect they would cover the majority of real-world scenarios.

Is there a solution that would allow the match to be done on a case-by-case basis?

A more flexible solution would be to query the server for the match. Presumably you would want to combine this with the request for new content, suggesting requests of the form:

    /raw.php?path=/blog/story2.html&decor={token}

which asks for raw content for /blog/story2.html, with decor={token} asserting that the content must be compatible with the decor represented by {token} (which I'll get to shortly).

Now, if the server responds with an error then the decor for the next URL does not match the current decor, so you want the browser to handle the navigation normally. Unfortunately the default handling had to be cancelled to run this check, so you would need to trigger navigation manually:

    location.assign(nextURL);

Ok, back to that decor token.

{token} can be anything so long as pages with the same decor configure the same token. One option would be to set it in a <meta> of the page, like

    <meta name="decor" content="story" />

A really useful, forward thinking option is for {token} to be the URL of the file that supplied the decor for the current URL. This URL would be added as a <link> in the <head> of all pages that shared that decor.

    <link rel="decor" href="/blog/story.html" />

Have you got any other really useful, forward thinking ideas?

Well, as long as you are making a server request anyway, and since you're using decor tokens within your page, you could just require that the raw content response also contains an appropriate decor token, and then do the match in the browser. The request URL might be like:

    /raw.php?path=/blog/story2.html

or, for a statically generated site:

    /raw/blog/story2.html

If the raw content for the next URL includes a decor token, and if it matches that of the current URL, then the content is accepted for merging into the current page. Otherwise you need to trigger manual navigation.

The down-side of this approach is that you might download the raw content and then not use it, but instead tell the browser to navigate to the fully decorated page.

It's a shame that you can't download the decor without any page-specific content and merge it with the raw content you just downloaded.

Yes, that's not a bad idea.

Is that what HTMLDecor does?

Not quite. If the decor for the next URL differs then HTMLDecor will still trigger the browser's normal navigation. What happens then is the browser loads the next URL from cache (assuming that HTTP caching headers allow it) which in turn loads the HTMLDecor script (again from cache) and then the appropriate decor is loaded and merged into the raw content.

So, assuming caching is configured, the only thing that needs to be fetched from the server is the decor for the next URL.

Obtain the raw content

I guess we've more-or-less covered fetching the raw content. You use XMLHttpRequest to get the raw content file (however that is made available), then grab the responseText and assign it as innerHTML of some container element in the page.

Yes, you could do that. That's what the example in the "Dive into HTML5" article uses. It's not particularly flexible though:

it assumes that the unique content will all belong inside one element in the page
a corollary of #1 is it doesn't handle page-specific title, meta-information, styles and scripts

I see. It would be more flexible to send the raw content as JSON. What does HTMLDecor do again?

HTMLDecor expects the raw content to be a usable, if not graceful, HTML document that includes a <head> and a <body> and any valid HTML elements. Child elements in the <head> are conditionally adopted into the <head> of the page and child elements of the <body> conditionally replace matching elements in the <body> of the page.

Ah yes. In HTMLDecor, pages are meant to be just raw content which then gets decorated in the browser. When pushState is added to the mix, HTMLDecor just does the reverse and inserts raw content into the decor.

Yes, that's the model.

Update the content being displayed in the browser

I'm guessing there's a few different approaches to updating the page, depending on how complex the raw content is allowed to be. Can you provide some details?

Well, the "Dive into HTML5" example assumes that all the raw content is used to replace all the previous content inside one container element on the page. This allows the update to be as simple as

    document.getElementById("gallery").innerHTML = req.responseText;

You're unlikely to leave it at that for production code. You'd at least make the container element configurable with a function call. Alternatively, you could use a <meta> or a <link> in the page to indicate the container element, like

    <meta name="raw-content" content="gallery" />

Another option would be to include the container element in the raw content. Then there is no need to specify the ID of the container element.

This option can be extended fairly intuitively to raw content that can be split and inserted into multiple places in the page. The raw content might look like

    <ul id="similar">
     <li>No similar photos</li>
    </ul>
    <aside id="gallery">
        <!-- photo markup here -->
        ...
    </aside>

And we almost got this far before when I speculated JSON for sending the raw content. HTMLDecor expects a normal HTML document, which does seem fairly flexible and has the benefit that it can be usable as a page with or even without decor.

Yes. I didn't consider pushState when initially developing HTMLDecor (in fact, pushState wasn't available) but in the typical scenario - over-riding page navigation within a site - they are dealing with the same problem - merging page-specific content and site-decor.

Like I said - they were made for eachother.

Should Twitter switch to HTMLDecor?

Well, I expect whatever solution they arrive at will be similar. I'm sure they have their own specific requirements, plus they would probably be hoping for some features that are only on the TODO list.

Can you elaborate on what's

Coming Soon in HTMLDecor

Sure.

Alternate decor

One potential benefit of decorating raw content in the browser is that you can also choose the decor according to the browser or even window context. Some possible options for choosing are

the window.name. If the content is in a named iframe then it might be appropriate to have a different decor
media queries. Imagine having different layout, navigation behavior, styles and scripts for different media. Same raw content (what the search-engines see) but the option for device specific decor.

Transistions

At the moment there is no transition effect when content is replaced in the page. But transitions can improve the user-experience, providing visual cues for what has changed and what hasn't. It would be beneficial is site developers could configure this.

Wow. So how many forward thinking people / sites are already using HTMLDecor?

If you want to see it in action you can visit my blog.

Anyone else?

Why don't we talk again next week.

You forgot to tell me when to call pushState().

I'm sorry, you have just encoutered a history.pushState() bug in Webkit.