pushState Enabled Twitter
A case-study (with a working implementation) of extreme progressive enhancement applied to mobile.twitter.com
Background
Hashbangs are dead
Late in May 2012, Twitter began migrating its twitter.com site from hashbangs to plain-old-URLs. This was done for a number of reasons, but one of the higher priorities was improving performance, most notably "reducing the time to first tweet".
The scripted page navigation that was facilitated by using AJAX and hashbangs is (will be)
replicated by using AJAX and history.pushState() (on browsers that support it).
What's still wrong with twitter.com?
- Not usable with javascript disabled
- Not ready for small (narrow) screens and windows
- Bandwidth. Before the migration,
Mike Migurski complained
about an initial download of up-to 2MB for viewing a single tweet.
The bandwidth for a single tweet is better now (and the initial startup is faster) but the site is still very heavy for what it does, and 2MB on initial page-load is still observable.
How could it be fixed anyway?
Funnily enough, twitter already has a site that uses plain-old-URLs and provides optimal time to first tweet - mobile.twitter.com. This site also:
- doesn't require javascript; and
- is usable on small screens (such as ... mobiles); and
- uses significantly less bandwidth.
What if Twitter had applied a mobile-first / content-first strategy to the twitter.com upgrade, using mobile.twitter.com as the starting point and relying on progressive enhancement in the browser to provide a richer UI?
This article provides a case study of how this could be done. There is also a javascript bookmarklet which implements (as near as possible) the approach detailed here - see the meeko-twitter page.
Design
What are we starting with?
All of the content pages on mobile.twitter.com have the similar structure and appearance:
- a site navigation bar at the top
- the main content in the middle
- a search form and more navigation at the bottom
For instance:
Home page
Account page
Discover page
What are we aiming for?
twitter.com has corresponding pages, but with a richer UI plus page-specific auxilliary content.
There are, however, only a few variations on page-layout and, conveniently, all the pages on twitter.com can be represented by one page template consisting of multiple panels, each of which may be separately hidden (or populated and revealed) as appropriate.
The following picture illustrates this single page-template, with labels on various panels (and also some source directives which I will explain later).
The goal of our progressive enhancement is that whenever the browser navigates to a page
this page-template is loaded as the content of the window (with all panels hidden).
the main panel is populated with the main-content of the page and shown.
appropriate auxilliary panels are populated and shown.
(If the navigation occurs with pushState + AJAX support then the template is already loaded and step #1 is skipped. See later in the article.)
What changes need to be made on mobile.twitter.com?
- a script must be added to each page.
- we probably want the script to be delivered from the mobile.twitter.com server
- ideally the script would prevent the page content from being displayed until
after the page-template has loaded. This requires the script to be
in the
<head>of the page. - we don't want to hard-code the page-template location into the script,
so a js config call or HTML element could be used to specify it.
The
<link>element is designed for this purpose so probably use that.
This suggests adding something like the following code to the <head> of each page:
<link rel="template" type="text/html" href="/template.html" />
<script type="text/javascript" src="/loader.js"></script>
No changes are necessary to the <body> of pages,
however there are some pages which have main-content that
we don't want to be placed inside the main panel.
The following pictures illustrate this by outlining the page-content desired for the main panel (red) and secondary content (green).
Home page
Account page
Discover page
This could be handled by changes to the server-generated pages, but since our enhancement is extracting the relevant content anyway, it is trivial to rearrange the page when it is received to move the secondary content outside of the main-content container.
How are the auxilliary panels handled?
After the main-content is inserted into the main panel the auxilliary panels are checked for relevance to the current page. If the panel is irrelavent it is hidden (if not already hidden), otherwise:
- if the panel requires site-content then a background task is initiated to fetch the content and populate and show the panel.
- if the panel doesn't require site-content then it can be shown immediately, although there may need to be pre-processing to make the panel content match the page. e.g. hyperlinks in the user-nav panel would need updating to match the user_id.
What about popup dialogs for composing / replying-to tweets?
While posting tweets to mobile.twitter.com will have a specific format, the overall process of wrapping a form in a dialog popup and serializing the form-data to post with XMLHttpRequest is hardly novel and is left as an exercise to the reader.
Example: Home page
When the browser navigates to the home page (mobile.twitter.com), the following steps should take place:
- page begins downloading
loader.jsscript loads and runs- the
template.htmllink is detected and loading initiated - when the template has loaded it is inserted into the browser window before any page-content
- the page-content is removed from the window and the main-content moved to the main panel of the template (that is, the main-content is moved to its proper location in the window)
- the "mini-profile" panel is processed. A task is initiated to
- load the
/accountpage - rearrange the content of that page so its profile is separate from its main-content (tweets)
- insert the profile content into the "mini-profile" panel (no processing is required because CSS takes cares of hiding and aligning)
- show the panel
- load the
- the "trends" panel is processed in a similar way to the "mini-profile". A task is initiated to
- load the
/i/discoverpage - rearrange the content of that page so its trends is separate from its main-content (Browse Categories)
- insert the profile content into the "trends" panel (again, CSS takes cares of hiding and aligning)
- show the panel
- load the
The final state of the page will look something like
pushState / onpopstate assisted navigation
This is quite simple because most of the work has already been implemented. At this stage of enhancement we don't even need to save state because the page URL completely defines the state of the page. All that is required is:
- detect navigation requests by
- listening for
clickevents on hyperlink -<a href>- elements - listening for
popstateevents on the window
- listening for
- initiate loading (via XMLHttpRequest) of the next URL
(and call
history.pushState()to update the document.URL if the navigation was from a clicked hyperlink) - when the next page has loaded, continue processing as for the first page load above (except that the page-template is already loaded and installed)
Advantages
The obvious advantages of creating a rich UI by progressive enhancement of a content first site (such as mobile.twitter.com) are:
- noscript, mobiles and other small-screens are already catered for
- all devices can use the one site so URLs can be shared to all devices
- the generation of content pages is the same for all devices, simplifying development and reducing maintenance
Performance of this approach must be measured of course, but it does have some potential gains:
- on browsers that don't implement
pushState, each page will be loaded in full, including the real content and the UI. At least with this approach the UI (the page-template) will already be cached. - even when the page-template isn't cached, the server can send the
<head>of the content document immediately, while it continues to lookup the appropriate main-content. This allows the browser to initiate fetching the page-template before the main-content has arrived.
Another potential benefit of this approach is that it would be trivial to allow different page-templates to be selected from within the browser. The appropriate page-template could depend on
- device
- screen dimensions, or
- user-preference
Notes
- this approach is effectively using the mobile.twitter.com site as a REST API that delivers HTML responses, in a similar way to how twitter.com uses the api.twitter.com site, but without the complication of authenticated cross-site requests.
- meeko-twitter is based on HTMLDecor
which was built specifically to allow content-first sites to be progressively enhanced with
richer UI and
pushStatesupport. (This blog also relies on HTMLDecor, so browse around to see what it's like when built into a site.) - meeko-twitter DOES NOT necessarily require
history.pushState()support from the browser, but without it the bookmarklet would need to be manually run after each page navigation.
Obviously this would be automatic if this approach was built into the site. This is the primary reason Internet Explorer is not supported. - meeko-twitter uses CORS XMLHttpRequest to load the page-template (although this could be worked around by a JSON-P approach). This is the secondary reason Internet-Explorer isn't supported.
- this design DOES NOT cover content caching, although meeko-twitter DOES implement it.
- this design DOES NOT cover content updating. meeko-twitter DOES NOT either.
- mobile.twitter.com uses
<table>s for layout. I am NOT endorsing tables for layout, and meeko-twitter DOES NOT depend on their presence.
Further Reading
- content-first and mobile-first are similar concepts which could also be described as extreme progressive-enhancement, where non-essential content of a URL may also be added via scripted enhancement.
- HTMLDecor is the term - and JS library - I use
for enhancing content-first pages with site-decor (and
history.pushState()where supported). - hinclude and Ajax-Include are JS libraries for conditionally or lazily including parts of other pages into the current one, in a similar manner to the way auxilliary panels are populated in meeko-twitter.
Contact
If you have any questions, or if there are errors or omissions in this article, feel free to contact me via the contact dialog on this page.