Snow Fall Breakdown: How The New York Times Built a Multimedia Story

Snow Fall

UPDATE 1/3/2013: My fellow MCDM alum Rebekah Peterson shared a great Q&A with the multimedia team behind “Snow Fall” on Source. I held my breath on first read, afraid they were going to say something that completely contradicted my analysis, but I fared pretty well. If my breakdown is too “in the weeds” for you, I think you’ll find Source’s Q&A pretty accessible.

The recent New York Times multimedia story “Snow Fall: The Avalanche at Tunnel Creek” got a lot of attention. As Rebecca Greenfield put it on The Atlantic Wire, the piece “…integrates video, photos, and graphics in a way that makes multimedia feel natural and useful, not just tacked on.” The Times was rightfully proud of the piece and what it means for online journalism, boldly stating “Clearly, this is something The Times hopes to do more of, and others will undoubtedly do it, as well.”

It struck me as a beautifully designed and stunningly effective example of multimedia online storytelling and I couldn’t wait to share it at the public television station where I am the Director of the Interactive department. I was motivated in part to share as a means of saying “this is what we need to be capable of doing to survive.”

What makes the piece so remarkable isn’t that the New York Times has created anything new in a technical sense. It’s that instead of retreating from what’s disrupting their business they have embraced it and made it even better by using it to showcase their traditional strength: Meaningful storytelling.

While I love it as a strategic approach to storytelling, I also understand it as a piece of front end web development. A decade ago, the third phase of my media career led me to learn in depth how web technologies are designed to work together. That dive into the logistics of web development continues to shape my strategic and tactical thinking as a manager and I believe good online managers need to be familiar with the actual work of building digital media products. Businesses that succeed best in the online medium have managers who apply their real working knowledge of how it works to strategic decision making.

What follows is my technical breakdown of the front end code that makes the Times piece such an effective experience. My target audience is digital media managers with some interest in knowing some of the technical aspects of what goes into a presentation like this.

Here are quick links to sections in the breakdown. (For variety, most of the headings include song titles from two different musical acts.)

Scrape Away: The Available Toolkit

In order to understand what it is that we’re looking at on the page, we need to understand what tools were available to the designers and developers who made it. This is like looking inside a carpenter’s toolbelt to understand how a house is built: The odds are that you’ll see a couple hammers in there, and you can assume that when he was doing the framing he used a framing hammer, but there’s always the off-chance that he drove one or two nails into the framing with his roofing hammer instead and you’ll never be able to tell the difference from just looking at it.

Absolute Beginners: The Web’s Language Trinity

When we talk about Web pages, we’re usually talking about three related languages that have been adopted by the World Wide Web Consortium (W3C) as standards (although for arcane reasons they sometimes call them “recommendations”):

  1. HTML is a markup language. It is used to markup pieces of data in <tags> that identify types of data. Together, opening and closing <tag></tag> combinations make elements and I will use that term a lot in my breakdown. Some tags are self closing, like <img />, and don’t need a separate closing tag, but they are still elements. By itself, HTML should only tell you what something is, not what it looks like. (Before HTML 4 was adopted in 1997, that was a lot less clear and you still sometimes see new pages built with old bad habits like using <table> elements to force layouts, but those are developers who are doing it wrong and they shouldn’t get your money.)
  2. CSS (Cascading Style Sheets) is a styling language. It’s how to define what HTML should look like. For example, to make all the text marked up as paragraphs appear in a 14 pixel font: 
    p { font-size: 14px; }

    or to make all the images with a class of “sidebar” appear on the right hand side of the page:

    img.sidebar { float:right; }

  3. JavaScript is used to make Web pages do stuff besides just sit there and look pretty with some clickable links. If something moves, changes color, or otherwise acts dynamic, odds are there’s JavaScript involved. (There’s still a slight chance the Adobe Flash plugin, or even Microsoft’s Silverlight plugin, is doing it, but that’s far less common than it used to be.) Even though I said up above that JavaScript is a W3C standard, that’s not really quite true — the JavaScript language is a standard of the Ecma International standards body (you might rarely see JavaScript refered to as ECMAscript), but the W3C defines how it interacts with HTML.

The most basic structure of a Web page is this HTML:

<html>
    <head></head>
    <body></body>
</html>

Everything that you see in a Web browser is inside that <body> element. But on most pages of any level of complexity, far more hours of work have gone into the stuff you can’t see. Much (but not all!) of that is inside the <head> element, and often the only thing in there that is displayed is the <title> element which is what you see displayed in the top titlebar of the browser — or at least used to: With the adoption of tabbed browsing there isn’t much room to show titles these days.

The <head> can contain information about the page, pointers to external resources the page needs to work properly, or contain those resources embedded directly in the HTML. But even before you get the <head>, choices are made that will have important impacts on how the stuff in the <body> is displayed.

Illustration 2: “Snow Fall”‘s page source

This is what is sent from the server before it is modified by JavaScript in the browser. The line numbers have been added for reference. (The page source was viewed in the Chrome browser for Mac OSX — different source may be sent to other browsers if the server is set up to do so.)

Standards: The Doctype

The very first line, <!DOCTYPE html> (line 1) tells us two things:

  1. The developers who built this page know that including a doctype declaration tells web browsers to act in similar, consistent ways, overriding many, but not all, of the idiosyncratic “quirks mode” behaviors browser makers build into their products to accomodate pages that don’t adhere to current standards.
  2. The simple doctype “html” means this page is intended to be treated as HTML5, the official, unofficial standard for web markup that has just become a W3C Candidate Recommendation after eight years of work and 15 years after the HTML 4 was adopted.

But I’m Different Now: Conditional Comments

Lines 2 through 6 are interesting examples of “conditional comments.” This proprietary Microsoft Internet Explorer technique allows different versions of the browser to use different snippets of code. These have proven to be very useful as Internet Explorer has a notorious reputation for being buggy and poor at following standards. With conditional comments, developers can, on a case by case basis, provide fixes and hacks to account for the oddities of different versions of IE. Normal HTML comments are marked up <!-- like this --> and browsers will just ignore them. But Internet Explorer looks inside to see if they contain instructions in a format that only it can read.

In this case, the opening <html> tag will be written with different “class” attribute values identifying the major IE browser version. Classes are used to define styling via Cascading Style Sheets and behaviors via JavaScript. The most complex conditional comment output on the page, <!--[if lt IE 7]> <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]-->, says that if the version of IE is less than 7, then write <html class="no-js lt-ie9 lt-ie8 lt-ie7">. That series of classes identifies the browser as less than IE9, less than IE8, and less than IE7. This will allow fine-grained targeting of workarounds for IE’s annoyances. A little trick of putting comments inside comments in line 6 ensures that for IE versions higher than 9, as well as any other type of browser, the <html> tag always is written with the “class” value of “no-js” and none of that IE version stuff: <html class="no-js">. We’ll come back to that “no-js” part when we get to how the page renders.

I like this solution because it’s more elegant than the way you more typically see conditional comments used, which is to include multiple external CSS files in order to fix IE’s peculiarities.

News of the World: The Meta Elements

The <meta> elements are places for storing information about the page and its contents. This information might be used by the Web browser, by processes internal to the New York Times publishing system, or by search engines or other external processes. The most interesting <meta> elements are various forms of structured data. The Times is supporting a variety of syntaxes used by different external processes to more effectively share its content. Note for example in how many different <meta> elements the full title is presented (lines 15, 17, 38) and different formats there are for the publishing date (lines 19, 21, 23). This isn’t sloppiness — those each are structured to be read by different services and help garner traffic.

Plug the page address into Google’s Structured Data Testing Tool and you can see what information is helping it show up in search results. (Bing has one too, but you have to make an account to use it.)

Some of the <meta> elements have “property” attributes with values that begin with “og:” such as <meta property="og:title" content="Snow Fall: The Avalanche at Tunnel Creek"/> (line 46). Those are used by Facebook’s Open Graph protocol which, simply put, means if the page is shared on Facebook the Times gets to control how it displays. The <meta> element with the “property” attribute value “fb:app_id” (line 50) is used to track pages on the domain “nytimes.com” in Facebook’s Insights analytics: If a Facebook user shares the page, the number of shares shown in Insights will increase by one.

The <meta> elements with “name” attribute values beginning with “twitter:” (lines 51-53) are, naturally enough, used by Twitter to enhance the experience when the page is share via tweets. For example, <meta name="twitter:creator" value="@JohnBranchNYT"> lets Twitter share the handle of the piece’s author.

Art School: CSS

Styling begins on line 65 with a large chunk of CSS that is embedded in the page. Since accepted best practice is to keep CSS in an external file where it can be updated without having to touch every page that uses it, this was a very strong indicator that “Snow Fall” is a highly-custom, perhaps entirely hand-built, experience.The “media” attribute of the <style> element has a value of “screen” which normally implies there are styles for “print” defined somewhere, but they are missing from “Snow Fall.” This is really intended to be an online experience first, and not just a digital newspaper.

Much of the embedded CSS is the “normalize.css” library, which the author describes as an “alternative to the traditional CSS reset.” That’s all well and good, as long as you know what a CSS reset is.

Every browser comes with its own built-in stylesheet — when it sees a <h1> tag it makes a big, bold heading with some amount of padding above and below; when it sees <p> tag it makes a paragraph, with some amount of spacing above and below and with some amount of line height on the text within. What these things look like are entirely up to the browser makers — there’s similarity between them, but trying to work with them in applying a design can be maddening.

CSS resets are stylesheets that level the playing field between browsers by overriding all browser styling and making everything look exactly the same: A heading is the same size and weight as a paragraph, a list item, or anything else. The front end developer can now work with this blank slate and apply new styles from scratch knowing that they will work exactly the same on all browsers.

Normalize.css approaches the problem differently: Its goal is to override all the differences between browser stylesheets without removing styling completely. A <h1> in IE will still be big and bold with padding above and below, but now it will be big, bold, and padded exactly the same way as it is in Firefox, Chrome, or Safari. In theory, it saves the front end developer the trouble of specifying “big and bold,” so he or she can concentrate on the parts of the design that matter. In my opinion, the reset approach makes more sense because it doesn’t rely on knowing what every browser maker does with every element, so it is inherently more future proofed against future changes with new browser versions.

CSS comments, which are marked up /* like this */ show that between line 451 and 452, the normalize CSS ends and New York Times styles begin. These look like they might be lifted from the normal Times stylesheet in order set a baseline of consistency in fonts and such. This is a good place to illustrate the tradeoffs of using CSS reset or normalize. On line 131, the normalize library generically sets the font-family for the entire page

font-family: sans-serif;

then on line 454 the Times‘ styles completely override that setting with a much more specific list of fallback styles

font-family:georgia, "times new roman", times, serif;

Typically the inefficiency of redundant styling is more than offset by the efficiencies gained in maintenance and development, but purists might prefer to modify the reset or normalize CSS to set all the styles in one pass.

The bulk of the actual page styling is in an external stylesheet, which has been “minified” by having all unneeded whitespace removed in order to make it smaller and faster to deliver. That also means it’s really hard for humans to read.

On hand-built projects like this, it’s pretty common to make the code just tidy enough to not be embarrassing, which is why there’s so much CSS inside the HTML. By contrast a normal Times article published with their CMS has no embedded CSS, just a single linked stylesheet that primarily imports other, neatly organized, external stylesheets. (Interestingly, I’m not finding any sort of CSS reset or normalize library in use on the CMS pages.)

This Is the Modern World: JavaScript

The last thing we find in the <head> element are two <script> elements. The first one just holds variables for Google Analytics, but the second does something much more interesting. It links to HTML5 Shiv.js (line 819). This JavaScript library is key to making sure the experience to come works across the chaos of browsers that online content creators must support. It identifies any HTML5 elements that the browser might not support and, with some clever tricks, persuades the browser to support them. Thanks in no small part to Apple’s refusal to support Flash video on iOS devices, to many people “HTML5” is synonymous with “video that doesn’t need a plugin.” There’s more to HTML5 than that, but in truth the most visually apparent use of HTML5 in “Snow Fall” actually is the <video> element.

But, surely such a complexly interactive experience must have more JavaScript than that? Indeed it does, but you won’t find it in the <head>. Following a best practice for user experience, the Times‘ developers only put the resources necessary to display the page at the beginning of the file so viewers don’t have to wait for everything to load first in order to see it. Way down at the end of the <body> element, below all the content, is the treasure trove of JavaScript that does the real work of creating the experience.

First, there’s a link to an external JavaScript file hosted on admin.brightcove.com (line 2314). So, we know that the Times uses Brightcove for at least some of its viewer-facing video services. Also on line 2314 is a link to an external JavaScript file called “AvalancheDeploy.js.” This, clearly, is where the magic happens.

Like the external CSS file, this JavaScript has been minified to save download time. But while the comments inside the scripts have been removed, the comments describing the scripts, identifying the authors, and giving copyright information are still there.

Copying the source into a JavaScript-aware text editor makes it easier to read, and when pretty-printed it turns into 25,380 lines of nicely-formatted code. What it contains is 13 different libraries and library plugins concatenated into one file, again to save download time, plus the actual page-specific JavaScript that was custom-written for “Snow Fall.” In order, these are the libraries and some related plugins, along with what I know about them:

  1. jQuery. The most popular JavaScript library, jQuery greatly simplifies the work of making dynamic user interfaces. As an example, this is one way to make all the elements with the class “note” have red text using native JavaScript: 
    function changeColor(matchClass,color) {
    var elems = document.getElementsByTagName('*'), i;
    for (i in elems) {
        if((' ' + elems[i].className + ' ').indexOf(' ' + matchClass + ' ')
                > -1) {
            elems[i].style.color = color;
        }
    }
}
changeColor('note','red');

    But jQuery does all that stuff in the background and provides a simple, logical, syntax that draws some inspiration from CSS syntax:

    $('.note').css('color', 'red');

    This example shows how, as with CSS reset or normalize libraries, increased overhead in page download times can be an acceptable price to pay for the efficiencies gained in development time.

  2. jQuery UI. jQuery UI is a library of user experience plugins written on top of jQuery. What’s especially nice is that it can be downloaded with as many or as few of the plugins as necessary. So if you all you need is a cool datepicker, you can start with that, then later download another copy with a datepicker and collapsable accordion sections as your site grows.
  3. MediaElement.js. MediaElement bridges the current chaos of support for HTML5 media elements by detecting a browser’s capabilities and, if necessary, providing a tiny Flash player that mimics the HTML5 <video> and <audio> APIs. Browsers that don’t support the video format being sent to them can use the plugin instead, but the developer doesn’t have to do any extra work. This ensures that even viewers on Internet Explorer 7 or 8 will be able to see HTML5 <video> elements, despite Microsoft not having included support for them in those versions.
  4. MediaElementPlayer.js. A companion library to MediaElement.js, this lets you control HTML5 video styling and hook into playback events. All the controls are in HTML, even when the video is playing in the Flash fallback for browsers that don’t like the video codec being sent to them. (When we switched video solutions at work to meet new FCC requirements for online closed captions, I picked MediaElement.js/MediaElementPlayer.js and have been very pleased with the results.)
  5. The next library is called “The New York Times Multimedia Desk.” In its minified form it contains no comments to describe what it does, but clearly it was developed by the Times. Rather than try to figure it out, let’s just move on…
  6. Underscore.js. Maybe you thought we were into really geeky stuff already, but now here’s Underscore. Where the primary purpose of jQuery and MediaElement is to make it easier to manipulate the display, Underscore goes deeper: It’s primary purpose is to make it easier to access the data inside the page, essentially treating HTML content the same way database administrators treat data in a database. So you might use Underscore to select very specific parts of a page in ways that jQuery can’t and which would take enormously complex functions in native JavaScript, then pass those parts off to jQuery to do something cool in the display with them.
  7. Track Device Orientation. This library looks like it was developed in-house to detect whether a mobile device is displaying the page in vertical or horizontal aspect. A note says it is “Based very heavily on: http://davidbcalhoun.com/2010/dealing-with-device-orientation.”
  8. Modernizr. As browsers adopt the new features available in HTML5 and CSS3, it can be hard to remember what you can do in each one. Modernizr provides a way for the page to check what the browser is being displayed in is capable of. In this example from the Modernizr site, the script is used to check support for HTML5’s Geolocation API and download different external JavaScript files depending on the answer: 
    Modernizr.load({
  test: Modernizr.geolocation,
  yep : 'geo.js',
  nope: 'geo-polyfill.js'
});
  9. jPlayer. jPlayer is another solution for supporting HTML5 <video> and <audio> playback across browsers, like MediaElement. A question that springs immediately to mind is “why are they including two libraries that do essentially the same thing?” (Unlike MediaElement’s lightweight Flash+HTML fallback, jPlayer’s Flash fallback solution for browsers that can’t natively play what they are being sent is a large, entirely Flash, player. Its API, while powerful, is also complex and it has no built-in support for displaying closed captions. All of these reasons combined are why I chose MediaElement.js over jPlayer for KCTS9.org, despite jPlayer’s wider adoption and developer network.)
  10. Mustache.js. This is a templating library that can take data formatted as JSON (JavaScript Object Notation) and turn it into HTML on the fly. For example, this data 
    {
  "firstname": "Brook",
  "lastname": "Ellingwood"
}

    and this template 

    Written by <strong>{{firstname}} {{lastname}}</strong>.

    produce this HTML:

    Written by Brook Ellingwood.

    jQuery and Underscore are both capable of some templating as well, but as a dedicated templating library Mustache has an especially powerful and easy to use syntax. (Several years ago, working at a $2 Billion retailer, I selected XML/XSLT as the data and templating formats for the company’s ecommerce site. If I were doing that work now, I’d likely choose JSON as the data format and something similar to Mustache for the templates.)

  11. SWFObject. This library greatly simplifies embedding Flash movies in pages, by smoothing out idiosyncrasies between browser and such.
  12. ImagesLoaded. This is a single purpose jQuery plugin that detects whether the images on a page are fully loaded or not, then triggers whatever JavaScript you want to run if they are. Basically, it makes the experience for the user smoother, so you don’t see things bouncing around all over the place as the page loads.
  13. ImagesLoaded. Hey, wait a minute! They put that one in here twice. Whoops. Moving on…
  14. jQuery Fixed Div (not really!). For whatever reason, the last big chunk of JavaScript on the page is identified this way: 
    /*
 * jQuery Fixed Div plugin v1.0.0 <https://github.com/rwbaker/jQuery.fixed/>
 * @requires jQuery v1.2.6 or later
 * is released under the MIT License <http://www.opensource.org/licenses/mit-license.php>
 *     very HEAVILY MODIFIED by Jon Huang
 */

    Maybe the 56 lines of code that make up jQuery Fixed Div truly are buried somewhere inside all this JavaScript (when pretty-printed, it comes in at 1,460 lines), but I’m not looking for them. What I do see in there are lots of references to things with the word “avalanche” in them, so I know this is the real deal. I’m also not going to go digging around and try to figure it all out: Without comments it would take forever, frankly some of it is probably beyond my technical comprehension, and the real test now is looking at it in action on the rendered page.

    What I will do is say that if Jon Huang (@huang_apiaries), Multimedia Producer for the New York Times, built all of that JavaScript, he should give himself a clearer credit because he deserves it. (Check out what information design guru Edward Tufte said in this tweet — there is no higher source of praise in my book.)

In doing my breakdown I wasn’t always able to tell if any given library is being used. For example, in the page’s custom JavaScript I can see functions that dynamically create HTML5 <video> elements with classes for use by MediaElement.js , but I don’t find that HTML anywhere in the rendered page. I was however able to set my browser up so that instead of using HTML5 <video> the page performed a fallback to the Flash player supplied by Brightcove. It may be that all these libraries are used, but not all of them are used in every browser.

After digging through all those CSS and JavaScript tools, what we have left is just a few bits of housekeeping related to analytics tracking and such (lines 2,323 – 2,342). Now, it’s time to turn to the page as its rendered in the browser and see how those tools are used.

Beautiful Way: What Happens When the Page Renders

In the section on tools, I provided a copy of the page source the way it is when it is sent from the New York Times servers. This is very different from what the source looks like by the time you see it in your browser, unless you happen to be someone who has their JavaScript disabled and I’m betting you aren’t.

In the rest of this section, I will be using the Developer Tools built into the Google Chrome Browser to examine how the HTML is modified by JavaScript during the page load and what the modifications do to create the experience.

Mixed Bizness: Top Level Classes

Remember way back in the section on conditional comments when I said that if you are using a browser other than Internet Explorer, or a version of Internet Explorer higher than 9, the opening <html> tag would be written like this: <html class="no-js">? Well, after the page renders with JavaScript, it looks like this:

<html class="js csstransforms nytmm_avalanche_desktop nytmm_avalanche_production wf-nytcheltenham-n2-active wf-nytcheltenham-i2-active wf-nytfranklin-n5-active wf-nytfranklin-n7-active wf-active nytmm_orientation_landscape">

All those different classes have been added by JavaScript and tell the CSS and JavaScript important things about how the page is being displayed in the browser. What sort of things? Well, with just a little further examination we can pretty confidently guess at all of them:

  1. “js” is what used to be “no-js” because now we know the browser supports JavaScript.
  2. “csstransforms” likely means the page has passed a test in Modernizr for supporting not-yet-standardized CSS Transforms, which allow for altering the appearance of text in a variety of ways, including rotating and skewing.
  3. “nytmm_avalanche_desktop” presumably means that they can tell I’m using a desktop operating system (Mac OSX) and not mobile operating system (iOS, Android, etc.). No doubt there is certain formatting that will be set based on this.
  4. “wf-nytcheltenham-n2-active,” “wf-nytcheltenham-i2-active,” “wf-nytfranklin-n5-active,” “wf-nytfranklin-n7-active”: When supported by the browser, the page uses web-fonts, meaning it doesn’t rely on the fonts you already have on your computer. Inside the page-specific JavaScript, there are clues that tell us the Times is licensing web fonts with Adobe’s Typekit. To see them in use (assuming your browser supports web fonts) the heading that reads “Snow Fall” is in “nyt-cheltenham” and the byline “By JOHN BRANCH” is in “nyt-franklin.” These class values let the page know that the fonts are in use and likely affect spacing and other properties in the CSS.
  5. “wf-active”: By poking around in the page-specific JavaScript so more, it looks like this means all the web fonts have been identified, downloaded and are now being displayed.
  6. “nytmm_orientation_landscape”: The screen I’m using is wider than it is tall. I guess I could turn my MacBook Pro on its side, but unlike my iPhone it won’t magically switch to a portrait view. Drat.

Chain Reaction: The Header and Navigating Through the Story

The first actual content inside the <body> element is the header bar that stays fixed in position at the top of the page. This is a nice application of the CSS position:fixed; property and value. Wherever you are in the story, you have the links to all the story components right there at the top.

Those links to the different story components are worth describing. In the raw source, all the sections of the story are visible. In the source, the links are simple anchors that move you up and down without leaving the page, so that when you click the first one the URL http://www.nytimes.com/projects/2012/snow-fall/ becomes http://www.nytimes.com/projects/2012/snow-fall/#tunnel-creek. It’s very simple.

But with JavaScript, the behavior is made more interesting. On page load, all the sections of the story but the first one are immediately hidden. A JavaScript “event listener” has been attached to the links, “listening” for the user to click on them. When that click happens, the link is rewritten so that our example becomes http://www.nytimes.com/projects/2012/snow-fall/#/?part=tunnel-creek and the right section of the story somehow appears.

If you clicked the link ending with #tunnel-creek, you might have noticed that it was redirected to yet a third variation on this URL. But really all we really care about is that in all three versions there’s a hash (“#”) followed by some more stuff.

The way URLs work, the web server stops paying attention to anything that comes after a hash. So, no matter what version of the URL you click on, the web server is going to send you the exact same stuff. Your browser, on the other hand, does read the stuff after the hash — if it matches the “id” attribute on an open tag on the page, say <div id="tunnel-creek">, the page will automatically jump to that point. (As an aside, another way to tell a developer who hasn’t been paying attention for over a decade is if they do this sort of anchoring with the obsolete <a name="..."> syntax…)

If the hash doesn’t match any “id”s, the browser does nothing. But that hash is still up there, where it can be read by JavaScript. This is why the event listener takes a hashtag that does link to an existing “id” and breaks the link by rewriting it so it no longer works directly, but can be used to dynamically change what content is visible. If you don’t have JavaScript, you’ll just jump to the section you want to read. But if you do have JavaScript, the section you are on will disappear and the section you want will appear with visual effects designed to enhance the mood of the story.

Where It’s At: The Video Cover

The thing that first grabs viewers of “Snow Fall” is the video cover for the “Tunnel Creek” section: Wind silently blows across a snowy landscape, devoid of color against a mottled gray sky, and plumes of ice crystals stream off the surface. It takes a moment before the words of the story’s title/byline hovering against the sky are even noticed. In fact, you wonder if they were even there at first. There is no other text visible. The effect is far more cinematic, like the opening credits in a documentary, than it is like anything we associate with the word “newspaper.” It’s a stunning effect. So how is it done?

Snow Fall Section 1 Cover Video
Illustration 1 (Repeat): The Cover Video for the First Section of “Snow Fall”

There is no <video> element in the page source. Instead, there is a generic <div> element that has no contents, so there’s nothing to be seen when the page loads. HTML5 has introduced new “data-” attributes, which let developers store data in attributes they can give any name they want as long as it starts with “data-” and this element uses a number of them to store pointers to two different versions of the video that work on different types of browsers, with two different encodings as well as a pointer to a still poster image and instructions on looping during playback:

<div class="nytmm_fs_video nytmm_touch_fs_video nytmm_touch_fs_leadphotoseq nytmm_ie_static" data-mp4="http://graphics8.nytimes.com/packages/video/multimedia/bundles/projects/2012/AvalancheDeploy/chapter1_new.mp4" data-webm="http://graphics8.nytimes.com/packages/flash/multimedia/bundles/projects/2012/AvalancheDeploy/chapter1_new.webm" data-poster="http://graphics8.nytimes.com/packages/images/multimedia/bundles/projects/2012/AvalancheDeploy/chapter1.jpg" data-loop="true"></div>

So there’s a bunch of useful information here, but that’s not the same thing as displaying a video. When the page is rendered and the JavaScript has run, that <div> still is there, but now inside it there’s also this element (as rendered in Chrome for OSX):

<video style="position: absolute; z-index: 1; top: 0px; left: 0px; min-width: 0px; min-height: 0px; width: 1276px; height: 717px;" poster="http://graphics8.nytimes.com/packages/images/multimedia/bundles/projects/2012/AvalancheDeploy/chapter1.jpg" preload="none" src="http://graphics8.nytimes.com/packages/video/multimedia/bundles/projects/2012/AvalancheDeploy/chapter1_new.mp4"></video>

Chrome loves the HTML5 <video> element, so that’s all it takes for me to see the video in my primary browser. But if I was using an older browser that couldn’t handle the video natively, I’d certainly be seeing the object and/or embed code necessary for falling back to a Flash player, perhaps added directly with SWFObject, or indirectly with MediaElement or jPlayer.

Because the “controls” attribute is not present in the element, the video displays without them. You can’t pause it, or get distracted by a progress bar. It just sits there and loops.

Quite a bit of CSS for this element is written inline using the “style” attribute. The “position,” “top,” and “left” properties basically say “pin this to the upper left hand corner.” There are some nerdy details about the difference between “fixed” and “absolute” positioning at play, but let’s save those for some other time. The “z-index” determines what layer number to assign to it — more on that in a minute. If those “width” and “height” numbers seem a little random, it’s because they are. They happen to be the size of the viewable area on my browser after I got done making it bigger and smaller a few times just to watch the numbers change. There’s a JavaScript event listener set up for window resize events and every time the window changes dimensions, the viewport (the part of the window that displays the page) gets measured and new values are set for the size of the video cover. (The 0px “min-width” and “min-height” values explicitly state that the window can be sized all the way down to the point where it can’t be seen, but I’m not really sure there’s a need to specify that.)

Now, back to that “z-index” property. The <div> element that contains the title/byline text has a “z-index” of “2.” Couple that with some positioning properties and you get text that sits one layer above the video and adjusts to changes in window size as well. Farther down in the text, something caught my eye. For some reason, the single letter “F” in the word “Fall” is wrapped in a <span> element. The reason is just so that one letter can have its letter-spacing property adjusted by 3 pixels without affecting the rest of the title. Turn that spacing adjustment off and the “F” is a bit awkward, just a little too far away from the “a” that follows it. You can’t get into this sort of single letter typographical tweaking on just any project, but when you’re looking to wow the world it can be worth it.

When the page is loaded, a simple animation of the CSS “opacity” property quickly fades in the video cover and title/byline text. This effect, which is certainly accomplished with jQuery, contributes to the cinematic feel and the sleight of hand which draws the viewer’s attention to the video first, only realizing there’s text after being mesmerized by the blowing snow.

All the other sections, except “The Descent Begins” feature video covers with similar effects.

Puttin’ It Down: Scrolling the Text

When the viewer starts to scroll the page, the video cover gets covered up and the title/byline text go away. The first bit is just another application of “z-index” in CSS, but it gets a little tricky. To see how this works, it’s important to understand child and sibling relationships. Here’s an illustration that demonstrates the relationships, but otherwise has nothing to do with the actual code:

<div id="section">
	<div id="video-and-title-container" style="z-index:1;">
		<div id="video-container" style="z-index:1;">
			I am a child of video-and-title-container and a sibling of title-container
		</div>
		<div id="title-container" style="z-index:2;">
			I am a child of video-and-title-container and a sibling of video-container
		</div>
	</div>
	<div id="article=text" style="z-index:1;">
		I am a sibling of video-and-title-container
	</div>
</div>

(In reality none of this “z-index” stuff is going to work unless we also set “position” values, but that would just make the example needlessly complex. We’ll come back to “position” in a bit.)

Video-container and title-container have “z-index” values of “1” and “2” respectively, which affects their stacking order in relation to each other. But they both are children of video-and-title-container and, because it has a “z-index” value, the stacking order of its children only matters in relation to each other. Title-container might have a “z-index” of “2” but as far as article-text is concerned, the only “z-index” it sees is the “1” assigned to “video-and-title-container.”

Article-text is a sibling of video-and-title-container, so the “z-index” relationship between the two is independent of what’s going on inside video-and-title-container. But wait… if they both have a “z-index” of “1” how come article-text can be scrolled to cover video-and-title-container? That’s because article-text comes after video-and-title-container in the source and, unless otherwise specified with “z-index,” things that come after sit on top of things that come before.

Now, if you’re really paying attention, you’ll be wondering why it’s even necessary to specify “z-index” at all in this example, since the things that come after are already the things we want to cover the things that come before. I can’t be 100% sure of all the reasons it’s being set explicitly, but I can think of one really good one: In most, if not all, browsers the Flash plugin wants to sit on top of the HTML no matter what order it appear unless you assign it a “z-index” value. By having “z-index” properties on these HTML elements, the code is prepared to deal elegantly with the Flash plugin in case the browser can’t handle the HTML5 <video> element natively.

Scrolling the article doesn’t just cover up the video cover and title/byline text: They visibly fade out as they gets covered. There’s another event listener set on the “scroll” event. Every time the viewer scrolls, it triggers a function that does something along the lines of dividing the position of the top of the article body text container in pixels from the top of the viewport by the height of the viewport, then uses the result as the value of the CSS “opacity” property. Opacity is set on a scale of 0-1, with .5 being half opaque, so this doesn’t quite fade the lettering out completely before it’s covered. It would be easy enough to tweak the math so it did, but in this case the effect works better by letting the letters stay visible until they are covered by the white background of the text, as if by snow.

After having said all that, when I do this on Chrome the video cover isn’t fading out at all, even though I can watch the “opacity” value change. If I manually set the opacity to “0” it disappears, but with any other value it displays as though the value were “1.” I assume this is a browser bug with the <video> element the Times decided to live with.

On reaching the bottom of each article section a full-width bar moves up in a simple animation, offering the viewer a text link prompt to proceed to the next article section or go back to the previous one. All the things that happen in between the video cover and this prompt will be described below.

Atmospheric Conditions: The Article Layout and Components

The article sections are divided into two columns. The one for text on the left is set to “width:100%;”. The one on the right is mainly for image boxes and is set to “width:348px;” with “position:absolute;”. As I hinted at in the previous section, “position” is related to “z-index.” If an element is given a “position” value, it automatically has a “z-index” value as well, even if it hasn’t been explicitly set. So what’s happening here is that because the right column has “position” and comes after the left column in the HTML it isn’t squeezed out of the two columns’ parent element by the left column’s “width:100%;” — instead it has “z-index” and sit on top of the right hand side of the left column.

Because the left hand column is filled with things like <p> paragraph elements, you might expect them to fill the whole width of the page with text. But in the CSS we find this:

.nytmm_article .left p {
	...
	padding-right: 50px;
	margin-right: 348px;
}

These properties apply to every <p> that’s a descendent of an element with a class of “left” that’s a descendent of an element with a class of “nytmm_article.” The “margin-right” value is the same as the width of the right column — because the text can’t go into the margin, it can’t cover any of the right hand column. The 50px of padding gives a little whitespace gutter between the two columns. (The “…” is where I removed some font styling properties that weren’t relevant to this example.)

No Complaints: Inline Video

Scrolling through the “Tunnel Creek” section, the first media element we come to isn’t an image box on the right, but a video embedded in the text on the left. Several videos are embedded the same way in other sections.

Illustration 3: Inline Video
Illustration 3: Inline Video

The embedding itself is very straightforward: In a series of <p> paragraph elements, a sibling <div> holding the video has a “float:right;” CSS property value, and the text adjusts to wrap around it. Because the <div> has the same margin and padding properties as the <p> elements, the video displays as part of the left hand column.

As the video comes into the viewport, it has an “opacity” value of “.4” or 40%. Then as it reaches the center, an event listener changes that to “1” or 100%. But it doesn’t just pop to the new value, it does a half second fade to it using a CSS Transform.

Although CSS Transforms are still not a standard, they do work on most current browsers through the use of browser extensions. These are a way for browser makers to add support for new techniques, before everyone involved has agreed on how to use them. The styling for the video includes the proposed syntax

transition: all .5s ease;

which means “if any of the CSS properties applied to this element change, fade the changes in over half a second and ease the fade so that it starts fast but slows down as it gets closer to being complete.” But because browser support for the generic property is not yet guaranteed, the same instructions are given using browser extensions:

-moz-transition: all .5s ease; /* This works on the Mozilla render engine in Firefox, SeaMonkey, etc. */
-webkit-transition: all .5s ease; /* This works on the WebKit render engine in Chrome, Safari, etc. */
-ms-transition: all .5s ease; /* This works on Microsoft Internet Explorer */
-o-transition: all .5s ease; /* This works on Opera */

It may seem silly because the syntax for all of them is exactly the same, but browser extensions have proven to be very useful for advancing the state of the art and are highly appreciated by anyone who tried to push the envelope of web experience during the early 2000s when browser feature development was at a standstill. This way, developers can take advantage of new features without worrying their pages will break in the future.

The video’s caption and play button also fade in, going from “opacity:0;” to “opacity:1;” but the transition time is a full second. Using staggered transitions further enhances the dynamic feel of the piece.

Inside the text, words describing the video are highlighted. Click on the name and the video starts playing. This requires JavaScript, and the solution for doing it elegantly ensures that the “no-js” version of the site doesn’t contain any extraneous code. In the source, the name is wrapped in <span class="nytmm_video_trigger">. Nothing in there says which video to trigger. Instead, what I believe happens is an application of JavaScript arrays.

For the purposes of this post, an array can be thought of as a list of items. Now imagine two arrays: One listing all the <span class="nytmm_video_trigger"> elements and one listing all the <div class="small_video_section"> elements. Then event listeners are attached to the triggers so that when they are clicked they tell the videos in corresponding list positions to play: Trigger #1 plays video #1, trigger #2 plays video #2, etc.

In actuality, there might only be one array with nested data about both the triggers and the videos, but the basic idea is the same. There’s also a strong likelihood that at some level, jQuery’s simple syntax is used to make the array. These are the sorts of details that are hardly worth digging into though — what’s important to understand is that if you have a data model in mind when you create your base HTML you don’t have to hardcode relationships between objects in order to have them dynamically interact.

Feather in Your Cap: Showpiece Videos

A few more paragraphs into the story, there’s a stunning bit of multimedia. A full-screen width video interrupts the text flow and a three dimensional landscape appears, fading in even as the point of view flies over the tree and snow-covered crags around Stevens Pass.

Google Earth vs. NYT
Illustration 4: Google Earth, leftt; NYT, right

While the effect is new, the code behind it is all things that we’ve just been looking at. The video data is stored the same way in attributes of a <div> element and can’t be seen if you don’t have JavaScript. Once turned into an HTML5 <video> element, it’s width and height are set to fill the viewport and as it scrolls into view its “opacity” value changes so that it fades in.

The real stunner is that as it scrolls into the view, the background of the story’s text fades to nearly match the gray of the clouds and fog at the top of the video frame. The border between the written story and the video all but disappears. More surprisingly, the text has become deemphasized — it’s still legible, but in essence this is one of the world’s preeminent newspapers sending a signal that you need to stop reading for a minute and really take in the video you’re about to see. If there’s a single game-changing takeaway to be found here, that might be it.

Impressed by the video, I tried to recreate it’s final perspective in Google Earth. It’s possible to get close to the same perspective the video ends on, but the effect is very different. This is where we tip our hats to Jeremy White (@blueshirt, the graphic designer at the times who generated this video as described in a Poynter article on the piece. Obviously, for an article set in the Cascades in winter, there needs to be snow and the Google Earth view has no snow. But what really makes the video work, both to set the mood and to focus attention on the relevant landscape, is White’s application of fog and clouds to fade the distant landscape out of view. In the Google Earth image, the eye is drawn, not to Tunnel Creek, but to Mount Baker on the far horizon.

This video presentation style will return several times throughout the article, illuminating and expanding on the article’s text. Here’s the complete list (the names are mine, not the Times‘):

  • “Tunnel Creek”: Flyover Video
  • “Tunnel Creek”: Satellite Map
  • “To the Peak”: Snow Conditions
  • “To the Peak”: Hike to Tunnel Creek
  • “Blur of White”: Avalanche in Real Time

Maximum Potential: Dynamic Background Image

The showpiece in the “The Descent Begins” section isn’t video, but a still image that appears to change to show the different routes taken by the skiers as the text scrolls. Because the text overlays the image, I’m describing it as a background image, but it shouldn’t be confused with the CSS “background-image” property. This image is basic HTML, with CSS being used to position the matching text on top of it.

Dynamic Background Image
Illustration 5: Dynamic Background Image

As the text is scrolled, the new images consisting of orange lines on a transparent background are positioned on top of the background image. Sprinkled throughout the text in the page source are data pointers that look like this: <span class="reel_tag" data-start="51"></span>>. The “data-start” attribute values correspond to the names of images showing routes taken down the mountain.

What’s actually in the page source is very different from what I’ve described. There, as a fallback, are images of the mountain and the descent paths combined. It seems that on page load, the JavaScript reads the “data-start” attribute values and generates calls to all the matching images, positioning them so they will line up properly with the background. Then, it listens for the “scroll” event and every time the page scrolls it checks to see if a <span> with a pointer to an image has made it to the middle of the viewport. If it has, the corresponding image is made visible.

Along with the images, individual slideshow launchers that have already been used on earlier pages make a reappearance, slightly altered in looks for use on this page. Somehat surprisingly, this duplicate data is actually in the page source twice instead of being reused with JavaScript. It’s hard to know why this is the case: Perhaps there was a blocking problem that couldn’t be solved otherwise, perhaps its to meet a design requirement for viewing without JavaScript, or perhaps with a looming deadline it was the quickest way to wrap the work up.

Half & Half: Column-Spanning Images and Video

In the “Tunnel Creek” section, there are two historical images of the 1910 Wellington Avalanche, side by side. After all the dynamic multimedia experience above, it’s almost startling that they just sit there. They introduce a third layout element to the story by extending from inside the left column all the way across the right column. This is easily accomplished by leaving the “margin-right:348px;” and “padding-right:50px;” declarations out of the CSS for their shared parent <div>.

Column-Spanning Images
Illustration 6: Column-Spanning Images

Variations on this simple still photo layout occur throughout the entire article, including the cluster of images halfway through the “To the Peak” section which are large enough that there is no room for text to wrap around them.

The next media element in the “Tunnel Creek” section is a large, but not full-width, video box that also spans the columns. As it scrolls into view it fades in like the inline video described above, with the only styling difference being the addition of 1px borders above and below. There’s a cool effect difference though: Once the video has faded in, a very short introduction video plays. Once it’s finished, a title and play button encourages the viewer to play the whole thing. In the raw page source, the data for the two different videos is stored as detailed above, but the <div>s are children of the same container.

I suspect that when the JavaScript that turns these into <video> elements finds this situation, it knows to set up a sort of playlist in which one video playing to its end automatically sets up the display of the next video — with enough poking around it would be possible to figure out how exactly it works, but the real takeaway is that HTML5 <video> has a number of events that you can use event listeners with and one of them is the event “ended.” So when a video finishes, it’s pretty straightforward to make something else happen.

A video presentation in the “Discovery” section is very similar, but lacks the autoplaying introduction.

Outcome: Slideshows

Individual Profile Slideshow Launchers
Illustration 7: Individual Profile Slideshow Launchers

So far in scrolling through the “Tunnel Creek” section, the right column has been sitting there, not doing anything except sometimes letting itself be covered up by images and video spilling over from the left column. But now as we get to the point in the story where a number individuals involved in the events are introduced, launchers for photo slideshows of the individuals are put into the right column, linked to highlighted names in the text in the left column. When the launcher is highlighted, its top border precisely aligns with the top border of that person’s name in the text.

This positioning is set with JavaScript. Each launcher is a <div> with a custom “data-tag” attribute containing an individual’s name. The names in the text are marked up as <span>s with matching “data-tag” attributes. This makes it fairly simple to measure the position from the top of the name in the left column and then set the position from the top of the launcher to be the same.

As more individuals are introduced in the “To the Peak” section we find cases when two names are so close to each other in the text that the launchers would overlap so they are stacked instead. One way to do this might be to save the last position measured and subtract it from the position currently being measured — if the result is less than 100 pixels, then don’t match the positioning of the launcher to the text, but simply position the launcher 100 pixels below the previous launcher instead.

Standard Slideshow Launcher
Illustration 8: Standard Slideshow Launcher

When the launchers are clicked, matching slideshows appear in the familiar “lighbox” overlay style. But none of the data used in the slideshows is in the page source. The slideshows are generated and destroyed as necessary using data supplied through Ajax calls back to the server. (Ajax originally was an acronym, but now it’s just become a name.) When you click a launcher, it retrieves data formatted in the JSON sytnax, then parses the data and turns it into the HTML that makes up the slideshow.

Several sections in the story include inline slideshows, launched from large poster images that span the left and right columns. Like the individual profile slideshows, these slideshows are generated as needed by retrieving and manipulating JSON data.

Painted Eyelids: Audio and Miscellaneous Video

Embedded Audio Player
Illustration 9: Embedded Audio Player

Right Column Video
Illustration 10: Right Column Video

The “Blur of White” section contains embedded audio of phone calls made to 911. Like the HTML5 <video> pointers to these pieces of media are stored in the page source in a way that they won’t be visible to browsers without JavaScript. When the page is rendered, they are turned into <audio> elements with controls and styling provide by the jPlayer library.

Both the “To the Peak” and “Discovery” sections contain small videos in the right hand column, illustrating avalanche air bags and the path taken by the avalanche respectively. They are nice pieces of design, but other then being positioned entirely on the right don’t differ technically from the other video elements already described.

Beat Surrender: Strategic Conclusions from a Technical Breakdown

Near the end of the article, there’s a link to buy an ebook version of the story. It’s an interesting strategic business choice for a newspaper, and it also reflects the fact that the popular ebook formats all use a variation of HTML for layout. Once you’ve published to the web, it isn’t that much more work to make a variety of ebooks for different devices.

At the very end are the credits for the piece. Tellingly, the names of graphic designers and multimedia producers who wrote code are all mixed together in a single “graphics and design” credit with 11 names. This is a collaborative production that is only possible with a skilled staff working together: Everyone has specialties, but the boundaries have to be blurred to make a unified experience like this.

The disruptions that are rocking print and broadcast business models are hardly secret — they are prominently displayed all across the Internet. I’ve been reminded of a book we passed around in the dotcom era and which I recently started to reread. Some took “The Innovator’s Dilemma,” as an entrepreneurial blueprint for identifying dinosaur companies to take market share from. I took it another way and have for much of the last decade been applying what I’ve learned as a disrupter to making established businesses stronger from within. But at times “change agent” can be a trying path to follow.

You can’t make someone change who doesn’t want to change. And most media companies haven’t yet accepted what “change” really means. They want to use new media to protect their old media business. They still think they can force the Internet to adapt to them. Unfortunately, it didn’t work out well for the record companies, it’s not working out very well for the newspapers, and the strain is starting to show in the television industry.

With this one article, the New York Times has conclusively demonstrated it understands real relevance in the digital age won’t come entirely from skeuomorphic digital newspaper apps for mobile devices or a highly-porous paywall. Those may yet prove to be elements of a successful business plan, but they can only help if the Times continues its investment in presentations that are designed and developed to take advantage of the unique opportunities offered by interactive media. Other media companies would be wise to follow suit.

Leave a Reply