EPUB3’s Media Overlays: Synchronised Narration

Continuing from my previous post, where I talked about how audio in ebooks was of special interest to poetry publishers, it is worth drawing some attention towards the incorporation of ‘media overlays’ in the epub3 specification, whereby narration can be synchronised with text:

A pre-recorded narration of a publication can be represented as a series of audio clips, each corresponding to part of the EPUB Content Document. A single audio clip, for example, typically represents a single phrase or paragraph, but infers no order relative to the other clips or to the text of a document. Media Overlays solve this problem of synchronization by tying the structured audio narration to its corresponding text (or other media) in the EPUB Content Document using SMIL markup.

iBooks has supported this since June; here’s a video of it in action.  Its primary commercial application has been children’s books, although I suspect IDPF were thinking more about accessibility. However, this could also be useful for incorporating readings by the poet (instead of with embedded audio), as narration doesn’t have to be linked word-for-word but perhaps by stanza, & so you can ‘read along’ (like in Faber/Touch Press’ Waste Land).

There are examples of media overlay code in the specification, but the epub3 Project page on code.google.com has a sample epub3 file of Moby Dick (currently 9780316000000_MobyDick_r9.epub) which includes some linked audio files, if you want to have a look at this in practice.

This is one of the reasons why we should giving most of our attention to epub3.