-
An Approach to Treating Videos
as Academic Documents
Stewart Arneil / Greg Newton
Humanities Computing and Media Centre
University of Victoria
-
What we're talking about
- Expectations for and capabilities of video documents and collections
- Implementations to provide support for academic tasks
- Where we think this is going
-
Definitions
What we mean by "Video" document
- video file plus structured data (TEI-XML) containing time-stamps
What we mean by "Academic" document
- Inclusion in accepted academic form: Lecture, Instructional Material, Critical Edition etc.
- Re-presentation with affordances for academic tasks: Discovering, Annotating, Comparing, Referring, Sampling, Illustrating, Representing
- Discovery: search engine, esp networked, conversation with others
- Sampling: selection (address relevant part of doc) based on a criterion
- Annotating: data attached to selection; [shared based on Web 2.0]
- Comparing: on any of a number of features of document or selection
- Referring: formal pointer e.g. link, stability
- Illustrating: assembling selections to make a point
- Representing: putting together elements in novel, enlightening way
- http://www3.isrl.illinois.edu/~unsworth/Kings.5-00/primitives.html
-
Our context
- Ideally concepts, code, and data are extensible, reusable
- New forms must be academically viable
- Copyright
- extension of, reused from; really;
- advance the state of the art (and maybe the state of the PI)
- To discuss this we needed to be able to ignore copyright issues
-
1) Online version of recorded lectures
Approach: Add value to playback + transcript [1] [2]
- Discovering - search collection or single video
- Annotating - by author, two threads
- Referring - to paragraph or event
- Comparing, Illustrating - bookmark lists
- Sampling - frequency counts in search results
- Representing - amenable to XSLT
- Search collection or document: ranked list of timestamp+utterance
- Transcripts (utterances, non-verbal, actions) + Events
- Bookmarks : list, save list
-
2) Collection of Francophone speakers
Approach: repurpose code to target instructional needs; also support Applied Linguistics research [1]
- Discovering, Comparing, Sampling - search with filters
- Annotating, Illustrating - by author with use of HTML features
- Referring - to sentence in transcript and video
- Representing - amenable to XSLT, spectral analysis
- Shorter utterances, text of all available
- pop-up explanatory notes
-
What we've learned so far
- These are performances, not "videos".
- The "added value" is always structured text.
- This type of effort is gaining credibility and legitimacy.
- There's a lot more we can do in this area.
Small discoveries
- We kept talking about video, but in a way, it's performances we're working with.
- The changes we were making were to XML files. We aren't editing the video beyond the most basic rough cuts.
- Each time we presented the concept in an academic forum, it was greeted with enthusiasm, but everyone saw something different. We saw this as a good thing.
- This kind of thing is gaining credibility and legitimacy.
- We are seeing DH types talking around the same core set of needs (TILE/AXE, THATCamp topics). It's just implementations that differ.
- We're only beginning to scratch the surface here.
-
So, now what?
- Performance and Film criticism
- Can we re-build it?
- Yes, we can.
- Framework guidelines: Unsworth's Scholarly Primitives
- Discovering, Annotating, Comparing, Referring, Sampling, Illustrating, Representing.
- Can we expand on the concepts introduced by Friedlander's "The Shakespeare Project" and change performance criticism?
- Traditional performance criticism is premised on the reader being familiar with both the play and performance/iteration
- No corollary evidence is expected by the reader, meaning they just take the critic's word as given.
- All reference is to, at best, a faded memory of the performance in the reader's mind.
- Static documents describing dynamic events are unable to fully encapsulate the performance for a reader. But the web can.
- A slightly modified version of Unsworth's Scholarly Primitives gave us a starting point for producing a spec for phase 3.
-
3) Specification for a Video Variorum
- Discover/Sample: searching/filtering/web services
- Annotate/Refer: multiple timelines and annotations - all referable by URL
- Compare/Re-present: Fluid, user-adjusted, saveable perspectives
- Illustrate: compare and contrast performances, or perhaps the critiques themselves.
- Discovery is not just internal any more, and web services extend the metaphor beyond the collection.
- We have no idea how you might want to interact with our information, but we can produce web services that will let you make that determination yourself.
- Sample: Interface exposes only available features, but allows traditional searching and filtering
- If we take as given that "comparison is one of the most basic scholarly operations" (Unsworth, 2000), how should we provide for this in the context of performance criticism? Tools need to be an empathetic reflection of a scholar's needs or they end up either producing flawed results or fundamentally changing the direction of research.
- fluid interface provides ample opportunity for the user to set up a "perspective" for comparison which can be saved and shared as URIs.
- multiple tiers/events and overlapping timelines, all "bookmarkable" and referable as URIs
- Tiers/events can include anything: media playback; transcriptions & canonical texts; annotations on lighting, costume, technique; animated sprites following objects or people
- "playback" of bookmarks can create dynamic, interactive arguments as well as provide exemplary detail for scholarly criticism.
- Traditional critiques can include links to supporting perspectives.
- With video extant, perspectives for old papers can conceivably be reverse engineered and re-examined.
- New critiques can utilise perspectives as native components of the argument.
-
More specifically: Implementation
- Frontend: standard XHTML
- Media Playback HTML5 video element, Flash, or both
- Backend: eXist/Cocoon
- Editor: oXygen, browser-based etc.
- Because we're only working with timestamps and text (even in the case of sprite overlays) we don't need anything fancy.
- eXist/Cocoon have well-developed methods for exposing a service for machine discovery/interaction
- Standards-based XHTML keeps it accessible to anyone.
- Using Flash in this case is not a problem. It's an agnostic application. But we *should* be able to do the same thing with the HTML5 video element. The problem for HTML5 video is in the codecs...
-
Conclusion
- Technically feasible
- Academic viability increasing
- Labour-intensive, but specifically requires digital humanities skills
- Copyright - the elephant in this particular room
- Video not as alien to academic treatment as might be thought
-
Thanks!
http://hcmc.uvic.ca/presentations/dh2009/