by Jaap Blom, Sound & Vision (NISV)
On 25-26th of February, the EuropeanaTV team organized a workshop at Rundfunk Berlin-Brandenburg (RBB) in Potsdam to work out a SmartTV app. It had to use RBB content about the Berlin wall together with related materials from Europeana.
The idea was to design the app using the LinkedTV editor tool, developed by Sound and Vision R&D, and Noterik’s multiscreen toolkit. The outcomes prepared us for the full-blown EuropeanaTV hackathon, which will be held at Waag Society in Amsterdam on 8-10 May as part of the Europeana Space project.
Day 1: Working on the concept
After a round of introductions, Annette Wilson from RBB started the day by presenting RBB’s initial ideas for the Smart TV app. Their idea was to extend an already developed Berlin Wall HbbTV app with related content – preferably videos – from Europeana. An editor from RBB would use the editor tool to select a number of RBB videos, then select a number of Europeana videos and publish these to the HbbTV app.
The first thing that we noted was that both the capabilities of the editor tool and the multiscreen toolkit could not be fully utilized in the proposed scenario. The editor tool is not currently optimized for selecting, but for the annotation of single videos. Furthermore, the multiscreen toolkit supports HbbTV, but is more interesting to use for applications that involve the interaction with multiple screens.
The Editor Tool
Right from the beginning there was ample ground for discussion, but before we went into that, it was time for us to introduce the editor tool and its capabilities.
In a nutshell, the editor tool is an annotation toolkit that can be configured, or extended, to provide an online dashboard for editors to annotate audiovisual content by either searching for related content through APIs (such as the Europeana API) or by simply providing a desired link and/or information.
To support the editor, the tool supports loading various kinds of pre-generated metadata to enhance the options for segmenting and annotating content. For example, when connected to the LinkedTV platform, editors using the tool can select named entities and keywords extracted from subtitles as input for queries to find related content – using for example one of LinkedTV’s enrichment APIs. Applying shot segmentations extracted from the video, the editor tool adds the option to select a range of camera shots to define chapter boundaries.
For this workshop, there unfortunately was no time to extract such information from the Berlin wall related videos. Fortunately these videos are mostly only a couple of minutes long and there was no need to segment each video prior to adding annotations. Having no access to automatically extracted keywords or named entities, the user in this case would be limited to using free-text search.
Example of a Berlin wall video, about Ronald Reagan is visiting West-Berlin, loaded in the editor tool Image credit: Sound and Vision CC-BY-SA.
Using the editor tool to search Europeana for content on ‘Reagan’ – Beeld en Geluid CC-BY-SA.
The Multiscreen Toolkit
Next up was the introduction of the multiscreen toolkit by Pieter van Leeuwen from Noterik. He explained that the toolkit enables programmers to quickly build prototypes to be able to quickly experience whether your billion dollar multiscreen application idea works or needs serious reconsideration. Considering the complexity of multiscreen applications, having access to this open source toolkit is great for testing out new ideas before starting a possibly expensive project.
Following the presentations, RBB showed the HbbTV app, implemented by Exozet, that was developed to browse the Berlin wall video collection using the remote control. The app supports navigating videos by selecting a decade, followed by a year in that decade, followed by cycling through a list of videos that fall within the selected year.
Shaping & Pruning Ideas
After the presentation of RBB’s initial idea – using the editor tool as a Europeana content selection tool and using the multiscreen toolkit to mimic the existing HbbTV app – the workshop participants decided to try and come up with an idea that would enable getting more out of the editor tool and multiscreen toolkit in order to be well prepared for the May hackathon, where Europeana Space offers participants will get the possibility to experiment with the same tools.
After some discussion, where we also took into account the feasibility of actually implementing the application the next day, we came up with the following idea:
Wireframes for the EuropeanaTV pilot, created using the Sqetch Toolkit Image credit: Pieter van Leeuwen CC-BY-SA.
In a nutshell, the idea is to extend the current HbbTV app with a new form of navigating the content, namely by selecting one of the tours available on the RBB web page dedicated to the Berlin wall. Instead of playing a single video, selecting a tour will result in the playout of a series of Berlin wall videos accompanied by related images from Europeana.
The related Europeana images are linked to each video by using the editor tool to search the Europeana API for images and having an editor select appropriate ones to be added as annotations.
To demonstrate the capabilities of the multiscreen toolkit, the user can select tours by using a second screen, which can be a tablet, phone or any other device with a browser. After making a selection the tour is played on the main screen, in this case the television; on the second screen the user can consult the location of the video on a map and inspect the related Europeana images in detail.
Day 2: Elevator Pitch & Implementation
On the second day we split up in two groups: RBB would focus on working out an elevator pitch to promote the app and work on searching Europeana to find content related to the Berlin wall. Noterik and Sound and Vision worked on the implementation, where Noterik would work on setting up the desired interaction model using the multiscreen toolkit as well as implement the user interface.
Developers from Sound and Vision enhanced the configuration options of the editor tool to enable the specification of certain content licenses when searching through the Europeana API. To help out, Sound and Vision also worked on writing a piece of code that could read the output of the editor tool and show it in the multiscreen toolkit. Unfortunately there was no time to integrate this by the end of the day.
To be in compliance with RBB policy, for the EuropeanaTV pilot the new configuration for searching the Europeana API was set to include all Public Domain licenses and the following variants of the creative commons licensing model: CC-0, CC-BY, CC-BY-SA, CC-NC and CC-NC-SA (Note that the non-commercial variants are also included as RBB is a non-profit organization).
After using this new configuration, we worked on annotating the Berlin wall videos of the first tour on the website. Being restricted to the aforementioned licenses we found out that there was not so much useful content remaining and we ended up adding mostly images from the Architekturmuseum der Technischen Universität Berlin in der Universitätsbibliothek.
The End result
The first end result of the day was the elevator pitch:
Concerning the app itself, the basic interaction was implemented, so that it was possible to click a link, representing a Berlin wall tour, on your second screen after which the main screen, i.e. the tv, would show the tour being loaded. Unfortunately there was no time to show the actual videos being played or the related Europeana content that was linked to each video in the editor tool.
Fortunately Pieter was so gracious to spend some time after the workshop to finish this fully functional result:
Summary & outlook
We can definitely say the two workshop days proved to be a very interesting experience where each partner learned about each other’s stand points and through discussion and working together, we ended up with an idea that was feasible to work out in a technical sense and something that could serve as the basis for a new RBB Smart TV application.
Lastly it was good to find out this experience gave us the confidence that the upcoming 3-day hackathon in May would definitely give participants enough time to come up with ideas and work them out into working prototypes, possibly using the multiscreen toolkit and/or the editor tool we developed.
More Info
- Read about the other EuropeanaTV prototype Rewind
- Read about the EuropeanaTV app Fall of the Berlin Wall
- Visit the Europeana Space project website for more info about the project
- Sign up for the EuropeanaTV hackathon here
This article originally appeared on Sound and Vision’s R&D blog.
