Open Source Tools for Validation in the Digital Archive Workflow

By Merle Friedrichsen from the German National Library of Science and Technology (view original post)

 

On 9 / 10 November I was able to attend the “No Time to Wait 2”, a conference on open source software in (film) archives held at the Austrian Film Museum in Vienna. In contrast to the first “No Time to Wait” conference the scope was widened to audiovisual preservation, open formats, and standardization of formats. As part of the program, I had the chance to present on “Open Source Tools in the Digital Archive Workflow” from our perspective.

 

Requirements for Open Source Tools for Validation

When we receive a file for our collection we will test if it fulfills the requests of the standard (e.g. does the PDF-file conform to the PDF-Standards?). There are different scenarios in which we use open source software for validation, and from each of these different requirements occur.

One requirement is that the tool is easy to implement into existing software and workflows, as we want to use the validation software within our archive framework. If we have implemented the tool, we can validate the files upon ingest into out archive. In another business case, we require a simple graphical user interface (GUI). With such a tool the acquisitions team can test the files as soon as they are received, and maybe even ask for another (valid) file – if this is possible. When we are working together with other institutions or as a service provider, we typically receive a large amount of files which we validate before we ingest them into out archive. Due to this is we require a command line interface (CLI) in order to automate the validation of a large amount of files recursively through different folders.

Of course the validation software should fulfill its purpose: checking against the standard of the format. But apart from the standard, an institution might have other requests – for example a specific resolution in TIFF files for a digitization project or a rule against embedded objects other than images and AV in PDF/A files. In order to easily enforce these, the tool should be able to check the files against a custom policy. As we want to store the report (success or failure) of the validation as preservation metadata, the report should be available in XML or a different structured format, to make it easy to integrate or process the output further. Any metadata schema will be appreciated, as it makes it more readable and (if applicable) the mapping to any other metadata schema would be easier (or already existing). The performance of the tool is another requirement.

In an ideal world, we would only receive valid files from data producers and service partners. A step to reach this goal would be an easy way to allow external providers to validate their files before they hand it over to the library. A web service, where anybody could upload files and check them against the standard (and a custom policy) would serve such a purpose. If the file is not valid, a repair-possibility for the error encountered in the file would help the data producer/provider, to hand over only valid files.

So, in no particular order a short overview of the requirements have regarding open source software for validation (besides from being open source):

  1. easy implementation into existing software and workflows
  2. simple Graphical User Interface
  3. Command Line Interface
  4. checking against file format standards
  5. checking against custom policy
  6. use standards for reports
  7. perform (fast) on a large amount of files
  8. webservice (standards and policies)
  9. repair-possibility

We‘ve tested two open source tools for different file formats: veraPDF (PDF/A), and mediaConch (for Matroska/FFV1 – for film). I am very happy to state that all of our requirements are fulfilled (or are possible to fulfill) by these tools.

 

VeraPDF

We have tested most of the requirements and the tool performs well. As we receive a lot of PDFs that do not need to fulfill the PDF/A standard but nevertheless must be without password, we created our own policy that checks whether a pdf is password encrypted or not. What we haven’t tested yet is the integration into our existing software, but the implementation is discussed in the Rosetta Format Library Working Group (FLWG), a user group responsible for – amongst other things – deciding which tools should be rolled out within the Rosetta archive framework. Until now I haven’t seen a web service based on veraPDF, but as the software is open source and well documented, it should be feasible to build one. Due to the fact that we could not find a file that could be repaired by veraPDF, we haven’t tested the given repair functionality. But it is possible to fix the PDF document metadata, e.g. if a file does not conform to the standard, the PDF/A flag can be removed automatically.

 

MediaConch

Most of our requirements were tested and are fulfilled by MediaConch. There are only a few requirements that we have not tested yet. One of these is the implementation into existing software – as with veraPDF, the FLWG is also discussing the integration of MediaConch. On the other hand we do not have special requirements for mkv / ffv1 files yet, so we couldn’t test a custom policy – simply because we currently do not have a custom policy. Due to the implemented checksums in ffv1 and Matroska there are several “repair-possibilities” for a file with a bitflip. But as we haven’t encountered a corrupted file (yet), we haven’t tested this possibility.

 

Conclusion

Both veraPDF and MediaConch are suitable for our needs regarding digital object validation. I have to admit that I had a lot of fun testing these new tools, looking into the reports and figuring out how to write my own policy. It is worthwhile to work with open source tools – especially when they are designed to fit the needs of a (digital) archive. And it is worth investing (money and / or time) to foster or to enhance these tools!

If you are interested in the conference you will find the recordings on the youtube channel of the conference.

Thanks to the organizers of this conference – it was inspiring and encouraging!

 


Virtual reality re-creation preserves iconic Indianapolis church
Bethel ame church

Bethel AME Church. Credit: Indiana University

A 3D digital re-creation and preservation of Bethel African Methodist Episcopal Church, the oldest African-American church in Indianapolis, was performed by Indiana University and recently presented to the public.

Bethel AME Church has historically been the heart of the African American community in Indianapolis and played vital roles to support the abolitionist movement and the Underground Railroad, which provided protection to slaves en route to Canada. Education was also an important mission of the church, establishing the first school for African American children at the church in 1858, as Indianapolis’s public schools did not allow African American children to enroll. Bethel AME Church was listed on the National Register of Historic Places in 1991; at that time it was the only African-American church building in Indianapolis to receive that recognition. Because of deteriorating building, and without the major funding needed to proceed with the church repairs, the congregation began considering offers to buy the property and in early 2016 the building was sold to an Indianapolis-based developer, who agreed to preserve as much of the structure as possible, while planning to build a hotel on the property. The congregation was then moved to a new building.

In February 2017 Indiana University announced that its School of Informatics and Computing at IUPUI had received a grant from the university to create a three-dimensional virtual reality model of the historic Vermont Street church. IUPUI students collected 3,000 images for use in the modeling project, thus protecting the church’s history and legacy with 3D digital preservation techniques using virtual reality.

“Many people have visited or attended special events at the Bethel AME Church, once located at 414 W. Vermont St., and now they have the opportunity to revisit the church in virtual reality,” said Andrea Copeland, associate professor and chair of the Department of Library and Information Science.

“They will experience a unique method of preserving a historic place,” said media arts and science faculty member Zeb Wood. “Through the use of a VR headset, they will see the inside of the church’s sanctuary to scale. Participants will be able to move around inside of the church, and soon they will be able to interact with and learn about its members and their contributions to the city of Indianapolis.”

The church was 3D scanned for reference to become an interactive, informal and formal educational space that instigates innovative, collaborative learning opportunities.

About Indiana University: https://www.indiana.edu/

Read the press release on Phys.org.


Promoting Cultural and Creative Industries: Maximising their Economic and Social Contribution in the EU

Europe is the second largest Cultural and Creative industries (CCIs or Cultural and Creative sector, CCS) market in the world after Asia. A study by ‘Cultural Times’ estimates that European Union creates 32% of global CCI revenues and 26% of world-wide CCI jobs. Numerous surveys have proven that this field is one of the most sustainable in times of crises and can more easily adapt to the changing conditions. According to Eurostat, in 2015 the cultural sector constituted 2.9% of the total European workforce. The whole CCIs sector represents around 4% of European GDP and provides jobs to 8 million people (European Commission, 2017). The CCI sector is an important player in the economic and political context, and its strength lays on its ability to encourage both competitiveness and inclusiveness in different fields of business, although its further development still encounters certain obstacles.

27-3

During the last years, the European Union has launched a number of initiatives to support and promote the CCIs. In addition to the €1.5 bn Creative Europe Programme adopted in the framework of the 2015-2018 Work Plan for Culture, in May the European Commission approved a strategy for international cultural relations based on the communication ‘’Culture in the heart of International Relations’’. Moreover, in the input document for the Summit of European leaders in Sweden (November 2017), the Commission invited Member States to include the CCIs in the education system, strengthen their position in the economic market and facilitate access to the available start-ups funds. The European Commission set the following ambitious objectives for 2025: boosting Creative Europe’s Guarantee facility, starting #Digital4Culture strategy and updating European Agenda for Culture.

Despite the many positive initiatives taken by the EU, the CCIs’ capacity is not fully exploited, often undervalued and lacking concrete support from the national and international authorities. At the same time, the cultural sector is undergoing considerable changes due to the digital revolution. Legislation struggles to follow these changing circumstances and CCIs’ potential remains blocked by bureaucratic and administrative requirements, lack of data and weak cross-sector coordination. This leads to challenges in the field of author rights protection, training possibilities and market harmonisation.

This timely symposium, organised in the European year of cultural heritage, will evaluate recent initiatives and policy developments in the field of Cultural and Creative Industries, and present good practice and emerging challenges to the sector. Participants will exchange views and perspectives on the possible strategies to tackle these challenges, and will debate the role of CCIs in national politics, education, international communication and social inclusion.


Delegates will:

  • Find out about the different CCIs funding opportunities and cooperation networks
  • Look into the European initiatives to boost the CCIs sector, such as Creative Europe Programme, Support to literary translation projects, Cultural and Creative Sector Guarantee Facility, European Creative Hubs
  • Address ways to overcome the lack of data and the need to systemise information
  • Enhance the knowledge of the digital capacity-building methods in the field of CCI
  • Identify ways to get access to the European market for companies operating in the cultural and creative sector, according to their size
  • Analyse the methods to eliminate the fragmentation of different educational domains using innovations from the CCIs
  • Learn from success stories on international communication based on cultural exchanges

Early bird rate: a 20% early registration discount off the standard delegate rates (subject to type of organisation and terms and conditions) for bookings received by the 16th February 2018. For further details, please refer to the enclosed event brochure.

Programme and registration: https://www.publicpolicyexchange.co.uk/events/IC27-PPE2

Event’s brochure: http://www.publicpolicyexchange.co.uk/media/events/flyers/IC27-PPE2_flyer.pdf


BlackBox – Arts & Cognition

BlackBox is an interdisciplinary ERC-funded project hosted by Universidad Nova de Lisboa (FCSH-UNL) since September 2014, under the direction of Prof. Carla Fernandes as its Principal Investigator.

With a wide-breadth duration of 5 years, the BlackBox project aims at developing a cutting-edge model for a web-based collaborative platform, dedicated to the documentation of compositional processes by contemporary performing artists with a focus on dance and theatre. The platform will enable a robust representation of the implicit knowledge in performing practices while applying novel visualization technologies to support it.

As an Arts&Cognition project, BlackBox aims at the analysis of the invited artists’ unique conceptual structures, by crossing the empirical insights of contemporary creators with research theories from Multimodal Communication (Human Interaction, Gesture Studies, Cognitive Science) and Computer Vision.

black box

General objectives:

  • Perform theoretical and innovative interdisciplinary research on the intersection of multimodal communication and cognition, performance studies and digital media.
  • Document, transmit and preserve the unexplored knowledge contained in performance composition processes.
  • Assist artists with creative tools to facilitate their choreographic/dramaturgic practices on a collaborative basis.

The ERC Starting-Grant which supports this project has allowed the recruitment of 3 Post-Docs and 2 PhD students in the research domains of Cognitive Linguistics, Digital Media applied to the Performing Arts and Computer Science. The project also hosts several freelance collaborators and external service providers in the areas of graphic and web design, video, photography and software development, as well as international consultants in Digital Archiving and Intangible Cultural Heritage.

It has additional collaborators from: Interactive Multimedia Group at FCT-UNL; the Neuroscience Programme at the Champalimaud Centre for the Unknown; C-DaRE at Coventry University, UK; The Linguistics Centre of Universidade do Porto; and the Motion Bank Institute, Frankfurt.

Black Box website: http://blackbox.fcsh.unl.pt/

Download an article about BlackBox produced by Science Impact, a series of high-quality, open access and free to access science reports designed to enable the dissemination of research impact to key stakeholders. The publication features content from the world’s leading research agencies, policy groups, universities and research projects. To read the full edition of Impact containing this article please visit: https://impact.pub/November2017digitaledition/


SINGULARITY NOW – Tales form the Event Horizon

The 14th Athens Digital Arts Festival exploring the unknown future of the upcoming Technological Apocalypse through art, science and technology, invites you to submit your proposals for its next edition, under the theme “SINGULARITY NOW”. Due to high demand by the artistic community, the applications deadline for ADAF 2018 Open Call is extended until 31 December 2017.
This year for its annual date, ADAF, the most renowned festival for digital arts in Greece, will present its content in May, at a TBA location in the historic center of Athens. The festival attracts and presents works and artists from all over the world.

athens digital artsEvent Horizon is the conceivable surface of a black hole, a place where gravity curves space-time so much, there is no escape, not even by light itself. Its core is the singularity, a zero point, where space and time are discontinued and is unknown what exactly happens further. A wormhole? Termination of everything? A new universe through a white hole?

When something enters the event horizon, all physics laws get distorted, while gravity becomes  such, that inevitably reaches singularity and from there on…..

Singularity is a notion that has different definitions in the various fields of science, nonetheless, it retains a common axis regarding its main idea. In mathematics, it is the point where a function or an equation degenerates or diverges toward infinity, changing its nature and thus becoming impossible to define. In physics, it is the core of a black hole as well as the state right before the Big Bang, the zero point of space and time where everything is compressed in zero dimensions and density is infinite. In mechanics, it is the calibration of a machine in a way that its behavior cannot be predicted and the physical variants involved are either not definite or infinite. In technology, it is a scenario where artificial intelligence surpasses humanity, the technological boom becomes infinite and the outcome is unpredictable.

Is evident that singularity as notion is associated with infinity, with the unknown, and the beginning of universes, both physical and conceptual, but how does it link to art and technology?

Every universe launches its existence in subject to time, while the perception of time is by birth connected to creation and technicality. In the Promethean myth, we can perceive as singularity the gesture of the maker to offer fire to the humankind, passing thusly the torch of knowledge to the creation and enabling it to create on its own. As a consequence comes torment for the creator, but also time emerges in the form of a circular punishment considered one of the first references to the circle of a clock. Additionally, fire, which symbolizes simultaneously both intellect and technicality, was the first medium for the development of all “tekhne”. Its arrival in humanity, signifies the beginning of consciousness, thus consequently the perception of time as well as the development of art (fine tehkne) and technology (the art of making) which resulted in civilization and culture, both means for recording time. In this way, technology and art as derivatives of singularity and instruments of time, have proclaimed the beginning of this world, but maybe also its very end.

Today, the further development of the promethean gift is slowly taking over the reins from its creator and “degenerates” tending to change its nature, holding us witness in an upcoming technological singularity, which can potentially get any form. The constant progress on robotics, AI, biotechnology, biomimicry, nanotechnology, quantum computing and space technology, is courting with the idea of surpassing humankind, constituting obvious for the spectator, that we have entered an “event horizon” where no one can predict what will follow.

Human-machine merger or enslave of humanity? Brain upload and immortality or cease of human species? Progression or Regression? Futurism or Archaism? Physics or Metaphysics? Utopia or Dystopia? Beginning or End?

ADAF_Call-for-entries-Extended

Following to this realization, the International Festival for Digital Arts of Greece, Athens Digital Arts Festival, is calling both the art world and the world of technology, to submit works and achievements which outline, comment, foresee, or determine the upcoming singularity, the human role in this new era, the role and form of art and technology in it, in order to compose the content for the upcoming 14th edition of the festival which will take place in spring 2018.

Will you tell us your tale from this event horizon…?

Detailed information about the submission process and the call for entries is available at the following link: www.adaf.gr


Call for Papers “Studies in Digital Heritage”, Indiana University journal

The aim of this special issue is to solicit interdisciplinary studies about the complex phenomenon that takes place during the aesthetic and synesthetic involvement of a human observer in the perception of Cultural Heritage and art mediated by virtual and augmented reality or other forms of multimedia technologies. The objective is to create a snapshot of the current state of the art about transdisciplinary methodological approaches.

Guest editors: Perla Gianni Falvo, Massimo Bergamasco, Giovanni Valeri Manera

EXTENDED Submission deadline: February 28, 2018

studies in DH

Introduction to the Special Issue on “Perceiving Cultural Heritage Through Digital Technologies”

Contemporary cognitive science supports Cultural Heritage design activities through new and revised scientific perspectives relating to different modes of learning. The concept of affordance, the discovery of mirror neurons, studies on emotional intelligence, multiple intelligences, and psycho-physiological analyses cast light on how we establish relationships, absorb information, and then process and embody it. These disciplines offer important cues for defining design strategies in developing virtual and augmented reality applications within the framework of places or objects of cultural interest, such as museums, historical buildings, monuments or archaeological sites. Rather than being merely desirable, it is now essential to take such cognitive science advances into due consideration in design activities related to the perception of Cultural Heritage.

trident-largeThe response of the human cognitive system to the perception of aesthetic and cultural contents may trigger its understanding at once, through to the so-called “Interior Epiphany”, a concept related to the Renaissance notion of the centrality of the human being. Multimedia technologies may help an observer in activating such an interior epiphany, but the cause-effect relationship is still a matter for further research. It is crucial to verify the effectiveness of the Information and Communications Technology (ICT) strategies and understand how the phenomenon of the contact between visitor and art can be subjected to an impact assessment analysis based on a protocol combining quantitative and qualitative data.

Topics of interests

The topics of interest include, but are not limited to:

  • Design strategies for virtual and augmented applications in museums
  • Neuroscience studies on aesthetic perception through digital technologies
  • Embodiment studies
  • Cognitive studies
  • Psychophysiological studies
  • Narrative approaches for enhancing user involvement in cultural visits
  • E-learning and art history
  • Virtual archeology
  • Analysis of user experience in virtual museums
  • Sustainability of ICT for Art and Cultural Heritage

More info: https://scholarworks.iu.edu/journals/index.php/sdh/announcement/view/110


veraPDF 1.10 released on International Digital Preservation Day

veraPDF-logo-600-300x149The latest version of veraPDF is now available to download. veraPDF is an open source PDF/A validator, and version 1.10 marks the first release of veraPDF as part of the Open Preservation Foundation’s (OPF) reference toolset.

This release has improved optimisation for the creation and cleanup of temporary files and parsing of text-related data in PDF documents. It also fixes some minor issues that users who have upgraded to Java version 9 may have encountered, including a failure to start. There are also a number of other fixes and improvements which are documented in the latest release notes.

Release notes
https://github.com/veraPDF/veraPDF-library/releases/latest.

Download veraPDF
http://downloads.verapdf.org/rel/verapdf-installer.zip.

 

Feedback

Most of the changes in this release are in response to the constructive feedback we’ve received from our users. Testing and user feedback is key to improving the software. Please download and use the latest release. If you experience problems, or wish to suggest improvements, please add them to the project’s GitHub issue tracker: https://github.com/veraPDF/veraPDF-library/issues.

Getting started
User guides and documentation are published at: http://docs.verapdf.org/.

 

Support veraPDF

veraPDF is open source, free to download, to use and to modify. However, it is not free to host, maintain, support or continue to develop the software. Currently only OPF members support veraPDF financially as part of their membership fees.

How can you help?

  • Become an OPF member;
  • Make a donation to OPF and veraPDF development;
  • Volunteer your time and skills by improving the software and documentation.

http://verapdf.org/donate/

 

About veraPDF

veraPDF was originally funded by the PREFORMA project (http://www.preforma-project.eu/). PREFORMA (PREservation FORMAts for culture information/e-archives) was a Pre-Commercial Procurement (PCP) project co-funded by the European Commission under its FP7-ICT Programme. The project’s main aim was to address the challenge of implementing standardised file formats for preserving digital objects in the long term, giving memory institutions full control over the acceptance and management of preservation files into digital repositories. veraPDF is now maintained by the Open Preservation Foundation.

Subscribe to the veraPDF mailing list at: http://lists.verapdf.org/.


CHCD – International Symposium on Cultural Heritage Conservation and Digitization

tsinghua

Organized by THID Tsinghua Heritage Institution for Digitalization in Beijing, this biannual event had attracted so far foreign experts from over 30 countries and regions promoting a tradition of international collaboration and exchange on the theme of cultural heritage, digital technologies, conservation and digitization.

The theme of 2018 conference was “Re-member: heritage driven economy”. The aim was to explore the potential of heritage as a driver for the economic growth (in response to the latest Chinese policy as well as the UNESCO mandate). The conference intended to generate conversation not only among the academia but also connecting the stakeholders of the heritage sites and the heritage-sensitive creative industries, focusing on the field of heritage conservation and cutting-edge research related to the application of digital technology in heritage conservation. International panels based on country, including multidisciplinary experts,  have showcase their research, projects, lessons, and opportunities, promoting their tangible and intangible cultural Heritage.

Register: CHCD Registration System

Official email of the CHCD committee secretariat for any information: chcdcommittee@thid.cn

beijing

Preliminary schedule:
• September 12: Registration
• September 13-14: Conference
• September 15-16: Optional excursion

 


ITS LIQUID international contest 2017: application deadline 1st December 2017

ITS LIQUID Group, a communication platform for contemporary art, architecture and design, is proud to present ITS LIQUID International Contest – 5th Edition 2017. The contest is a great opportunity to present your works in galleries and art spaces in the most influential cities in the world. Taking part in ITS LIQUID Contest, you can win the participation in contemporary art exhibitions in Venice during the 16th Architecture Biennale, in New York, Bogotá, Buenos Aires, interviews and featured articles on specialized magazines and platforms, one year contracts for the representation in influential art galleries.

ITS LIQUID Contest is developed in partnership with a professional jury, composed by renowed curators, gallerists, architects, fashion designershigh level experts and important professionals of the Art, Architecture, Design and Fashion Worlds.

ITS LIQUID Contest is composed by ten categories: painting, photography, video-art, sculpture and installation, performing art, architecture, product design, fashion design, computer graphic, illustration and drawing.

itsliquid_contest_prizes_2017

More info and application form: http://www.itsliquid.com/contest/
The deadline for registration is December 01, 2017. REGISTER NOW!