Deliverable D8.8 reports on monitoring of the Open Source Project implementations. Based on development efforts for each supplier, this updated deliverable (version 2.1) provides feedback on their use of: an open work practice for development; frequent open releases; and promotion activities aiming towards a sustainable community. In particular, it focuses on establishing sustainable communities, together with an assessment of how this is succeeded. The deliverable presents an evaluation of how each open source project implementation adheres to requirements expressed in deliverable D4.3 and how projects and suppliers have acted upon feedback and recommendations from PREFORMA. In so doing, the deliverable provides an evaluation of the extent to which best practices from community driven open source projects have been adopted with adherence to full transparency for all digital assets. Specifically, the evaluation considers software and associated digital assets provided via links to developed and provided resources (including source code, executables, and test files) and tools (including software configuration management system, mailing lists, and build environment) used in each open source project. An important outcome from this evaluation is a report on adherence to requirements (as specified in D4.3 and clarified in feedback from PREFORMA) and an assessment of how contracted organisations have managed to provide open source software and establish thriving and long-term sustainable open source communities of relevance for memory institutions and other stakeholder groups. Based on these outcomes, recommendations are given for further actions by the suppliers, the PREFORMA Consortium, and any potential adopter of software from the Open Source Portal provided by the PREFORMA Consortium.

This deliverable presents the results of the second part of the Prototyping phase in the PREFORMA project, thereby to some extent also summarising the outcomes of this phase in its entity. The document consists of two parts:

The first part (chapters 1 to 3) gives the overall context, including the aims and objectives of the second part of the Prototyping phase. It also reports on formal procedures, like the discussions with the suppliers and internally in the PREFORMA Consortium, and presents specific issues of general nature in more depth.

The second part (chapters 4 to 5) focuses on the results achieved during the Prototyping phase, particularly its second part, but also on progress in the development of the PREFORMA prototypes, which will be further advanced in the Testing and validation phase.

This deliverable is considered to be the final report on the activities related to the preparation and procedure of the design phase #2, internally also called the re-design phase. It is important to notice here already in the beginning of this report that the PREFORMA project, in its Description of Work (DoW) document, had not planned to go for a formal evaluation at the end of the design phase #2, like the one performed at the end of the design phase #1, but for an informal review allowing the project members as well as the suppliers to evaluate and rate the results of their work, their working procedure, the error finding mechanisms, the connection between the phases of design and prototyping, and the progress of the suppliers compared to their own Description of Work (DoW) documents. The final and concluding official and formal review will be performed toward the end of the prototyping phase #2, which eventually means in M36.

Reflected in this document, the second and final design phase of the suppliers’ work to create and update functional and technical specification as well as interoperability documents and the related software modules started with the review of the design, the definition, and the specification of the functional and the technical part of their preparatory work according to the call for tender, the submissions (description of work) of the three winning supplier teams and consortia, the review results of the design phase #1, and the work results of the prototyping phase #1 that has followed the design phase #1 completion.

The document will thus include a short summary of the prototyping phase #1 description (originally compiled in D8.3) and the basic statements related to the two phases of WP5 including citations, references and methodologies for:

• PREFORMA evaluation strategy (from D8.1)
• PREFORMA lessons learned from the design phase #1 (from D8.2)
• Short summary of the findings of the prototyping phase #1 (from D8.3)
• Short summary of the informal review procedure
• Procedure of the evaluation of the suppliers’ documentation
• Results of the PREFORMA consortium visits to the suppliers
• Individual meetings with the suppliers on results of evaluation
• Informal decision making process
• Statements of the end of design phase #2 report
• Open source workshop and the suppliers’ performance
• Final decisions made by PREFORMA consortium

The previously completed tasks in WP8 laid the foundation for informal evaluation strategy for comparing the results of the suppliers, ranging from the end of the design phase #1 to the prototyping phase #1 to the beginning of the design phase #2. The formal evaluation framework, also applied to this informal evaluation for the design phase #2, has been defined in D8.1 and has been successfully applied in D8.2, based on contributions of the technical partners as well as of the memory institutions, either being partners in PREFORMA, or being invited as external experts. The strategy negotiated and established in T8.2, and consequently described in D8.2, too, was used as an input for evaluating the suppliers’ results informally at the beginning of the design phase #2 to value whether or not the suppliers have fulfilled the tasks they were expected to do.

The document D8.4 is thus intended to include all useful information for the internal and external work process for the design phase #2 as well as to give an idea on how a PCP project does follow and rate the progress of suppliers during the development process, and finally how the informal evaluation in PREFORMA has been performed.

This second release of D8.1 “Competitive Evaluation Strategy” [Agosti et al., 2014] has a twofold goal:

• In December 2016 there will be the final tender of the PREFORMA project which is aimed at selecting the suppliers which will participate in the “Testing Phase” scheduled from January to June 2017. This deliverable defines the criteria according to which suppliers participating in this tender will be evaluated and compared in order to determine which of them will actually proceed to the “Testing Phase”.
• The “Testing Phase” will evaluate the tool produced by the suppliers on real experimental collections in order to assess their overall quality for conformance checking. This deliverable defines the methodologies and protocols which will be used in this phase to assess the suppliers’ tools.

The document is organized as follows: Section 2 describes the new instantiation of the “PREFORMA Evaluation Matrix” tailored for evaluating the access to the “Testing Phase”; Section 3 introduces the procedures according to which the tender in December 2016 will be managed; Section 4 introduces the framework which will be adopted to evaluate suppliers’ tools during the “Testing Phase”; Section 5 details, for each media type targeted by PREFORMA, the testing classes which will be used; Section 6 describes the practical workflow which will be followed to operate the “Testing Phase”.

This deliverable describes the initial version of the sustainability and exploitation plan of PREFORMA, including a preliminary impact assessment conducted through an analysis of the level of activity generated in the community, of the test cases carried out so far and of the results of a survey circulated to the memory institutions participating to the project, both as partners and as associate partners within the network of common interest.

The following communities have been considered in relation to their expected engagement in exploiting the PREFORMA applications:

• the open-source community of researchers and developers interested in contributing to the code;
• memory institutions willing to integrate the PREFORMA software in their infrastructure;
• the community of enterprises interested in PREFORMA tools and in developing services around them (first of all the suppliers working in the project);
• the standardization bodies looking for feedback on how to improve and advance the specifications of the standard file formats.

The Experience Workshop took place Berlin on 23 November 2016, in connection with the second Prototype Demonstration. The PREFORMA partners shared their experiences of working with suppliers under R&D service agreements with other memory institutions.

The morning session focused on use cases for conformance checkers from memory institutions, while the afternoon explored the PREFORMA challenge with an overview of the upcoming testing phase. This was followed by presentations from the three suppliers who develop the conformance checkers.

All presentation slides and more photos from the workshop are available at: http://experienceworkshop.PREFORMA-project.eu/programme/.

Deliverable D3.4 reports on the Open Source Workshop that was based on the availability of the first prototype on the Open Source Portal and the results of the first demonstration organised by the suppliers.

The workshop was organised by the PREFORMA project on 7 April 2016 in Stockholm and was hosted by Kungliga Biblioteket. The overall structure for the full day workshop was to devote the morning session to presentations and the afternoon session to interaction and discussion amongst workshop participants. To address the fundamental community aspects of open source in the archival domain and in memory institutions involved in (or planning) digital preservation initiatives, the PREFORMA team invited Peter Bubestinger as a keynote speaker. To address the licensing aspects of open source, the PREFORMA team invited Dr. Till Jaeger as a keynote speaker.

The aim for the morning session was to convey an overview of the PREFORMA project and insights concerning key challenges for successful open source development as perceived by the two keynote speakers. The aim for the afternoon session was to report on development efforts undertaken by PREFORMA suppliers, with highlights on open source tools being developed, and to offer suppliers an opportunity for exhibiting their tools to workshop participants. A further goal during the afternoon was to further stimulate interaction and dialogue between suppliers’ representatives and other workshop participants.

The workshop facilitated a unique opportunity for attendees to raise different issues and challenges of specific interest with suppliers’ representatives, PREFORMA partners’ representatives, the invited speakers, and other workshop participants. During discussions, several attendees established valuable contacts and networking amongst participants was highly appreciated. For the suppliers, the interactive networking session gave many opportunities for disseminating and communicating their efforts to the broader communities, including potential adopters of open source software developed in PREFORMA.