This second release of D8.1 “Competitive Evaluation Strategy” [Agosti et al., 2014] has a twofold goal:
- In December 2016 there will be the final tender of the PREFORMA project which is aimed at selecting the suppliers which will participate in the “Testing Phase” scheduled from January to June 2017. This deliverable defines the criteria according to which suppliers participating in this tender will be evaluated and compared in order to determine which of them will actually proceed to the “Testing Phase”.
- The “Testing Phase” will evaluate the tool produced by the suppliers on real experimental collections in order to assess their overall quality for conformance checking. This deliverable defines the methodologies and protocols which will be used in this phase to assess the suppliers’ tools.
The document is organized as follows: Section 2 describes the new instantiation of the “PREFORMA Evaluation Matrix” tailored for evaluating the access to the “Testing Phase”; Section 3 introduces the procedures according to which the tender in December 2016 will be managed; Section 4 introduces the framework which will be adopted to evaluate suppliers’ tools during the “Testing Phase”; Section 5 details, for each media type targeted by PREFORMA, the testing classes which will be used; Section 6 describes the practical workflow which will be followed to operate the “Testing Phase”.