Testing methodology

The objective of this Test Framework is to support conformance and interoperability testing. It describes a test architecture and its software components, how these can be combined to create a test configuration for various types of testing. It also describes the test material to be processed by this architecture, a mark-up language and format for representing test requirements, Test Cases and messages exchanged. The main purpose of the Methodology & Technology Testing Report is to provide a methodology and an IT architecture in order to validate, by means of the piloting activities, the research results coming from the AL A projects.

The Test Framework is composed by the following elements:

1. Test Components

   • Test Driver: The Test Driver is the component that drives the execution of each step of a Test Case. Depending on the test type, the Test Driver may drive the Test Case by interacting with other components. Therefore the primary function of the Test Driver is to parse and interpret the Test Case definitions that are part of a Test Suite, as described in the Test Framework mark-up language.

   • Test Service: it is the component that implements some test operations (actions) triggered by received messages. These actions support and automate the execution of Test Cases.

2. Test Suite Documentation

   • The Test Suite documentation is a collection of several OASIS IIC XML Schemas, with some added attributes required for the ATHENA test platform, containing configuration data, documentation and executable Test Cases description (XML).

   •  Test Suite Metadata provides documentation used by the Test Driver to generate a Test Report for all executed Test Cases.

3. Test Platform

   • From a high level description we can consider the composition of two macro blocks. The first one is the component under test (system, application or service that is being tested). The second block is included in the ATHENA Conformance/Interoperability testing platform that is the component from which the test execution is conducted.

   • It is easy to recognize that the ATHENA Conformance/Interoperability testing platform plays the role of Test Driver previously described while the System under test the one of the Test Service.

With the Test Framework described above it is possible to perform the Conformance and Interoperability Test. The Test Framework is intended for the following modes of operation, whether testing for conformance or for interoperability. In order for a testing process (or validation process) to conform to this specification, the following phases need to be implemented:

• Test Requirements Design: a list of Test Requirements established for the tested specification.

• Test Procedure Design: in the previous methodology the formalised procedures described a sequence of steps that the operator had to perform during the test execution. In this case the sequence of operations to be performed is embedded in an XML document

• Validation Conditions: Validation criteria have to be defined for the profile or level being tested, and expressed as a general condition over the Test Report document.