# Testing Controllers

## Fully Automated Controller Tests

The testing tool supports fully automated testing of Controllers by replacing the default Testing Façade application with your own application implementing the [Question API](../testingfacade/APIs/QuestionAPI.raml) to receive and process the requests from the NMOS Testing Tool. A complete example utilising Flask and Selenium to test the Sony nmos-js Controller can be found in the [sony/nmos-js repository](https://github.com/sony/nmos-js/tree/master/TestingFacade). 

## Building your fully automated test interface

The Testing Façade IP/Hostname and Port on the test suite should be set to the location of your application.
To carry out each test `POST` requests are sent from the Test Suite to the Question API endpoint `http://<ip address or hostname>:<port>/x-nmos/testquestion/<version>`. 

On receiving a `POST` request from the Testing Tool, your application uses the `question_id` to instruct your chosen automated test framework (e.g. Selenium) to carry out the required tasks on your Controller and determine appropriate answers.

Your application `POST`s the `question_id` and answer(s) to the [Answer API](../testingfacade/APIs/AnswerAPI.raml) endpoint given in `answer_uri`.

JSON schema and RAML files for both the Question and Answer API endpoints can be found in the [testingfacade folder](../testingfacade/APIs). 
An example python class for storing and retrieving the Question data can be found at [testingfacade/DataStore.py](../testingfacade/DataStore.py)

Some tests contain multiple questions, all of which must be answered correctly to pass the test. 

There are three types of question that need different kinds of answer:
- `single_choice` - report a single `answer_id` from the `answers` object
- `multi_choice` - report a list of multiple `answer_id`'s from the `answers` object
- `action` - no answers given, simply report `null` when ready to move to the next question

Example JSON files for each type of Question and Answer that could be sent and received can be found in the [examples folder](../testingfacade/examples).

On receiving a `POST` request to the Answer API, the test suite will process the answer and send the next question to the Question API. If the Answer API does not receive a request within the timeout period, the current test will expire and the first question of the next test will be sent.

The test suite begins with a `pre_tests_message` and ends with a `post_tests_message`. These are action questions so the `answer_response` is `null`  to start and end the actual tests.
