annotation-protocol/README.md - external/github.com/web-platform-tests/wpt - Git at Google

 Annotation-Protocol: Tests for the Web Annotation Protocol
 ==========================================================

 The [Web Annotation Protocol](https://www.w3.org/TR/annotation-protocol)
 specification presents set of messages to allow Annotation clients and servers
 to interact seamlessly.

 The purpose of these tests is to help validate that clients send and are
 capable of receiving correctly formatted messages, and that servers are
 able to receive and respond to correctly structured requests.

 The general approach for this testing is to enable both manual and
 automated testing. However, since the specification has no actual user
 interface requirements, there is no general automation mechanism that
 can be presented for testing clients.  Also the server tests need to be
 pointed at a server implementation to exercise.  However, once provided
 the basic information, testing is automated.

 Implementors could take advantage of the plumbing we provide here to
 help their implementations talk to the endpoint we provide or exercise
 their endpoint with the provided server tests.  This assumes knowledge
 of the requirements of each test / collection of tests so that the input
 data is relevant.  Each test or test collection contains information
 sufficient for the task.

 With regard to server tests, the browser tests we provide can be
 pointed at an endpoint and will exercise that endpoint using well
 defined messages.  This is done semi-automatically, although some set-up
 is required.

 Running Tests
 -------------

 In the case of this test collection, we will be initially creating manual
 tests.  These will automatically determine pass or fail and generate output for
 the main WPT window.  The plan is to minimize the number of such tests to
 ease the burden on the testers while still exercising all the features.

 The workflow for running these tests is something like:

 1. Start up the test driver window and select the annotation-protocol tests -
    either client or server - then click "Start".
 2. A window pops up that shows a test - the description of which tells the
    tester what is required.  The window will contain fields into which some
    information is provided.
 3. In the case of client testing the tester (presumably in another window) brings up their
    annotation client and points it at the supplied endpoint.  They they perform the
    action specified (annotating content in the test window, requesting an annotation from the server, etc.).
 4. The server receives the information from the client, evaluates it, and reports the result of testing.
    In the event of multi-step messages, the cycle repeats until complete.
 5. Repeat steps 2-4 until done.
 6. Download the JSON format report of test results, which can then be visually
    inspected, reported on using various tools, or passed on to W3C for
    evaluation and collection in the Implementation Report via github.

 **Remember that while these tests are written to help exercise implementations,
 their other (important) purpose is to increase confidence that there are
 interoperable implementations.** So, implementers are our audience, but these
 tests are not meant to be a comprehensive collection of tests for an implementor.
 The bulk of the tests are manual because there are no UI requirements in the
 Recommendation that would make it possible to effectively stimulate every client portably.

 Having said that, because the structure of these "manual" tests is very rigid,
 it is possible for an implementer who understands test automation to use an
 open source tool such as [Selenium](http://www.seleniumhq.org/) to run these
 "manual" tests against their implementation - exercising their implementation
 against content they provide to create annotations and feed the data into our
 test input field and run the test.

 Capturing and Reporting Results
 -------------------------------

 As tests are run against implementations, if the results of testing are
 submitted to [test-results](https://github.com/w3c/test-results/) then they will
 be automatically included in documents generated by
 [wptreport](https://www.github.com/w3c/wptreport). The same tool can be used
 locally to view reports about recorded results.

 Automating Test Execution
 -------------------------

 Writing Tests
 -------------

 If you are interested in writing tests for this environment, see the
 associated [CONTRIBUTING](CONTRIBUTING.md) document.
	Annotation-Protocol: Tests for the Web Annotation Protocol
	==========================================================

	The [Web Annotation Protocol](https://www.w3.org/TR/annotation-protocol)
	specification presents set of messages to allow Annotation clients and servers
	to interact seamlessly.

	The purpose of these tests is to help validate that clients send and are
	capable of receiving correctly formatted messages, and that servers are
	able to receive and respond to correctly structured requests.

	The general approach for this testing is to enable both manual and
	automated testing. However, since the specification has no actual user
	interface requirements, there is no general automation mechanism that
	can be presented for testing clients. Also the server tests need to be
	pointed at a server implementation to exercise. However, once provided
	the basic information, testing is automated.

	Implementors could take advantage of the plumbing we provide here to
	help their implementations talk to the endpoint we provide or exercise
	their endpoint with the provided server tests. This assumes knowledge
	of the requirements of each test / collection of tests so that the input
	data is relevant. Each test or test collection contains information
	sufficient for the task.

	With regard to server tests, the browser tests we provide can be
	pointed at an endpoint and will exercise that endpoint using well
	defined messages. This is done semi-automatically, although some set-up
	is required.

	Running Tests
	-------------

	In the case of this test collection, we will be initially creating manual
	tests. These will automatically determine pass or fail and generate output for
	the main WPT window. The plan is to minimize the number of such tests to
	ease the burden on the testers while still exercising all the features.

	The workflow for running these tests is something like:

	1. Start up the test driver window and select the annotation-protocol tests -
	either client or server - then click "Start".
	2. A window pops up that shows a test - the description of which tells the
	tester what is required. The window will contain fields into which some
	information is provided.
	3. In the case of client testing the tester (presumably in another window) brings up their
	annotation client and points it at the supplied endpoint. They they perform the
	action specified (annotating content in the test window, requesting an annotation from the server, etc.).
	4. The server receives the information from the client, evaluates it, and reports the result of testing.
	In the event of multi-step messages, the cycle repeats until complete.
	5. Repeat steps 2-4 until done.
	6. Download the JSON format report of test results, which can then be visually
	inspected, reported on using various tools, or passed on to W3C for
	evaluation and collection in the Implementation Report via github.

	**Remember that while these tests are written to help exercise implementations,
	their other (important) purpose is to increase confidence that there are
	interoperable implementations.** So, implementers are our audience, but these
	tests are not meant to be a comprehensive collection of tests for an implementor.
	The bulk of the tests are manual because there are no UI requirements in the
	Recommendation that would make it possible to effectively stimulate every client portably.

	Having said that, because the structure of these "manual" tests is very rigid,
	it is possible for an implementer who understands test automation to use an
	open source tool such as [Selenium](http://www.seleniumhq.org/) to run these
	"manual" tests against their implementation - exercising their implementation
	against content they provide to create annotations and feed the data into our
	test input field and run the test.

	Capturing and Reporting Results
	-------------------------------

	As tests are run against implementations, if the results of testing are
	submitted to [test-results](https://github.com/w3c/test-results/) then they will
	be automatically included in documents generated by
	[wptreport](https://www.github.com/w3c/wptreport). The same tool can be used
	locally to view reports about recorded results.

	Automating Test Execution
	-------------------------

	Writing Tests
	-------------

	If you are interested in writing tests for this environment, see the
	associated [CONTRIBUTING](CONTRIBUTING.md) document.