This page explains how to follow the Rest.li Test Suite Specification to validate a new Rest.li client binding. This guide is based on experience with Python, and your language implementation may diverge from this guide.
For a broader overview of code structure and test coverage, refer to test suite overview. For Java client binding example, please refer to the provided Java TestNG suite, which follows the Rest.li Test Suite Specification to validate a Java Rest.li client implementation.
To use the Rest.li Test Suite Specification for a new language implementation of Rest.li, you must set up code generation within the project so you can test Rest.li client’s code generation. You must also write methods in the new language so you can use the language-independent data included in the Rest.li Test Suite Specification.
Here is a step-by-step guide on how to set up a test suite for a new language implementation of Rest.li:
build.gradle, apply the plugin for your new implementation of Rest.li to enable code generation.
apply plugin: 'li-python-restli'
Run the Rest.li code generators against all the files in the
schemas/ directory (for data
bindings) and/or the
restspecs/ directory (for rest client bindings).
For Python and Java, this means running a
gradle build. In Java, the code is generated in
src/. In Python,
the code is generated in
The generated code will be tested later in the testsuite.
Select a test framework. You should use the same test framework you usually use for the language your Rest.li client is written in. For the Python test suite, we use pytest.
In your test framework, load the
manifest.json file into a convenient in-memory representation before running the tests.
In Python, a Manifest class is generated from the
.pdsc provided by the test suite.
manifest.json file is loaded into a Manifest representation that is used by the Python test suite.
Based on your test framework, decide how you want to use your Manifest representation to drive your automated tests.
For example, consider json tests in the Python test suite, shown below. We get the list of manifest tests from the Manifest representation in json_test_data(). Then we use the pytest @parametrize decorator to pass the test names and data files into the function test_json(), which contains the testing logic.
def json_test_data(): """Data provider for json_test""" results =  for test in manifest.get_json_test_data(): results.append(test) return results json_data = json_test_data() @pytest.mark.parametrize("json_test", json_test_data(), ids=[test.data_ for test in json_data]) def test_json(json_test): """Tests for correct serialization/deserialization of provided 'data' files""" ...
Set up utility methods in your new language. These methods should provide a way to read in the files referenced by
manifest.json. You also need methods to compare and load flat HTTP requests and responses from the test suite’s
provided test data.
In the Java TestNG suite, these utility methods are part of a test suite base class that is extended by the classes containing actual tests. The Python pytest suite has no subclass/superclass structure because it makes pytest harder to use.
Now your test suite is set up to generate code and read language-independent files from the Rest.li Test Suite Specification.
The next step is to implement tests using the spec’s guidelines and provided test data.
This will be a combination of manual assertions and data driven tests specified by
The tests are driven by
manifest.json and cover three main categories:
Json tests ensure that Rest.li correctly serializes/deserializes all the provided
These files include corner cases to check that they are supported.
One possible approach, which is used by the Java and Python test suites:
def test_json(json_test): """Tests for correct serialization/deserialization of provided 'data' files""" deserialized = load_json_file(os.path.join(*json_test.get_data_().split("/"))) serialized = JSONRestliCodec.encode(deserialized) twice_deserialized = JSONRestliCodec.decode(serialized) for field in deserialized.keys(): assert deserialized[field] == twice_deserialized[field] assert len(deserialized) == len(twice_deserialized)
Schema tests ensure that Rest.li correctly generates language-specific data templates from language-independent data schemas.
For implementations that use data schemas to provide convenience data bindings from data schemas (either dynamically or via code generation):
gradle buildto construct data bindings for all the
For implementations that provide validation:
When testing wire protocol, we want to ensure that the generated request builders can build correct HTTP responses. We also want to ensure that the Rest.li client can correctly create a Rest.li Response representation from an HTTP response.
To test requests and responses, flat files containing HTTP requests and responses are provided in
Rest.li Test Suite Specification’s
responses/ folders, which are for Rest.li protocol 1.0.0.
These flat files have counterparts in
responses-v2/ that use Rest.li protocol version 2.0.0.
but are otherwise identical.
A few possible approaches one might take to use these test files:
The Java and Python test suites use approach 1.
For Rest.li implementations that use restspecs to provide convenience rest client bindings from restspecs (either dynamically or via code generation):
Use the constructed rest client bindings to produce HTTP requests. Do this for all the ‘restspecs.operations.request’
manifest.json. Instead of sending the requests, keep the request in an in-memory representation (a string,
or basic HTTP representation) or write it to a file.
In Python, the logic for this is as follows:
Verify the HTTP request created by the rest client bindings matches the expected HTTP as found in the ‘restspecs.operations.request’ files. This may require writing a routine to compare HTTP requests that ignores header order, and does not fail for optional headers. For instance, Python includes an optional “User-Agent” header, which the flat requests do not include. You may want to ignore such differences in wire protocol.
For implementations that do not provide convenience rest client bindings:
.reqfiles can be produced correctly using these utilities.
Some tests cannot be easily automated. For instance, if you want to check the correct value of a field of an in-memory representation, you do not want to validate it against another in-memory representation. Instead, you can use manual assertions.
Examples of test cases for manual assertions:
data/, so assert that the accessors return the correct test data.