This is an automated email from the ASF dual-hosted git repository.
bneradt pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
The following commit(s) were added to refs/heads/master by this push:
new 32d37f29f1 autest.en.rst: Add dev autest docs (#12693)
32d37f29f1 is described below
commit 32d37f29f12f9d9e13ba33b686468d24d0138406
Author: Brian Neradt <[email protected]>
AuthorDate: Fri Nov 21 17:43:16 2025 -0600
autest.en.rst: Add dev autest docs (#12693)
This commit adds a new documentation page (autests.en.rst) that provides
guidance for writing end-to-end tests using the AuTest framework, with a
focus on the new ATSReplayTest approach that uses YAML-based replay
files for traffic specification and verification. This, hopefully living
document, will be helpful for devs and AI tools that could use guidance
on how to write autests.
---
.cursor/rules/writing-autests.mdc | 319 +----------------------
doc/developer-guide/testing/autests.en.rst | 392 +++++++++++++++++++++++++++++
doc/developer-guide/testing/index.en.rst | 1 +
3 files changed, 399 insertions(+), 313 deletions(-)
diff --git a/.cursor/rules/writing-autests.mdc
b/.cursor/rules/writing-autests.mdc
index 2bfa96df24..f93dd6a7ab 100644
--- a/.cursor/rules/writing-autests.mdc
+++ b/.cursor/rules/writing-autests.mdc
@@ -1,318 +1,11 @@
---
-description: When working with or adding autests, the end to end tests for the
project
+description: Consider this when adding tests.
alwaysApply: false
---
-This rule describes writing of end to end autests
+For comprehensive documentation on writing autests for Apache Traffic Server,
see:
-Apache Traffic Server (ATS) has two types of tests: catch tests for unit tests
and the autest framework for end-to-end testing. autests are declarative rather
than imperative, using a specific Python framework syntax. Tests reside in
`tests/gold_tests/` with `.test.py` extensions. For autest framework
documentation, see: https://autestsuite.bitbucket.io/index.html
+**doc/developer-guide/testing/autests.en.rst**
-File Structure and Naming:
-- Place tests in appropriate subdirectories under `tests/gold_tests/` (e.g.,
`cache/`, `pluginTest/<plugin_name>`, `tls/`, etc.)
-- Use descriptive names with `.test.py` extension (e.g., `cache-auth.test.py`,
`stats_over_http.test.py`)
-- Organize related tests in feature-specific subdirectories
-- autest, a general testing framework, is extended to add ATS testing support
via `.test.ext` files in `tests/gold_tests/autest-site`
-
-Running Autests:
-If ATS cmake build is properly configured, tests can be run with:
-```bash
-cmake --build build
-cmake --install build
-cd build/tests
-pipenv install
-./autest.sh --sandbox /tmp/sbcursor --clean=none -f
<test_name_without_test_py_extension>
-```
-
-For example, to run `cache-auth.test.py`:
-```bash
-./autest.sh --sandbox /tmp/sbcursor --clean=none -f cache-auth
-```
-
-Test File Template Structure
-
-Always start a test file with:
-
-```python
-'''
-Brief description of what the test validates
-'''
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-```
-
-Test Configuration:
-
-Set global test properties:
-
-```python
-Test.Summary = 'Brief description of test purpose'
-Test.ContinueOnFail = True # Usually set to True for multi-step tests
-```
-
-
-Class-Based Test Organization
-
-Prefer organizing tests as Python classes and use Python type hints:
-
-```python
-class YourTestClass:
- """
- Docstring explaining the test purpose and any relevant documentation links
- """
-
- def __init__(self):
- self._setupOriginServer() # or self.setupServers()
- self._setupTS() # or self.setupATS()
-
- def _setupOriginServer(self):
- # Create origin server(s)
- self._server = Test.MakeOriginServer("server")
- # OR for replay-based tests:
- # self.server = Test.MakeVerifierServerProcess("server-name",
"replay/file.replay.yaml")
-
- def _setupTS(self):
- # Create ATS process
- self._ts = Test.MakeATSProcess("ts")
-
- # Configure records.config
- self._ts.Disk.records_config.update({
- "proxy.config.diags.debug.enabled": 1,
- "proxy.config.diags.debug.tags": "relevant_debug_tags",
- # Add other configuration as needed
- })
-
- # Configure remap.config if needed
- self._ts.Disk.remap_config.AddLine(
- f"map / http://127.0.0.1:{self.server.Variables.http_port}/";
- )
-
- # Configure plugins if needed
- # self._ts.Disk.plugin_config.AddLine('plugin_name.so plugin_args')
-
- # Set up log validation if needed
- # self._ts.Disk.diags_log.Content += Testers.ContainsExpression(
- # "expected log message", "Description of what this validates"
- # )
-
- def run(self):
- # Execute test scenarios
- self._testScenario1()
- self._testScenario2()
- # etc.
-
- def testScenario1(self):
- tr = Test.AddTestRun('Description of this test scenario')
-
- # Start processes
- tr.Processes.Default.StartBefore(self._server)
- tr.Processes.Default.StartBefore(self._ts)
-
- # Execute test command
- # Option 1: curl command
- tr.MakeCurlCommand(f"-vs --http1.1
http://127.0.0.1:{self._ts.Variables.port}/path";, ts=self._ts)
-
- # Option 2: custom command
- # tr.Processes.Default.Command = "your_command_here"
-
- # Option 3: verifier client for replay tests
- # tr.AddVerifierClientProcess("client-name", "replay/file.replay.yaml",
- # http_ports=[self._ts.Variables.port])
-
- # Set expectations
- tr.Processes.Default.ReturnCode = 0
- tr.Processes.Default.Streams.stdout += Testers.ContainsExpression(
- 'expected_output', 'Description of what this validates'
- )
- # Optionally compare output to a gold file.
- # tr.Processes.Default.Streams.stderr = "gold/expected_stderr.gold"
- tr.Processes.Default.TimeOut = 5
-
- # Keep processes running for subsequent tests
- tr.StillRunningAfter = self._server
- tr.StillRunningAfter = self._ts
-
-# Execute the test
-YourTestClass().run()
-```
-
-Key Framework Objects and Methods
-
-Test Configuration
-- `Test.Summary`: Brief test description
-- `Test.ContinueOnFail`: Whether to continue after failures
-- `Test.SkipUnless()`: Conditional test execution
-- `Test.AddTestRun()`: Create new test run. Each test contains one or more of
these.
-
-Process Creation
-- `tr.MakeATSProcess` or `Test.MakeATSProcess("name")`: Create an ATS instance
for a test run or a global one.
-- `tr.MakeOriginServer("name")` or `Test.MakeOriginServer("name")`: Create an
origin server for a test run or a global one.
-- `tr.AddVerifierServerProcess` or `Test.MakeVerifierServerProcess("name",
"replay_file")`: Create replay-based server
-- Note that every process, except for the Default process for each TestRun,
takes a name and the name has to be unique across all Process names in the
test.py file.
-
-Proxy Verifier Client and Server
-- `tr.MakeoriginServer` is the legacy server type. It just sets up a server.
Typically client HTTP traffic is generated using curl. New tests should use
`AddVerifierServerProcess`
-- `tr.AddVerifierServerProcess` is paired with `tr.AddVerifierClientProcess`
which both use the same external replay config yaml file. These two processes
then exchange traffic to test ATS per the replay yaml file that has
verification rules.
-- For proxy-verifier documentation, see:
https://github.com/yahoo/proxy-verifier
-
-Here's an example for how the server can be configured:
-
-```python
- def _configure_server(self, tr: 'TestRun'):
- """Configure the server.
-
- :param tr: The TestRun object to associate the server process with.
- """
- server =
tr.AddVerifierServerProcess(f"server_{Test_ip_allow.server_counter}",
self.replay_file)
- Test_ip_allow.server_counter += 1
- self._server = server
-```
-
-Here's an example of how the client is configured:
-
-```python
- tr.Processes.Default.StartBefore(self._server)
- tr.Processes.Default.StartBefore(self._ts)
-
- tr.AddVerifierClientProcess(
- f'client-{Test_ip_allow.client_counter}',
- self.replay_file,
- http_ports=[self._ts.Variables.proxy_protocol_port],
- https_ports=[self._ts.Variables.ssl_port],
- http3_ports=[self._ts.Variables.ssl_port],
- keys=self.replay_keys)
-```
-
-ATS Configuration
-- `ts.Disk.records_config.update`: Basic ATS configuration
-- `ts.Disk.remap_config.AddLine()`: Add a remap rule.
-- `ts.Disk.remap_config.AddLines()`: Add multiple remap rules.
-- `ts.Variables.port`, `ts.Variables.ssl_port`: Access ATS HTTP or HTTPS port
-
-ATS HTTPS Configuration
-Many tests require testing HTTPS. This requires some specific configuration:
-- `ts.addDefaultSSLFiles` Copy the default SSL certs for ATS to use.
-- `ts.Disk.ssl_multicert_config.AddLine("dest_ip=* ssl_cert_name=server.pem
ssl_key_name=server.key")` configure ATS to use the certs
-
-Also for HTTP, add these configurations to the records_config.update:
-
-```python
-"proxy.config.ssl.server.cert.path": f"{self.ts.Variables.SSLDir}",
-"proxy.config.ssl.server.private_key.path": f"{self.ts.Variables.SSLDir}",
-```
-
-Plugin testing considerations:
-Some tests verify plugin behavior. Follow these guidelines when testing a
plugin:
-- `ts.Disk.plugin_config.AddLine()`: Configure plugins. This must be done for
any plugin under test.
-- Add `Test.SkipUnless(Condition.PluginExists('plugin_name.so'))` at the top
of the test to only run if the plugin is installed.
-
-Traffic Generation
-- `tr.Processes.Default.Command`: Set custom command. Sometimes a test has an
adhoc Python script to generate traffic, for example.
-- `tr.MakeCurlCommand()`: Execute curl command. This sets
tr.Processes.Default.Command to run a curl command.
-- `tr.AddVerifierClientProcess()`: This also sets tr.Processes.Default.Command
to use the Proxy Verifier replay client to generate traffic. This takes, among
other parameters, the name of a replay file.
-- `tr.Processes.Default.StartBefore()`: Start process before the client
starts. This must be done for ats and all servers before the client starts.
Although, if there are global servers, they can only be started once by the
first Default process of the first TestRun.
-
-Validation
-- `Testers.ContainsExpression(pattern, description)`: Validate output contains
pattern. Generally += these.
-- `tr.Processes.Default.ReturnCode`: Expected exit code. By default it is 0,
but certain negative tests should expect 1.
-- `tr.Processes.Default.Streams.all`: Stream validation
-- `tr.Processes.Default.TimeOut`: Test timeout
-- `tr.StillRunningAfter`: Ensure the server is still running after the client
finishes.
-- For VerifierClient and VerifierServer, much validation is done in the replay
files themselves. The test.py should add output verification that each
transaction happened as expected.
-
-Here's an example Verifier replay file:
-
-```yaml
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-sessions:
-
-- transactions:
-
- - client-request:
- method: GET
- url: /
- version: '1.1'
- headers:
- fields:
- - [ Host, fqdn1 ]
- - [ uuid, 1 ]
-
- proxy-response:
- status: 200
- headers:
- fields:
- - [ Content-Length, { value: 19, as: equal } ]
- - [ Cache-Control, { value: no-cache, as: equal } ]
- - [ Content-Type, { value: text/plain, as: equal } ]
- - [ Server, { value: ATS, as: contains } ]
- content:
- encoding: plain
- data: |
- small body content
- verify: { as: equal }
-
- - client-request:
- method: GET
- url: /path
- version: '1.1'
- headers:
- fields:
- - [ Host, fqdn1 ]
- - [ uuid, 2 ]
-
- proxy-response:
- status: 404
- headers:
- fields:
- - [ Cache-Control, { value: no-store, as: equal } ]
- - [ Content-Type, { value: text/html, as: equal } ]
- - [ Server, { value: ATS, as: contains } ]
- content:
- encoding: plain
- data: Error
- verify: { as: contains }
-```
-
-Replay Yaml Notes:
-- Each transaction will have a client-request and a server-response which each
generate the HTTP request and response, respectively.
-- Each transaction's client-request has a uuid header whose value uniquely
identifies the transaction.
-- proxy-request and proxy-response nodes do not generate traffic but rather
verify content. See their value: and as: content above.
-- The proxy-response status: node verifies the status. This is generally
important for verifying a 200 or some other HTTP response from ATS.
-
-Best Practices:
-- **Class Organization**: Use classes to group related test scenarios
-- **Separation of Concerns**: Separate server setup, ATS setup, and test
execution as separate functions in the class.
-- **Descriptive Names**: Use clear, descriptive names for test runs and
processes. Pass a descriptive name to `tr.AddTestRun('<concise test run
description>')`
-- **Documentation**: Include docstrings explaining test purpose and setup as
well as type hints.
-- **Gold Files**: Use gold files for complex expected outputs in
subdirectories. These can be slow, so prefer
ContainsExpression/ExcludesExpressions.
-- **Debug Configuration**: Enable relevant debug tags for troubleshooting.
-- **Cache IO**: if testing the ATS cache, one transaction has to populate the
cache. If using VerifierClient, put a `delay: 100ms` in the client-request for
the following transaction to make a 100ms delay for the cache to finish IO.
-
-
-When generating autests, follow these patterns and adapt the template to the
specific testing scenario while maintaining the declarative,
configuration-driven approach that characterizes the autest framework.
+Most test coverage comes from end to end autests in tests/gold_tests. Of these,
+most should be writting using ATSReplayTest with the configuration and traffic
+specification in the associated replay.yaml file.
diff --git a/doc/developer-guide/testing/autests.en.rst
b/doc/developer-guide/testing/autests.en.rst
new file mode 100644
index 0000000000..8c83cff326
--- /dev/null
+++ b/doc/developer-guide/testing/autests.en.rst
@@ -0,0 +1,392 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. include:: ../../common.defs
+
+.. _autest-writing:
+
+Writing Autests
+***************
+
+|TS| has two types of developer tests: (1) Catch2 tests for unit tests and (2)
+AuTest framework tests for end-to-end testing. The Catch tests reside next the
+the associated source code being tested while AuTests are located in
+``tests/gold_tests/`` with ``.test.py`` extensions.
+
+* For catch test framework documentation, see:
https://github.com/catchorg/Catch2
+* For AuTest framework documentation, see:
https://autestsuite.bitbucket.io/index.html
+
+This document focuses on AuTest framework tests because they are rather unique
+to the |TS| project while Catch2 documentation can be found elsewhere.
+
+File Structure and Naming
+==========================
+
+Here is a summary of the file structure and naming conventions for AuTest
tests:
+
+- AuTest tests are placed in appropriate subdirectories under
+ ``tests/gold_tests/`` (e.g., ``cache/``, ``pluginTest/<plugin_name>``,
``tls/``,
+ etc.)
+- AuTest test files have a descriptive name with a ``.test.py`` extension
(e.g.,
+ ``cache-auth.test.py``, ``stats_over_http.test.py``). When the tests are run,
+ be aware that their names, sans the ``.test.py`` extension, are used to
+ identify the test.
+- The ``.test.py`` typically are thin and reference the associated
+ ``replay.yaml`` file that describes the test via the ``Test.ATSReplayTest()``
+ method.
+- ``tests/gold_tests/autest-site`` is a special directory. AuTest, a general
+ testing framework, is extended to add domain specific support, |TS| in this
+ case, via ``.test.ext`` extension files. The files in here customize the
+ command line arguments recognized by the ``autest`` command, the functions
+ availabe to the ``Test`` and ``TestRun`` AuTest objects, specific ``Process``
+ objects available to test, ``Skip`` conditions for individual tests, etc.
+
+Running Autests
+===============
+
+If |TS| cmake build is configured via ``-DENABLE_AUTEST=ON``, tests can be run
with:
+
+.. code-block:: bash
+
+ cmake --build build
+ cmake --install build
+ cd build/tests
+ pipenv install
+ ./autest.sh --sandbox /tmp/sbcursor --clean=none -f
<test_name_without_test_py_extension>
+
+For example, to run ``cache-auth.test.py``:
+
+.. code-block:: bash
+
+ ./autest.sh --sandbox /tmp/sbcursor --clean=none -f cache-auth
+
+Recommended Approach: ATSReplayTest
+====================================
+
+Currently, many tests are specified largely entirely using the generic AuTest
+framework specific syntax via ``.test.py`` files. These use the generic AuTest
+framework syntax, which is generically very capable, but not tuned to the
+specific |TS| environment and not generally parseable by code editors and AI
+tools.
+
+AuTest itself has a solution for this via its extensibility mechanism. A
+concerted effort is underway to make more full use of the AuTest extension
+mechanism to simplify the test writing process specifically for |TS|. At a
high
+level, the extension is called ``ATSReplayTest`` and it is used in a
``test.py``
+file to reference an associated ``replay.yaml`` file which fully describes the
+test. The goal is that a large percentage, maybe 90%, of the tests can be
+written using this approach while certain requests, perhaps requiring ad-hoc
+clients and servers, will be written using the generic AuTest framework syntax.
+
+The traffic portion of the ``replay.yaml`` files specify Proxy Verifier HTTP
+traffic behavior and follow the replay and verification syntax described
+extensively in its project's README.md file here:
+https://github.com/yahoo/proxy-verifier
+
+Simple Test File Structure
+---------------------------
+
+Here is an example of a test file using ``ATSReplayTest``:
+
+.. code-block:: python
+
+ '''
+ Brief description of what the test validates
+ '''
+ # Licensed to the Apache Software Foundation (ASF) under one
+ # or more contributor license agreements. See the NOTICE file
+ # distributed with this work for additional information
+ # regarding copyright ownership. The ASF licenses this file
+ # to you under the Apache License, Version 2.0 (the
+ # "License"); you may not use this file except in compliance
+ # with the License. You may obtain a copy of the License at
+ #
+ # http://www.apache.org/licenses/LICENSE-2.0
+ #
+ # Unless required by applicable law or agreed to in writing, software
+ # distributed under the License is distributed on an "AS IS" BASIS,
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ # See the License for the specific language governing permissions and
+ # limitations under the License.
+
+ Test.Summary = '''
+ Brief description of test purpose
+ '''
+
+ Test.ATSReplayTest(replay_file="replay/my-test.replay.yaml")
+
+For tests with multiple |TS| configuration scenarios, you can call
+``Test.ATSReplayTest()`` multiple times, each with a different replay file
+specifying different configurations of |TS|, dns, servers, clients, etc.
+
+.. code-block:: python
+
+ Test.ATSReplayTest(replay_file="replay/scenario1.replay.yaml")
+ Test.ATSReplayTest(replay_file="replay/scenario2.replay.yaml")
+ Test.ATSReplayTest(replay_file="replay/scenario3.replay.yaml")
+
+Replay File Structure
+----------------------
+
+The replay file contains both the test configuration (in the ``autest`` YAML
+node) and the traffic replay and verification specification (in the
``sessions``
+YAML node). Here is an example:
+
+.. code-block:: yaml
+
+ # Licensed to the Apache Software Foundation (ASF) under one
+ # or more contributor license agreements. See the NOTICE file
+ # distributed with this work for additional information
+ # regarding copyright ownership. The ASF licenses this file
+ # to you under the Apache License, Version 2.0 (the
+ # "License"); you may not use this file except in compliance
+ # with the License. You may obtain a copy of the License at
+ #
+ # http://www.apache.org/licenses/LICENSE-2.0
+ #
+ # Unless required by applicable law or agreed to in writing, software
+ # distributed under the License is distributed on an "AS IS" BASIS,
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ # See the License for the specific language governing permissions and
+ # limitations under the License.
+
+ meta:
+ version: "1.0"
+
+ # Configuration section for autest integration
+ autest:
+ description: 'Test description for this scenario'
+
+ # Optional (but typical) DNS configuration.
+ dns:
+ name: 'dns'
+ # Other MakeDNServer parameters can be set, see microDNS.test.ext
+
+ # Required: Server configuration.
+ server:
+ name: 'server'
+ # Other AddVerifierServerProcess parameters can be set, see
verifier_server.test.ext
+
+ # Required: Client configuration.
+ client:
+ name: 'client'
+ # Other AddVerifierClientProcess parameters can be set, see
verifier_client.test.ext
+
+ # Required: ATS configuration.
+ ats:
+ name: 'ts'
+
+ # Optional: Enable cache (default is determined by process_config)
+ # enable_cache: true
+
+ # Other parameters can be set, see trafficserver.test.ext
+
+ # Optional: ATS process configuration
+ process_config:
+ enable_cache: true
+ # Other MakeATSProcess parameters can be set, see
trafficserver.test.ext
+
+ # ATS records.config settings
+ records_config:
+ proxy.config.diags.debug.enabled: 1
+ proxy.config.diags.debug.tags: 'http|cache'
+ proxy.config.http.cache.http: 1
+
+ # Remap configuration (list format)
+ remap_config:
+ # Option 1: String format.
+ - "map http://test.com/ http://backend.test.com:8080/";
+
+ # Option 2: Dict format with automatic port substitution
+ # Note: Using hostnames like backend.example.com requires DNS
configuration
+ - from: "http://example.com/";
+ to: "http://backend.example.com:{SERVER_HTTP_PORT}/";
+ # Optional plugins
+ plugins:
+ - name: "conf_remap.so"
+ args:
+ - "proxy.config.http.cache.required_headers=0"
+
+ # Optional: Copy test-specific files/directories to ATS config directory
+ copy_to_config_dir:
+ - "my-plugin-config.txt"
+ - "cert-directory/"
+
+ # Optional: Log (traffic.out or diags.log) validation
+ log_validation:
+ traffic_out:
+ contains:
+ - expression: "Expected log message in traffic.out"
+ description: "Verify this appears in traffic.out"
+ excludes:
+ - expression: "Unwanted log message"
+ description: "Verify this does NOT appear in traffic.out"
+ diags_log:
+ contains:
+ - expression: "Expected log message in diags.log"
+ description: "Verify this appears in diags.log"
+ excludes:
+ - expression: "Unwanted message in diags.log"
+ description: "Verify this does NOT appear in diags.log"
+
+ # Traffic specification using Proxy Verifier format
+ # client-request and server-response generate request and response traffic
+ # toward the ATS proxy.
+ # proxy-request and proxy-response verify the content of the request and
response
+ # after proxying through ATS.
+ sessions:
+ - transactions:
+
+ # First transaction: populate cache
+
+ # Send a request to ATS.
+ - client-request:
+ method: GET
+ url: /path
+ version: '1.1'
+ headers:
+ fields:
+ - [Host, example.com]
+ - [uuid, transaction-1]
+
+ # Verify request headers from ATS.
+ proxy-request:
+ headers:
+ fields:
+ - [X-Added-Header, { value: some_field_value, as: equal }]
+
+ # Send a response to ATS.
+ server-response:
+ status: 200
+ reason: OK
+ headers:
+ fields:
+ - [Content-Type, text/plain]
+ - [Content-Length, "4"]
+ - [Cache-Control, "max-age=300"]
+
+ # Verify respone headers from ATS.
+ proxy-response:
+ status: 200
+ headers:
+ fields:
+ - [Content-Length, { value: 4, as: equal }]
+
+ # Second transaction: verify cache hit with delay
+ - client-request:
+ # Add delay for cache IO to complete
+ delay: 100ms
+
+ method: GET
+ url: /path
+ version: '1.1'
+ headers:
+ fields:
+ - [Host, example.com]
+ - [uuid, transaction-2]
+
+ # Server should not receive this request (cache hit)
+ server-response:
+ status: 404
+ reason: Not Found
+
+ # Expect cached 200 response
+ proxy-response:
+ status: 200
+ headers:
+ fields:
+ - [Content-Length, { value: 4, as: equal }]
+
+Replay File Components
+-----------------------
+
+autest Configuration Section
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``autest`` section configures the test environment:
+
+- **description** (required): A brief description of what this test scenario
+ validates. This is helpful to document the intetion of the test for the human
+ reading the file and is also helpful when tests fail as this description is
included
+ in failure output.
+- **dns** (optional): DNS server configuration with ``name`` and optional
+ ``process_config``. See the ``microDNS.test.ext`` file for more details.
Including
+ the DNS allows remap entries to contain hostnames rather than localhost IP
+ addresses.
+- **server** (required): Proxy Verifier Server configuration with ``name`` and
+ optional ``process_config``. See the ``verifier_server.test.ext`` file for
+ more details. This acts as the HTTP origin server that receives requests from
+ |TS| as they are proxied from the client to the server. It also provides any
+ request verification and generates the configured HTTP response.
+- **client** (required): Proxy Verifier Client configuration with ``name`` and
+ optional ``process_config``. See the ``verifier_client.test.ext`` file for
+ more details. This acts as the HTTP client that requests content from the
+ server via the |TS| proxy and validates the response.
+- **ats** (required): |TS| configuration including:
+
+ - **name**: ATS process name
+ - **enable_tls**: Set to ``true`` for HTTPS testing
+ - **process_config**: Parameters passed to ``MakeATSProcess`` (e.g.,
``enable_cache``)
+ - **records_config**: Dictionary of records.config settings
+ - **remap_config**: List of remap rules (string or dict format)
+ - **copy_to_config_dir**: List of files/directories to copy to ATS config
directory
+ - **log_validation**: Log validation rules for ``traffic_out`` and
``diags_log``
+
+Log Validation
+~~~~~~~~~~~~~~
+
+The ``log_validation`` section allows you to verify the contents of
+``traffic.out`` and ``diags.log`` after the test completes.
+
+.. code-block:: yaml
+
+ log_validation:
+ traffic_out:
+ contains:
+ - expression: "cache hit"
+ description: "Verify cache hit occurred"
+ excludes:
+ - expression: "cache miss"
+ description: "Should not be a cache miss"
+ diags_log:
+ contains:
+ - expression: "Plugin initialized"
+ description: "Verify plugin loaded"
+
+Sessions and Transactions
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``sessions`` section uses the Proxy Verifier format to specify HTTP traffic
+and validation. The below provides a brief overview. For details, see the Proxy
+Verifier documentation: https://github.com/yahoo/proxy-verifier
+
+Key points:
+
+- The client uses ``client-request`` to generate an HTTP requests to the |TS|
+ proxy. This must contain a ``uuid`` header value to uniquely identify the
+ transaction which is later used by the server.
+- The server uses ``proxy-request`` to verify the contents of the proxied
+ request |TS| sent to it.
+- The server uses ``server-response`` to specify the HTTP response to send to
+ the client. The server uses the ``uuid`` header value as the key to look up
+ which transaction applies to the received request.
+- The client uses the ``proxy-response`` to verify the contents of the response
+ from the |TS| proxy.
+- Use ``delay`` in ``client-request`` to wait between requests (e.g., for
cache IO).
+- Verficiation in ``proxy-*`` nodes uses ``{ value: X, as: <directive> }``
+ syntax to check header values.
+- Status codes can be verified with the ``status`` field in ``proxy-response``.
diff --git a/doc/developer-guide/testing/index.en.rst
b/doc/developer-guide/testing/index.en.rst
index 2cac843da2..3810a5f00b 100644
--- a/doc/developer-guide/testing/index.en.rst
+++ b/doc/developer-guide/testing/index.en.rst
@@ -25,4 +25,5 @@ Testing Traffic Server
.. toctree::
:maxdepth: 2
+ autests.en
blackbox-testing.en
