Update runtime test README.

PiperOrigin-RevId: 631860904
This commit is contained in:
Ayush Ranjan
2024-05-08 11:10:52 -07:00
committed by gVisor bot
parent b9b626d061
commit 00ac57acdd
+13 -13
View File
@@ -80,20 +80,22 @@ docker system prune # Remove unused data.
To bump the version of an existing runtime test:
1. Create a new [Docker image](runtime-images) for the new runtime version.
This will likely look similar to the older version, so start by copying the
older one. Update any packages or downloaded urls to point to the new
version. Test building the image with `docker build
images/runtime/<new_runtime>`
1. Update the [Docker image](../../images/runtimes) for with the new runtime
version. Rename the `Dockerfile` directory name and update any packages or
downloaded urls to point to the new version. Test building the image with
`docker build images/runtimes/<new_runtime>`.
2. Create a new [`runtime_test`](BUILD) target. The `name` field must be the
dirctory name for the Docker image you created in Step 1.
2. Update [`runtime_test`](BUILD) target. The `name` field must be the
directory name for the `Dockerfile` created in Step 1.
3. Run the tests, and triage any failures. Some language tests are flaky (or
3. Update [Buildkite pipeline](../../.buildkite/pipeline.yaml).
4. Run the tests, and triage any failures. Some language tests are flaky (or
never pass at all), other failures may indicate a gVisor bug or divergence
from Linux behavior. Known or expected failures can be added to the
[exclude](exclude) file for the new version, and they will be skipped in
future runs.
from Linux behavior.
5. Update the [exclude](exclude) file by renaming it with the right version and
adding any failing tests to it with a reason.
### Cleaning up exclude files
@@ -128,5 +130,3 @@ except that Step 1 is a bit harder. You have to figure out how to download and
run the language tests in a Docker container. Once you have that, you must also
implement the [`proctor/TestRunner`](proctor/lib/lib.go) interface for that
language, so that proctor can list and run the tests in the image you created.
[runtime-images]: ../../images/runtimes/