mirror of
https://github.com/wavetermdev/backup.git
synced 2026-04-22 15:26:58 -07:00
WIP: TestDriver: Update to v3 (#690)
**Action Required:** We've moved to an authenticated system and you must now provide an authentication key within our script. 1. Create an account or log into https://app.dashcam.io. 2. Copy the API key from https://app.dashcam.io/team. You must be the team "owner" 3. Add the API key as `DASHCAM_API_KEY` within GitHub actions by following the guide [here](https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions). This PR moves to our v3 system which includes - Faster and more stable tests. Notice we're using a `yml` file to guide the test rather than open prompts. This means the AI will follow the same steps every time. - Smarter AI. Better text matching, and more commands. We optimized how the AI matches text and images, making it way faster and more fault-tolerant. - Better summaries. The AI will summarize the full test execution as well as errors - Assertions. We can now explicitly ask the AI to verify things on screen - Embedded tests. We can nest scripts for reusable components. --------- Co-authored-by: orliesaurus <orliesaurus@users.noreply.github.com>
This commit is contained in:
@@ -1,4 +1,4 @@
|
||||
name: TestDriver.ai Regression Testing
|
||||
name: TestDriver.ai Regression Testing - Waveterm
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
@@ -14,53 +14,37 @@ permissions:
|
||||
contents: read # To allow the action to read repository contents
|
||||
pull-requests: write # To allow the action to create/update pull request comments
|
||||
|
||||
|
||||
jobs:
|
||||
test:
|
||||
name: TestDriver
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: dashcamio/testdriver@main
|
||||
id: testdriver
|
||||
with:
|
||||
version: v2.12.12
|
||||
prerun: |
|
||||
rm ~/Desktop/WITH-LOVE-FROM-AMERICA.txt
|
||||
cd ~/actions-runner/_work/testdriver/testdriver/
|
||||
brew install go
|
||||
brew tap scripthaus-dev/scripthaus
|
||||
brew install corepack
|
||||
brew install scripthaus
|
||||
corepack enable
|
||||
yarn install
|
||||
scripthaus run build-backend
|
||||
echo "Yarn"
|
||||
yarn
|
||||
echo "Rebuild"
|
||||
scripthaus run electron-rebuild
|
||||
echo "Webpack"
|
||||
scripthaus run webpack-build
|
||||
echo "Starting Electron"
|
||||
scripthaus run electron 1>/dev/null 2>&1 &
|
||||
echo "Electron Done"
|
||||
exit
|
||||
prompt: |
|
||||
1. wait 10 seconds
|
||||
1. click "Get Started"
|
||||
1. validate that overlapping text does not appear in the application
|
||||
1. focus the Wave input with the keyboard shorcut Command + I
|
||||
1. type 'ls' into the input
|
||||
1. press return
|
||||
1. validate Wave shows the result of 'ls'
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
- uses: peter-evans/create-or-update-comment@v4
|
||||
if: ${{always()}}
|
||||
with:
|
||||
issue-number: ${{ github.event.pull_request.number }}
|
||||
body: |
|
||||
## TestDriver Summary
|
||||
${{ steps.testdriver.outputs.markdown }}
|
||||
${{ steps.testdriver.outputs.summary }}
|
||||
reactions: |
|
||||
+1
|
||||
-1
|
||||
test:
|
||||
name: "TestDriver"
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: dashcamio/testdriver@main
|
||||
id: testdriver
|
||||
with:
|
||||
version: v3.5.6
|
||||
key: ${{secrets.DASHCAM_API}}
|
||||
prerun: |
|
||||
cd ~/actions-runner/_work/testdriver/testdriver/
|
||||
brew install go
|
||||
brew tap scripthaus-dev/scripthaus
|
||||
brew install corepack
|
||||
brew install scripthaus
|
||||
corepack enable
|
||||
yarn install
|
||||
scripthaus run build-backend
|
||||
echo "Yarn"
|
||||
yarn
|
||||
echo "Rebuild"
|
||||
scripthaus run electron-rebuild
|
||||
echo "Webpack"
|
||||
scripthaus run webpack-build
|
||||
echo "Starting Electron"
|
||||
scripthaus run electron 1>/dev/null 2>&1 &
|
||||
echo "Electron Done"
|
||||
cd /Users/ec2-user/Downloads/td/
|
||||
npm rebuild
|
||||
exit
|
||||
prompt: |
|
||||
1. /run /Users/ec2-user/actions-runner/_work/testdriver/testdriver/.testdriver/wave1.yml
|
||||
|
||||
@@ -1,18 +0,0 @@
|
||||
rm ~/Desktop/WITH-LOVE-FROM-AMERICA.txt
|
||||
cd ~/actions-runner/_work/testdriver/testdriver/
|
||||
brew install go
|
||||
brew tap scripthaus-dev/scripthaus
|
||||
brew install scripthaus
|
||||
corepack enable
|
||||
yarn install
|
||||
scripthaus run build-backend
|
||||
echo "Yarn"
|
||||
yarn
|
||||
echo "Rebuild"
|
||||
scripthaus run electron-rebuild
|
||||
echo "Webpack"
|
||||
scripthaus run webpack-build
|
||||
echo "Starting Electron"
|
||||
scripthaus run electron 1>/dev/null 2>&1 &
|
||||
echo "Electron Done"
|
||||
exit
|
||||
@@ -0,0 +1,29 @@
|
||||
steps:
|
||||
- step:
|
||||
- command: focus-application
|
||||
name: Electron
|
||||
- command: hover-text
|
||||
description: Get Started
|
||||
action: click
|
||||
- command: hover-text
|
||||
description: Settings
|
||||
action: click
|
||||
- command: hover-text
|
||||
description: 13px
|
||||
action: click
|
||||
- command: hover-text
|
||||
description: 12px
|
||||
action: click
|
||||
- command: hover-text
|
||||
description: Dark
|
||||
action: click
|
||||
- command: hover-text
|
||||
description: Light
|
||||
action: click
|
||||
- command: hover-text
|
||||
description: workspace-1
|
||||
action: click
|
||||
- step:
|
||||
- command: assert
|
||||
description: the terminal is white
|
||||
|
||||
Reference in New Issue
Block a user