Name: running-release-tests
Author: aws

Run automated release testing (UI or API) via the AWS DevOps Agent using a pre-configured test profile. Use when the user wants to validate multi-step workflows, verify features, check for regressions, or test API endpoints. Trigger words include run tests, UAT, test my app, test profile, UI test, A

Release Testing AgentSpace routing (SigV4 only): If list agent spaces is available in your tool list and the multi space orchestration skill has NOT been invoked yet this session, invoke it first to determine which agent space id to use. Then pass agent space id on all tool calls below. For bearer token auth this is unnecessary — the token is already scoped to one space. Run automated release testing in the cloud via the AWS DevOps Agent's Release Testing Agent. Supports UI testing (browser based) and API testing (OpenAPI spec based). Uses pre existing test profiles that define target URL, agent type, personas, and credentials. Input is a test profile — the test profile already contains the target URL, agent type (UI or API), test personas, and credentials. Do NOT ask the user for a URL directly; the URL is defined in the test profile. Prerequisites A pre existing test profile (Knowledge Item ID like ki 12345 ) created from the AWS DevOps Agent console Gathering test parameters Before starting any workflow, you MUST gather the following parameters. Do NOT proceed to job creation until answered. Step 1 — Test profile (required) Ask the user which test profile to use. The test profile already contains the target URL, agent type (UI or API), test personas, and credentials configuration — these do NOT need to be gathered separately. Note: A pre existing test profile is a prerequisite. Test profiles are created using the AWS DevOps Agent console or API, not through this tool. If the user asks whether one can be created here, inform them it must already exist. Step 2 — Test requirement (optional) If the user has not already mentioned a test focus, ask: "Do you have a specific test requirement or focus area? If not, I'll run a full exploratory test." Wait for the user's response. If they provide one, use it as the test requirement . If they say no or skip, proceed without it. IMPORTANT: You MUST wait for the user to respond before proceeding to job creation. Core workflow 1. Select Agent Space List available agent spaces: Present the list to the user and ask which agent space they'd like to use. Do NOT proceed until the user has selected one. Use the selected agentSpaceId as SPACE ID in all subsequent calls. 2. Check tool availability Verify that the following tools are available: aws devops agent create release testing job , aws devops agent get task , aws devops agent list journal records , aws devops agent get release ui testing report , aws devops agent get release api testing report . These tools are NOT deferred/lazy loaded — if they do not appear in your tool list, they are unavailable. Do NOT search for them via ToolSearch. If any are missing, skip the remaining steps in this section and use the "Fallback (aws mcp)" path below instead. 3. Start the Job Record the taskId and executionId from the response. 4. Poll for Status Call aws devops agent get task(task id=TASK ID) every 30 seconds until the status transitions to IN PROGRESS or a terminal state. 5. Monitor Until Completion Once IN PROGRESS , poll for progress in a loop: 1. Call aws devops agent list journal records(execution id=EXEC ID, order="ASC") to fetch new findings. 2. Present each record to the user with a friendly progress update. 3. Use next token from the response to fetch only new records on subsequent polls. 4. Wait 20 seconds between each poll iteration. 5. Check aws devops agent get task(task id=TASK ID) periodically — stop when terminal status ( COMPLETED , FAILED , CANCELED , TIMED OUT ). 6. Present Results Once the job reaches a terminal status: If COMPLETED : 1. Determine the report type from the test profile's agent type (UI or API). Call aws devops agent get release ui testing report(execution id=EXEC ID) for UI profiles or aws devops agent get release api testing report(execution id=EXEC ID) for API profiles. 2. Write the report contents to a markdown file: 3. Inform the user that the report was saved, including the file path. If FAILED or TIMED OUT : Present the error information and suggest next steps. If CANCELED : Inform the user the job was canceled and no report is available. Cancelling a job Error handling 1. If the task status changes to FAILED , stop the workflow and report the error. 2. If the task does not reach IN PROGRESS within 5 minutes, cancel it using cancel release testing job . 3. If any output contains "NoCredentialsError", "ExpiredTokenException", or auth failures, suggest the user refresh their credentials or check the bearer token. 4. If throttled ( 429 or ThrottlingException ), wait 30 seconds before retrying. After 3 retries, inform the user. Fallback (aws mcp) If the aws devops agent remote server is unavailable, use the AWS CLI directly: Tell the user: "Remote server unavailable — using direct AWS API fallback." 1. Select Agent Space List available agent spaces: Present the list to the user and ask which agent space they'd like to use. Do NOT proceed until the user has selected one. Use the selected agentSpaceId as SPACE ID in all subsequent calls. 2. Start the Job If the user provided a test requirement, include it as webhookEventMessage . If not, omit the field or leave it empty. 3. Poll for Status Poll every 30 seconds until the status transitions to IN PROGRESS or a terminal state ( COMPLETED , FAILED , CANCELED , TIMED OUT ). 4. Monitor Until Completion Once IN PROGRESS , poll for progress in a loop: 1. Present each record to the user with a friendly progress update. 2. Use next token from the response to fetch only new records on subsequent polls. 3. Wait 20 seconds between each poll iteration. 4. Check get backlog task periodically — stop when terminal status ( COMPLETED , FAILED , CANCELED , TIMED OUT ). 5. Present Results Once the job reaches a terminal status: If COMPLETED : 1. Retrieve the report using the appropriate record type: UI testing : record type qa ui testing report API testing : record type qa api testing report 2. Write the report contents to a markdown file: 3. Inform the user that the report was saved, including the file path. If FAILED or TIMED OUT : Present the error information and suggest next steps. If CANCELED : Inform the user the job was canceled and no report is available. Cancelling (fallback)