Test a voice agent with test cases

A test case simulates a conversation with your Voice Agent without a live call. You describe what the simulated caller does and what a successful result looks like. Kipps.AI runs the conversation against the agent's active instructions and tools, then reports whether it passed.

Use test cases to check changes to instructions or functions before you take calls.

Create a test case

1. Open your Voice Agent and select Test Cases in the sidebar.
2. On the Test Cases tab of the Test Cases Manager, select Add Test Case.
3. Enter a Name. Each test case name must be unique.
4. In User Prompt, describe what the simulated caller wants and how they behave. For example, "You want to book an appointment for tomorrow morning. Give the name Priya Shah."
5. In Success Criteria, describe what the agent must do for the test to pass. For example, "The agent confirms a booking date and time."
6. Under LLM Settings, select the Provider and Model for the test. If your workspace lets you use your own keys, select an API key.
7. Optional: under Test Variables & Mocks, select Add Variable and enter a Variable Name and Test Value for each dynamic variable the agent's instructions use.
8. Select OK.

Run test cases

• To run one test, select Test on its row.
• To run several tests, select Bulk Execute, check the test cases to run (or Select All), and select Execute.

Runs start in the background. Open the Test Results tab and select Refresh to see new results.

Review test results

The Test Results tab lists each run with the test case, the LLM used, a status, and when it ran. The status is PASSED, FAILED, or PENDING.

Select View on a run to open Test Run Details. It shows the score as a percentage, the status, any error message, and a Conversation & Tools Timeline with each user and assistant message and each tool the agent ran.

Edit or delete a test case

On the Test Cases tab, select Edit to change a test case, or Delete and confirm to remove it.

Tips for reliable tests

• Write one goal per test case so a failure points to one cause.
• State success criteria that you can check from the conversation, such as a confirmed date or a tool the agent must use.
• Run the same tests again after you change the agent's instructions or functions.

Troubleshooting

The dashboard shows "Failed to execute test case". Check that an LLM API key is set. Select an API key in LLM Settings on the test case, or set a key for the agent's LLM on the AI tab.

The result stays on PENDING. Select Refresh on the Test Results tab. Runs finish in the background and can take a short time.

The agent does not use the latest instructions. Tests run against the agent's active version. Save your changes on the AI tab first.