Google Home Test Suite

Cloud-to-cloud Local Home SDK

The Google Home Test Suite is a web application that allows you to self-test your smart home Action. The Test Suite automatically generates and runs test cases based on the devices and traits associated with your account. When you complete the tests, you can view the test results.

To use the Test Suite and submit your tests results to Google for certification, see the Test and share smart home Actions page for information. This version of the Test Suite is specifically for development and quality assurance.

The tests executed by the Test Suite include, but are not limited to, the following:

Checks that Report State and Request Sync are implemented for your devices and traits.
Verifies that the QUERY intent is implemented for your device and its results match Report State results.

You can launch the Test Suite tool by clicking the button below:

Launch Test Suite

Self-test your Action

The following section describes how to run the Test Suite on your device and smart home Action project. The Test Suite requires you to sign in so that it can inspect devices in your Google Home Graph and send commands directly to Google Assistant.

To self-test your Action, follow these steps:

In your web browser, open the Test Suite.
Sign-in to Google using the top-right corner button.

Important: Make sure you sign-in with an account that is associated with the Cloud-to-cloud integration you want to test.
Configure the Project Details:
1. In the Project ID field, enter the project identifier for your smart home Action.
2. Click Next.
Configure Test Settings:
1. If certifying your Cloud-to-cloud integration, leave the default Report State/Request Sync Timeout value unchanged. However, for testing, you may change the timeout value if your agent takes more time to report state to Home Graph.
2. If certifying your Cloud-to-cloud integration, make sure the Test Request Sync option is enabled—during testing, you will be prompted to add, update, or remove a device and check if the device list has changed in Home Graph. Change actions take place through the integration you are testing. Your integration under test must always have at least one device attached to the agent user.
  
  Warning: When removing a device, make sure to do this from your third-party app (not from the Google Home app (GHA)). Do not remove the device if there is only one device in your app that is account-linked, as this will cause a sync error.
3. If you are using a Scene trait, select Test Scene trait.
  
  Note: If you want to test traits with secondary user verification, select the Enable Secondary User Verification option in the Test Settings page. When this setting is enabled, the test app prompts you if the device responds with a challenge during the test. This option is only to help you complete the self-test. Devices using Secondary User Verification must still be sent to Google for manual verification.
4. If you are running the test suite for Local Home devices, select Supports Local Home SDK.
  1. Select Local Fulfillment or Seamless Setup.
  2. Click Supports Local Query if the device can handle QUERY intent locally.
Perform a Test Run and view results:
1. Click Next to start running tests. The interface shows the devices and traits that are being tested.
View the Results & Submission page:
1. Once the tests are complete, click Next to view the results.

The test results include the number of tests passed and failed, and a final score indicating percentage of passes.

Account linking validation test

Account linking plays an important role in the Google smart home ecosystem. The account linking validation test in Test Suite checks your OAuth service for potential issues.

The account linking validation test contains tests to validate access tokens and refresh tokens:

Access token validation test

Access tokens returned from your token exchange endpoint are tested to ensure they have a valid format and are returned with a valid refresh token.

Test Items	Explanation
Check if access token is JWT format	Google account linking does not recommend JWT format access tokens. If a JWT format is found, the following warning is displayed: `The access token seems to be JWT, this is not good. Because it unnecessarily exposes information about partner implementation and we recommend opaque symmetric encrypted access token.`
Check expirable access token has refresh token.	A refresh token must be provided when the access token is expirable. This test will fail if no refresh token is found.

Refresh token validation test

Refresh tokens are tested to ensure your token exchange endpoint properly exchanges them for new access tokens.

Test Items	Explanation
Refresh token with invalid refresh token, checking partner response.	Your server should return an HTTP 400 Bad Request error with `{"error": "invalid_grant"}` to an invalid refresh token request. If the response does not match the "error code or message", this test case will fail. Visit exchange refresh tokens for access tokens for more information.
Test if access token has been updated after token refresh.	New access tokens should be returned in response to refresh token requests. If your server provides the same access token, the test case will fail.
Check if the unexpired access token is still valid.	The access token expiry time will be displayed.
Check if refresh token has been rotated during refreshing.	We check if refresh tokens are changed after a refresh token request. If the refresh token changes, your server should only invalidate an old refresh token after a new refresh token has been used, to prevent race conditions that may break a user's account linking. The test will fail if you invalidate the old refresh token before the new one is used.

Error messages

The following table lists some common error messages you may encounter and the suggested way to resolve them.

Error Message How to Resolve

HomeGraph API has not been used in project <_id_> before or it is disabled Make sure that you have enabled the HomeGraph API.

Requested Entity not found Check that your agentUserId is valid and is linked to your service.

Error Message	How to Resolve
HomeGraph API has not been used in project <_id_> before or it is disabled	Make sure that you have enabled the HomeGraph API.
Requested Entity not found	Check that your `agentUserId` is valid and is linked to your service.
Make sure you have `resourcemanager.projects.get` permission on project `${your project id}` and have granted all the permissions in the consent screen. Error getting module data: getModuleData error: Http failure response for https://hometestsuiteproxy-pa.googleapis.com/v1/module:get: 403 OK 403	Follow the instructions at Troubleshoot IAM permissions to make sure your account has the `resourcemanager.projects.get` permissions for the project. Go to the Apps with access to your account page to check the following permissions for "Test suite for smart home": Basic account info See your primary Google Account email address See your personal info, including any personal info you've made publicly available Additional access Use your Assistant: broad access to your Google account Inspect and control the smart home devices in your Home Graph If you get a permission error, remove access first and grant all the permissions in the consent screen when you log in again.

Make sure you have resourcemanager.projects.get permission on project ${your project id} and have granted all the permissions in the consent screen.

Error getting module data: getModuleData error: Http failure response for https://hometestsuiteproxy-pa.googleapis.com/v1/module:get: 403 OK 403

Follow the instructions at Troubleshoot IAM permissions to make sure your account has the resourcemanager.projects.get permissions for the project.
Go to the Apps with access to your account page to check the following permissions for "Test suite for smart home":
- Basic account info
  1. See your primary Google Account email address
  2. See your personal info, including any personal info you've made publicly available
- Additional access
  1. Use your Assistant: broad access to your Google account
  2. Inspect and control the smart home devices in your Home Graph
If you get a permission error, remove access first and grant all the permissions in the consent screen when you log in again.

Submit test results

Your Test Suite results must pass 100% of tests before submitting your certification form. If you have any traits that require manual testing, you should still run the Test Suite tool and attach the result even if it has not passed at 100%.

To submit your test results and certify your Cloud-to-cloud integration, follow these steps:

After you receive your test results and you are ready to certify your Cloud-to-cloud integration, click Submit in the Test Suite tool.
Click the DOWNLOAD TEST RESULTS button to retrieve your test results.
Click the GO TO SUBMISSION FORM to submit the test results to Google.
Fill out all of the required fields, then click Submit.

Note: The test results that you submit must include the Request Sync test results. Once you submit the test results to Google, you need to submit your Cloud-to-cloud integration for review.