Run a Field Trial

Your Field Trial will take place in two main phases:

  1. Plan the Field Trial
  2. Run the Field Trial

Plan the Field Trial

Planning a Field Trial involves the following steps:

  1. Recruit users

    • Identify users for field trials.
    • Collect shipping and contact details.
  2. Create a help page with the following information:

    • Project details
    • Known issues/bugs
    • Links to download the Google Home app (GHA) as well as your app(s)
    • Setup instructions, instructional videos
    • Things to try
    • Instructions for resetting the device to factory defaults
    • Troubleshooting steps
    • Link to the survey

Run the Field Trial

The first stage in running your Field Trial is to set it up on the Google Home Developer Console and submit it for Google's policy review. Once Google approves it, you can proceed with the Field Trial.

Set up and submit for policy review

Go to the Developer Console

  1. Go to Matter > Field Trial.

  2. Select the integration version from the Ready for Field Trial review list that will be the focus of the Field Trial, then click Submit.

    Ready for Field
Trial

  3. In the pop-up dialog, enter the emails of the users who will field test your integration. Optionally, click Open template to make a copy of a Field Trial Survey that you can customize for your product. When ready, click Submit.

Your Field Trial plan is now listed in the Submitted for review section and is sent to Google for policy review. The Field Trial automatically starts upon approval.

Policy review focuses on such things as your device or product name, device images, and video links.

Administer the Field Trial

Once the Field Trial is approved and appears in the Started section:

  1. Establish a formal communication channel with users over email or chat.

  2. Ship devices to field testers.

  3. Send resources to field testers users such as your help page and a link to the survey.

  4. Support your field testers:

    • Answer questions and help resolve issues experienced in the field.
    • Encourage users to share logs, file bugs, and send feedback from the GHA when issues arise.
  5. Monitor dashboards and logs using the Google Cloud Monitoring dashboard (accessible from the View link in the Developer Console Manage projects page):

    • Monitor setup success, including provisioning, registration, and request execution
    • Monitor engagement metrics such as Daily Active Users (DAU), Weekly Active users (WAU), or your own custom metrics.
    • Monitor logs for errors.

Edit testers

The list of testers can be updated for a Field Trial that is in progress.

Click Edit testers for the integration version in the Started section. This brings up a dialog where the list of testers may be updated. Click Save to save any changes.

Stop a Field Trial

To stop a Field Trial:

Go to the Developer Console

  1. Go to Matter > Field Trial.
  2. Select the Field Trial in the Started section you wish to stop.
  3. Click the Stop button.

The Field Trial should now appear in the Ended section.

Restart a Field Trial

To restart a Field Trial:

Go to the Developer Console

  1. Go to Matter > Field Trial.
  2. Select the Field Trial in the Ended section you wish to restart.
  3. Click the Restart button.

The Field Trial should now appear in the Started section.

Console statuses

The following Console statuses are encountered in the Field Trial phase:

Section Status Description Requirement What to do next
Ready for Field Trial review Ready This version of the integration can be submitted for Field Trial review. Withdrawn integrations are also labeled 'Ready'. The integration must be versioned. Submit this version of the integration for Field Trial review.
Submitted for review In review This version of the integration is in review. FT successfully submitted.

Wait for the version to be approved so the Field Trial can start.

or

Withdraw, if desired.

Submitted for review Rejected This version of the integration didn't meet policy review criteria, so a Field Trial can't be started. Field Trial criteria weren't met.

Fix issues in the existing integration, or create a new version of the integration that meets the Field Trial criteria, then contact your account manager and submit for Field Trial review.

or

If you don't agree with the issues, you can dispute the outcome with your account manager.

Started Started This version of the integration was approved and testers can start testing. Field Trial was approved.

Send info to invited testers.

or

Stop the Field Trial.

Ended Complete Field Trial for this version was stopped and testers can no longer participate. Field Trial ran. Restart Field Trial, if desired.