Quality Monitoring Dashboard

This suite of dashboards and alerts helps you proactively maintain a high-quality integration with the Google Home ecosystem. Google is committed to supporting partners in developing a high quality ecosystem for all customers

The dashboard has three sections, each covering a key part which contributes to the quality of an overall integration.

1. Google to Partner Metrics - Measures the health of calls from Google to your cloud backend.

2. System Health - Partner to Google Metrics - Measures the health of calls from your system to Google.

3. Device Health - State Accuracy - Measures the accuracy of states stored in Google systems, which are used to serve user queries.

When metrics don't meet their target values, they are highlighted in red to indicate an issue that could impact the user experience. The following information provides details on each target and why it matters to your users.

Go to Dashboard

Google to Partner metrics

The Query/Execute Success Rate >= 99.5% metric measures how often users' commands are fulfilled correctly, which helps avoid Assistant responses such as "I can't reach the device" or incorrectly confirming a command that wasn't fulfilled.

What defines a "Success"?

A transaction is marked as a success if the Google Home platform receives a valid response indicating the intended action was fulfilled or the requested state was retrieved.

Responses that include non-blocking exceptions (for example, a SUCCESS status accompanied by a lowBattery exception) are counted as successful transactions. The command reached the device and the intent was satisfied despite the warning.

What defines a "Failure"?

The errors found on Common platform error codes which are marked as Partner Actionable are considered "Failures" when calculating the QUERY and EXECUTE Success Rates.

The Query/Execute Latency (p90) <= 1000ms metric measures requested action wait time and helps ensure users don't have to wait too long, for example, waiting a few seconds for their light to turn off.

Latency metrics

Latency is a critical indicator of how responsive your integration feels to the end-user. The dashboard tracks the 90th Percentile (P90) Latency, which represents the experience of your "slowest" users (for example, a P90 of 800ms means 90% of requests are acknowledged in 800ms or less).

Google measures latency differently for status checks versus device commands to ensure technical accuracy.

1. QUERY Latency (Interrogative)

This measures the Cloud-to-cloud round trip time when Google asks for the current state of a device.

• Start: Google dispatches an action.devices.QUERY request to your fulfillment URL.
• Measurement Window: The time taken for your cloud to receive, process, and transmit the full HTTP response back to Google.
• End: Google receives and acknowledges the final response payload from your service.

2. EXECUTE Latency (Action)

This measures the command acknowledgment time when Google sends a control request to a device.

• Start: Google dispatches an action.devices.EXECUTE request to your fulfillment URL.
• Measurement Window: The time taken for your cloud to receive the command and return an acknowledgment response.
• End: Google receives the SUCCESS, PENDING, or OFFLINE status response.
• Technical Scope: This metric measures the "Response Ack" time between Google's cloud and your cloud. It does not measure the time it takes for the physical hardware (for example, a lightbulb) to complete the physical state change, as that often involves local mesh network latency outside of the cloud-to-cloud path.

System Health - Partner to Google metrics

Maintaining a Success Rate >= 99.5% helps ensure that device states are correct in Google Home, devices are added and removed, automations trigger, and history events appear in the Google Home app (GHA)'s Activity tab.

The Success Rate is calculated based on the HTTP response codes returned by Google when your cloud pushes state updates. To ensure partners are not penalized for Google-side infrastructure issues, the metric excludes Google internal errors from the failure count.

What defines a "Success"?

• 2xx (Success): The state update was successfully received and processed by Home Graph.

What defines a "Failure"?

• 4xx (Partner Error): These represent Failures and indicate an issue with the request sent from your cloud. Common codes include:
  • 400 Bad Request: The server was unable to process the request due to invalid syntax. Common causes include malformed JSON or using null instead of "" for a string value.
  • 404 Not Found: The requested resource couldn't be found. Typically, this means Google cannot find the requested device. It may also mean the user account is not linked or an invalid agentUserId was received. Ensure the agentUserId matches the value provided in your SYNC response and that you are properly handling DISCONNECT intents.
  • 429 Resource Exhausted: Your integration has exceeded its allotted quota. See the instructions in the "Step 1" section higher in the dashboard for quota management.

Device Health - State accuracy

Meeting or exceeding a State Accuracy >= 99.5% helps ensure users see correct results when they view device states or use AI features like Ask Home. If state accuracy is low, automations may not fire and history entries may not appear in the GHA's Activity tab at the right time. For more information, see Report State.

The quality dashboard tracks this hourly using two distinct metrics: Overall Accuracy and the Lowest Type/Trait Combo.

1. Accuracy Components

The metric is derived from "samples" where Google can verify the reported state against a known intent outcome.

2. Dashboard Metrics (Hourly Calculation)

The dashboard calculates accuracy based on a 1-hour interval. If an hour has fewer than 100 total samples (S_Total < 100), the accuracy for that hour is set to N/A.

View 1: Overall Accuracy (Global Average)

This represents the total accuracy of your integration across all device types and traits combined. It provides a weighted average of your entire ecosystem's health.

• Calculation: Total State Accuracy across all devices / Total State Total across all devices.

View 2: Lowest Type/Trait Combo

This identifies the most unreliable specific category in your integration. It prevents high-volume devices which are at high-quality from hiding low-volume devices which are at low quality. For example, if you have a high volume of lights above 99.5% State Accuracy, but a low volume of Switches at a low State Accuracy, this highlights the improvement needed on switches that may be lost in an average value.

• Calculation: Minimum of State Accuracy / State Total for all trait/device combinations.