Notifications for smart home Actions

Notifications allow your Cloud-to-cloud integration to use Google Assistant to communicate with users about important device-related events or changes. You can implement notifications to alert users to timely device events, for example when someone is at the door, or to report on a requested device state change, such as when a door lock bolt has been successfully engaged or has jammed.

Your Cloud-to-cloud integration can send the following types of notifications to users:

  • Proactive notifications: Alerts the user of a smart home device event without any preceding user requests to their devices, such as the doorbell ringing.

  • Follow-up responses: A confirmation that a device command request succeeded or failed, for example when locking a door. Use these alerts for device commands that take time to complete. Follow-up responses are only supported when device command requests are sent from smart speakers and smart displays.

Assistant provides these notifications to users as announcements on smart speakers and smart displays. Proactive notifications are turned off by default. Users have the ability to turn on or off all proactive notifications from the Google Home app (GHA).

Events that trigger notifications

When device events occur, your fulfilment sends a notification request to Google. The device traits that your Cloud-to-cloud integration supports determines what types of notification events are available and the data you can include in those notifications.

The following traits support proactive notifications:

Trait Events
ObjectDetection Objects detected by the device, such as when a recognized face is detected at the door. For example: "Alice and Bob are at the front door."
RunCycle Device completes a cycle. For example: "The washing machine cycle has completed."
SensorState Device detects a supported sensor state. For example: "The smoke detector detects smoke."

The following traits support follow-up responses:

Trait Events
LockUnlock Completion status and state change following execution of the action.devices.commands.LockUnlock device command. For example: "The front door has been locked" or "The front door is jammed."
NetworkControl Completion status and state change following execution of the action.devices.commands.TestNetworkSpeed device command. For example: "Your network speed test has finished. The download speed on the office router is currently 80.2 Kbps, and the upload speed is 9.3 Kbps."
OpenClose Completion status and state change following execution of the action.devices.commands.OpenClose device command. For example: "The front door has opened" or "The front door couldn’t be opened."

All device types support notifications for the applicable traits.

Build notifications for your Cloud-to-cloud integration

Add notifications to your Cloud-to-cloud integration in these stages:

  1. Indicate to Google if notifications are enabled from your smart home device app. If users turn notifications on or off in your app, send a SYNC request to inform Google of the device change.
  2. When a relevant device event or state change occurs that triggers a notification, send a notification request by calling the Report State reportStateAndNotification API. If the device state changed, you can send both a state and notification payload together in your Report State and Notification call.

The following sections cover these steps in more detail.

Indicate if notifications are enabled in your app

Users can choose whether they want to receive proactive notifications by enabling this feature in the GHA. In the app for your smart home device, you can also optionally add the ability for users to explicitly toggle notifications from the device, for example, from your app settings.

Indicate to Google that notifications are enabled for your device by making a Request SYNC call to update device data. You should send a SYNC request like this whenever users change this setting in your app.

In your SYNC response, send one of these updates:

  • If the user explicitly toggled notifications on in your device app, or if you do not provide a toggle option, set the devices.notificationSupportedByAgent property to true.
  • If the user explicitly toggled notifications off in your device app, set the devices.notificationSupportedByAgent property to false.

The following snippet shows an example of how to set your SYNC response:

devices: [{
   id: 'device123',
   ...
   notificationSupportedByAgent: true,
}]

Send notification requests to Google

To trigger notifications on the Assistant, your fulfillment sends a notification payload to the Google Home Graph via a Report State and Notification API call.

Enable the Google HomeGraph API

  1. In the Google Cloud Console, go to the HomeGraph API page.

    Go to the HomeGraph API page
  2. Select the project that matches your smart home project ID.
  3. Click ENABLE.

Create a Service Account Key

Follow these instructions to generate a service account key from the Google Cloud Console:

Note: Ensure that you are using the correct GCP project when performing these steps. This is the project that matches your smart home project ID.

  1. In the Google Cloud Console, go to the Create service account key page.

    Go to the Create Service Account Key page
  2. From the Service account list, select New service account.
  3. In the Service account name field, enter a name.
  4. In the Service account ID field, enter a ID.

  5. From the Role list, select Service Accounts > Service Account Token Creator.

  6. For the Key type, select the JSON option.

  7. Click Create. A JSON file that contains your key downloads to your computer.

Send the notification

Make the notification request call using the devices.reportStateAndNotification API. Your JSON request must include an eventId, which is a unique ID generated by your platform for the event triggering the notification. The eventId should be a random ID that is different every time you send a notification request.

In the notifications object that you pass in your API call, include a priority value which defines how the notification should be presented. Your notifications object may include different fields depending on the device trait.

Follow one of these paths to set the payload and call the API:

Send a proactive notification payload

To call the API, select an option from one of these tabs:

HTTP

The Home Graph API provides an HTTP endpoint

  1. Use the downloaded service account JSON file to create a JSON Web Token (JWT). For more information, see Authenticating Using a Service Account.
  2. Obtain an OAuth 2.0 access token with the https://www.googleapis.com/auth/homegraph scope using oauth2l:
    3. oauth2l fetch --credentials service-account.json \
  --scope https://www.googleapis.com/auth/homegraph
  3. Create the JSON request with the agentUserId. Here's a sample JSON request for Report State and Notification:
    4. {
  "agentUserId": "PLACEHOLDER-USER-ID",
  "eventId": "PLACEHOLDER-EVENT-ID",
  "requestId": "PLACEHOLDER-REQUEST-ID",
  "payload": {
    "devices": {
      "notifications": {
        "PLACEHOLDER-DEVICE-ID": {
          "ObjectDetection": {
            "priority": 0,
            "detectionTimestamp": 1534875126750,
            "objects": {
              "named": [
                "Alice"
              ],
              "unclassified": 2
            }
          }
        }
      }
    }
  }
}
  4. Combine the Report State and Notification JSON and the token in your HTTP POST request to the Google Home Graph endpoint. Here's an example of how to make the request in the command line using curl, as a test:
    5. curl -X POST -H "Authorization: Bearer ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d @request-body.json \
  "https://homegraph.googleapis.com/v1/devices:reportStateAndNotification"

gRPC

The Home Graph API provides a gRPC endpoint

  1. Get the protocol buffers service definition for the Home Graph API.
  2. Follow the gRPC developer documentation to generate client stubs for one of the supported languages .
  3. Call the ReportStateAndNotification method.

Node.js

The Google APIs Node.js Client provides bindings for the Home Graph API.

  1. Initialize the google.homegraph service using Application Default Credentials.
  2. Call the reportStateAndNotification method with the ReportStateAndNotificationRequest. It returns a Promise with the ReportStateAndNotificationResponse.
const homegraphClient = homegraph({
  version: 'v1',
  auth: new GoogleAuth({
    scopes: 'https://www.googleapis.com/auth/homegraph'
  })
});

const res = await homegraphClient.devices.reportStateAndNotification({
  requestBody: {
    agentUserId: 'PLACEHOLDER-USER-ID',
    eventId: 'PLACEHOLDER-EVENT-ID',
    requestId: 'PLACEHOLDER-REQUEST-ID',
    payload: {
      devices: {
        notifications: {
          'PLACEHOLDER-DEVICE-ID': {
            ObjectDetection: {
              priority: 0,
              detectionTimestamp: 1534875126750,
              objects: {
                named: ['Alice'],
                unclassified: 2
              }
            }
          }
        }
      }
    }
  }
});

Java

The HomeGraph API Client Library for Java provides bindings for the Home Graph API.

  1. Initialize the HomeGraphApiService using Application Default Credentials.
  2. Call the reportStateAndNotification method with the ReportStateAndNotificationRequest. It returns a ReportStateAndNotificationResponse.
// Get Application Default credentials.
GoogleCredentials credentials =
    GoogleCredentials.getApplicationDefault()
        .createScoped(List.of("https://www.googleapis.com/auth/homegraph"));

// Create Home Graph service client.
HomeGraphService homegraphService =
    new HomeGraphService.Builder(
            GoogleNetHttpTransport.newTrustedTransport(),
            GsonFactory.getDefaultInstance(),
            new HttpCredentialsAdapter(credentials))
        .setApplicationName("HomeGraphExample/1.0")
        .build();

// Build device notification payload.
Map<?, ?> notifications =
    Map.of(
        "ObjectDetection",
        Map.of(
            "priority", 0,
            "detectionTimestamp", 1534875126,
            "objects", Map.of("named", List.of("Alice"), "unclassifed", 2)));

// Send notification.
ReportStateAndNotificationRequest request =
    new ReportStateAndNotificationRequest()
        .setRequestId("PLACEHOLDER-REQUEST-ID")
        .setAgentUserId("PLACEHOLDER-USER-ID")
        .setEventId("PLACEHOLDER-EVENT-ID")
        .setPayload(
            new StateAndNotificationPayload()
                .setDevices(
                    new ReportStateAndNotificationDevice()
                        .setNotifications(Map.of("PLACEHOLDER-DEVICE-ID", notifications))));
homegraphService.devices().reportStateAndNotification(request);
Send a follow-up response payload

The payload for a follow-up response contains the status of the request, error codes for event failures, if applicable, and the valid followUpToken, provided during the EXECUTE intent request. The followUpToken must be used within five minutes to remain valid and to properly associate the response with the original request.

The following snippet shows an example EXECUTE request payload with a followUpToken field.

{
  "requestId": "ff36a3cc-ec34-11e6-b1a0-64510650abcf",
  "inputs": [{
    "intent": "action.devices.EXECUTE",
    "payload": {
      "commands": [{
        "devices": [{
          "id": "123",
        }],
        "execution": [{
          "command": "action.devices.commands.TestNetworkSpeed",
          "params": {
            "testDownloadSpeed": true,
            "testUploadSpeed": false,
            "followUpToken": "PLACEHOLDER"
          }
        }]
      }]
    }
  }]
};

Google uses the followUpToken to output the notification only on the device the user was originally interacting with, and not broadcast to across all the user's devices.

To call the API, select an option from one of these tabs:

HTTP

The Home Graph API provides an HTTP endpoint

  1. Use the downloaded service account JSON file to create a JSON Web Token (JWT). For more information, see Authenticating Using a Service Account.
  2. Obtain an OAuth 2.0 access token with the https://www.googleapis.com/auth/homegraph scope using oauth2l:
    3. oauth2l fetch --credentials service-account.json \
  --scope https://www.googleapis.com/auth/homegraph
  3. Create the JSON request with the agentUserId. Here's a sample JSON request for Report State and Notification:
    4. {
  "agentUserId": "PLACEHOLDER-USER-ID",
  "eventId": "PLACEHOLDER-EVENT-ID",
  "requestId": "PLACEHOLDER-REQUEST-ID",
  "payload": {
    "devices": {
      "notifications": {
        "PLACEHOLDER-DEVICE-ID": {
          "NetworkControl": {
            "priority": 0,
            "followUpResponse": {
              "status": "SUCCESS",
              "followUpToken": "PLACEHOLDER",
              "networkDownloadSpeedMbps": 23.3,
              "networkUploadSpeedMbps": 10.2
            }
          }
        }
      }
    }
  }
}
  4. Combine the Report State and Notification JSON and the token in your HTTP POST request to the Google Home Graph endpoint. Here's an example of how to make the request in the command line using curl, as a test:
    5. curl -X POST -H "Authorization: Bearer ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d @request-body.json \
  "https://homegraph.googleapis.com/v1/devices:reportStateAndNotification"

gRPC

The Home Graph API provides a gRPC endpoint

  1. Get the protocol buffers service definition for the Home Graph API.
  2. Follow the gRPC developer documentation to generate client stubs for one of the supported languages.
  3. Call the ReportStateAndNotification method.

Node.js

The Google APIs Node.js Client provides bindings for the Home Graph API.

  1. Initialize the google.homegraph service using Application Default Credentials.
  2. Call the reportStateAndNotification method with the ReportStateAndNotificationRequest. It returns a Promise with the ReportStateAndNotificationResponse.
const followUpToken = executionRequest.inputs[0].payload.commands[0].execution[0].params.followUpToken;

const homegraphClient = homegraph({
  version: 'v1',
  auth: new GoogleAuth({
    scopes: 'https://www.googleapis.com/auth/homegraph'
  })
});

const res = await homegraphClient.devices.reportStateAndNotification({
  requestBody: {
    agentUserId: 'PLACEHOLDER-USER-ID',
    eventId: 'PLACEHOLDER-EVENT-ID',
    requestId: 'PLACEHOLDER-REQUEST-ID',
    payload: {
      devices: {
        notifications: {
          'PLACEHOLDER-DEVICE-ID': {
            NetworkControl: {
              priority: 0,
              followUpResponse: {
                status: 'SUCCESS',
                followUpToken,
                networkDownloadSpeedMbps: 23.3,
                networkUploadSpeedMbps: 10.2,
              }
            }
          }
        }
      }
    }
  }
});

Java

The HomeGraph API Client Library for Java provides bindings for the Home Graph API.

  1. Initialize the HomeGraphApiService using Application Default Credentials
  2. Call the reportStateAndNotification method with the ReportStateAndNotificationRequest. It returns a ReportStateAndNotificationResponse
// Get Application Default credentials.
GoogleCredentials credentials =
    GoogleCredentials.getApplicationDefault()
        .createScoped(List.of("https://www.googleapis.com/auth/homegraph"));

// Create Home Graph service client.
HomeGraphService homegraphService =
    new HomeGraphService.Builder(
            GoogleNetHttpTransport.newTrustedTransport(),
            GsonFactory.getDefaultInstance(),
            new HttpCredentialsAdapter(credentials))
        .setApplicationName("HomeGraphExample/1.0")
        .build();

// Extract follow-up token.
ExecuteRequest.Inputs executeInputs = (Inputs) executeRequest.getInputs()[0];
String followUpToken =
    (String)
        executeInputs
            .getPayload()
            .getCommands()[0]
            .getExecution()[0]
            .getParams()
            .get("followUpToken");

// Build device follow-up response payload.
Map<?, ?> followUpResponse =
    Map.of(
        "NetworkControl",
        Map.of(
            "priority",
            0,
            "followUpResponse",
            Map.of(
                "status",
                "SUCCESS",
                "followUpToken",
                followUpToken,
                "networkDownloadSpeedMbps",
                23.3,
                "networkUploadSpeedMbps",
                10.2)));

// Send follow-up response.
ReportStateAndNotificationRequest request =
    new ReportStateAndNotificationRequest()
        .setRequestId("PLACEHOLDER-REQUEST-ID")
        .setAgentUserId("PLACEHOLDER-USER-ID")
        .setEventId("PLACEHOLDER-EVENT-ID")
        .setPayload(
            new StateAndNotificationPayload()
                .setDevices(
                    new ReportStateAndNotificationDevice()
                        .setNotifications(Map.of("PLACEHOLDER-DEVICE-ID", followUpResponse))));
homegraphService.devices().reportStateAndNotification(request);

Logging

Notifications support event logging as outlined in the Cloud logging for Cloud-to-cloud. These logs are useful for testing and maintaining notifications quality within your Action.

The following is the schema of a notificationLog entry:

Property Description
requestId Notification request ID.
structName Name of the notification struct, such as "ObjectDetection".
status Indicates the status of the notification.

The status field includes various statuses that may indicate errors in the notification payload. Some of these may only be available on Actions that have not been launched to production.

Example statuses include:

Status Description
EVENT_ID_MISSING Indicates that the required eventId field is missing.
PRIORITY_MISSING Indicates that a priority field is missing.
NOTIFICATION_SUPPORTED_BY_AGENT_FALSE Indicates that the notifying device's notificationSupportedByAgent property provided in SYNC is false.
NOTIFICATION_ENABLED_BY_USER_FALSE Indicates that the user has not enabled notifications on the notifying device in the GHA. This status is only available on integrations that have not been launched to production.
NOTIFYING_DEVICE_NOT_IN_STRUCTURE Indicates that the user has not assigned the notifying device to a Home/Structure. This status is only available on integrations that have not been launched to production.

In addition to these general statuses that can apply to all notifications, the status field may also include trait-specific statuses where applicable (for example, OBJECT_DETECTION_DETECTION_TIMESTAMP_MISSING).

Previous
Implement Report State
Next
Test a Cloud-to-cloud integration