Connect smart home devices to the Google Assistant

1. Before you begin

As an Internet of Things (IoT) developer, you can build Cloud-to-cloud integrations that give your users the ability to control their devices through touch controls in the Google Home app and voice commands with the Assistant.

Cloud-to-cloud integrations rely on Home Graph to provide contextual data about the home and its devices, creating a logical map of the home. That context gives the Assistant a more natural understanding of the user's requests relative to their location in the home. For example, Home Graph can store the concept of a living room that contains multiple types of devices from different manufacturers, such as a thermostat, lamp, fan, and vacuum.

Prerequisites

• Create a Cloud-to-cloud integration Developer's Guide

What you'll build

In this codelab, you'll publish a cloud service that manages a virtual smart washing machine, then build a Cloud-to-cloud integration and connect it to the Assistant.

What you'll learn

• How to deploy a smart home cloud service
• How to connect your service to the Assistant
• How to publish device state changes to Google

What you'll need

• A web browser, such as Google Chrome
• An iOS or Android device with the Google Home app installed
• Node.js version 10.16 or later
• A Google Cloud billing account

2. Getting started

Enable Activity controls

In order to use the Google Assistant, you must share certain activity data with Google. The Google Assistant needs this data to function properly; however, the requirement to share data is not specific to the SDK. To share this data, create a Google Account if you don't already have one. You can use any Google Account—it does not need to be your developer account.

Open the Activity Controls page for the Google Account that you want to use with the Assistant.

Ensure the following toggle switches are enabled:

• Web & App Activity - In addition, be sure to select the Include Chrome history and activity from sites, apps, and devices that use Google services checkbox.
• Device Information
• Voice & Audio Activity

Create a Cloud-to-cloud Integration project

1. Go to the Developer Console.
2. Click Create Project, enter a name for the project, and click Create Project.

Select the Cloud-to-cloud Integration

On the Project Home in the Developer Console, select Add cloud-to-cloud integration under Cloud-to-cloud.

Install the Firebase CLI

The Firebase Command Line Interface (CLI) will allow you to serve your web apps locally and deploy your web app to Firebase hosting.

To install the CLI, run the following npm command from the terminal:

npm install -g firebase-tools

To verify that the CLI has been installed correctly, run:

firebase --version

Authorize the Firebase CLI with your Google Account by running:

firebase login

3. Run the starter app

Now that you set up your development environment, you can deploy the starter project to verify everything is configured properly.

Get the source code

Click the following link to download the sample for this codelab on your development machine:

file_downloadDownload source code

You can also clone the GitHub repository from the command line:

git clone https://github.com/google-home/smarthome-washer.git

About the project

The starter project contains the following subdirectories:

• public: A frontend UI to easily control and monitor the state of the smart washer.
• functions: A fully implemented cloud service that manages the smart washer with Cloud Functions for Firebase and Firebase Realtime Database.

Create a Firebase project

1. Go to Firebase.
2. Click Create a project and enter your project name.
3. Check the agreement checkbox and click Continue. If there's no agreement checkbox, you may skip this step.
   
4. Once your Firebase project is created, find the project ID. Go to Project Overview and click the settings icon > Project Settings.
   
5. Your project is listed under the General tab.
   

Connect to Firebase

Navigate to the washer-start directory, then set up the Firebase CLI with your integration project:

cd washer-start
firebase use <firebase-project-id>

Configure Firebase project

Initialize a Firebase project.

firebase init

Select the CLI features, Realtime Database, Functions, and the Hosting feature that includes Firebase Hosting.

? Which Firebase CLI features do you want to set up for this directory? Press Space to select features, then
 Enter to confirm your choices.
❯◉ Realtime Database: Configure a security rules file for Realtime Database and (optionally) provision default instance
 ◯ Firestore: Configure security rules and indexes files for Firestore
 ◉ Functions: Configure a Cloud Functions directory and its files
 ◉ Hosting: Configure files for Firebase Hosting and (optionally) set up GitHub Action deploys
 ◯ Hosting: Set up GitHub Action deploys
 ◯ Storage: Configure a security rules file for Cloud Storage
 ◯ Emulators: Set up local emulators for Firebase products
 ◯ Remote Config: Configure a template file for Remote Config
 ◯ Extensions: Set up an empty Extensions manifest

This will initialize the necessary APIs and features for your project.

When prompted, initialize Realtime Database. You can use the default location for the database instance.

? It seems like you haven't initialized Realtime Database in your project yet. Do you want to set it up?
Yes

? Please choose the location for your default Realtime Database instance:
us-central1

Since you are using the starter project code, choose the default file for the Security rules, and ensure you don't overwrite the existing database rules file.

? File database.rules.json already exists. Do you want to overwrite it with the Realtime Database Security Rules for <project-ID>-default-rtdb from the Firebase Console?
No

If you are reinitializing your project, select Overwrite when asked if you want to initialize or overwrite a codebase.

? Would you like to initialize a new codebase, or overwrite an existing one?
Overwrite

When configuring your Functions, you should use the default files, and ensure you don't overwrite the existing index.js and package.json files in the project sample.

? What language would you like to use to write Cloud Functions?
JavaScript

? Do you want to use ESLint to catch probable bugs and enforce style?
No

? File functions/package.json already exists. Overwrite?
No

? File functions/index.js already exists. Overwrite?
No

If you are reinitializing your project, select No when asked if you want to initialize or overwrite functions/.gitignore.

? File functions/.gitignore already exists. Overwrite?
No

? Do you want to install dependencies with npm now?
Yes

Finally, configure your Hosting setup to use the public directory in the project code, and use the existing index.html file. Select No when asked to use ESLint.

? What do you want to use as your public directory?
public

? Configure as a single-page app (rewrite all urls to /index.html)?
Yes

? Set up automatic builds and deploys with GitHub?
No

? File public/index.html already exists. Overwrite?
 No

If ESLint was accidentally enabled, there are two methods available to disable it:

1. Using the GUI, go to the ../functions folder under the project, select the hidden file .eslintrc.js and delete it. Do not mistake it for the similarly named .eslintrc.json.
2. Using the command line:

   cd functions
   rm .eslintrc.js
   

In the washer-done/firebase.json file, complete the code with:

{
  "database": {
    "rules": "database.rules.json"
  },
  "hosting": {
    "public": "public",
    "ignore": [
      "firebase.json",
      "**/.*",
      "**/node_modules/**"
    ],
    "rewrites": [
      {
        "source": "**",
        "destination": "/index.html"
      }
    ]
  },
    "headers": [{
      "source" : "**/*.@(js|html)",
      "headers" : [ {
        "key" : "Cache-Control",
        "value" : "max-age=0"
      } ]
    }],
  "functions": [
    {
      "source": "functions",
      "codebase": "default",
      "ignore": [
        "node_modules",
        ".git",
        "firebase-debug.log",
        "firebase-debug.*.log",
        "*.local"
      ]
    }
  ]
}

Deploy to Firebase

Now that you have installed the dependencies and configured your project, you are ready to run the app for the first time.

firebase deploy

This is the console output you should see:

...

✔ Deploy complete!

Project Console: https://console.firebase.google.com/project/<firebase-project-id>/overview
Hosting URL: https://<firebase-project-id>.web.app

This command deploys a web app, along with several Cloud Functions for Firebase.

Open the Hosting URL in your browser (https://.web.app) to view the web app. You will see the following interface:

This web UI represents a third-party platform to view or modify device states. To begin populating your database with device information, click UPDATE. You won't see any changes on the page, but the current state of your washer will be stored in the database.

Now it's time to connect the cloud service you've deployed to the Google Assistant using the Google Home Developer Console.

Configure your Developer Console project

On the Develop tab, add a Display Name for your interaction. This name will appear in the Google Home app.

Under App branding, upload a png file for the app icon, sized 144 x 144px, and named .png.

To enable Account linking use these account linking settings:

Under Cloud fulfillment URL, enter the URL for your cloud function that provides fulfillment for the smart home intents.

https://us-central1--cloudfunctions.net/smarthome

Click Save to save your project configuration, then click Next: Test to enable testing on your project.

Now you can begin implementing the webhooks necessary to connect the device state with the Assistant.

4. Create a washer

Now that you configured your integration, you can add devices and send data. Your cloud service needs to handle the following intents:

• A SYNC intent occurs when the Assistant wants to know what devices the user has connected. This is sent to your service when the user links an account. You should respond with a JSON payload of all the user's devices and their capabilities.
• A QUERY intent occurs when the Assistant wants to know the current state or status of a device. You should respond with a JSON payload with the state of each requested device.
• An EXECUTE intent occurs when the Assistant wants to control a device on a user's behalf. You should respond with a JSON payload with the execution status of each requested device.
• A DISCONNECT intent occurs when the user unlinks their account from the Assistant. You should stop sending events for this user's devices to the Assistant.

You will update the functions that you previously deployed to handle these intents in the following sections.

Update SYNC response

Open functions/index.js, which contains the code to respond to requests from the Assistant.

You will need to handle a SYNC intent by returning the device metadata and capabilities. Update the JSON in the onSync array to include the device information and recommended traits for a clothes washer.

index.js

app.onSync((body) => {
  return {
    requestId: body.requestId,
    payload: {
      agentUserId: USER_ID,
      devices: [{
        id: 'washer',
        type: 'action.devices.types.WASHER',
        traits: [
          'action.devices.traits.OnOff',
          'action.devices.traits.StartStop',
          'action.devices.traits.RunCycle',
        ],
        name: {
          defaultNames: ['My Washer'],
          name: 'Washer',
          nicknames: ['Washer'],
        },
        deviceInfo: {
          manufacturer: 'Acme Co',
          model: 'acme-washer',
          hwVersion: '1.0',
          swVersion: '1.0.1',
        },
        willReportState: true,
        attributes: {
          pausable: true,
        },
      }],
    },
  };
});

Deploy to Firebase

Deploy the updated cloud fulfillment using the Firebase CLI:

firebase deploy --only functions

Link to Google Assistant

In order to test your Cloud-to-cloud integration, you need to link your project with a Google Account. This enables testing through Google Assistant surfaces and the Google Home app that are signed in to the same account.

1. On your phone, open the Google Assistant settings. Note that you should be logged in as the same account as in the console.
2. Navigate to Google Assistant > Settings > Home Control (under Assistant).
3. Click the search icon in the upper right.
4. Search for your test app using the [test] prefix to find your specific test app.
5. Select that item. The Google Assistant will then authenticate with your service and send a SYNC request, asking your service to provide a list of devices for the user.

Open the Google Home app and verify that you can see your washer device.

5. Handle commands and queries

Now that your cloud service properly reports the washer device to Google, you need to add the ability to request the device state and send commands.

Handle QUERY intent

A QUERY intent includes a set of devices. For each device, you should respond with its current state.

In functions/index.js, edit the QUERY handler to process the list of target devices contained in the intent request.

index.js

app.onQuery(async (body) => {
  const {requestId} = body;
  const payload = {
    devices: {},
  };
  const queryPromises = [];
  const intent = body.inputs[0];
  for (const device of intent.payload.devices) {
    const deviceId = device.id;
    queryPromises.push(queryDevice(deviceId)
        .then((data) => {
        // Add response to device payload
          payload.devices[deviceId] = data;
        }
        ));
  }
  // Wait for all promises to resolve
  await Promise.all(queryPromises);
  return {
    requestId: requestId,
    payload: payload,
  };
});

For each device contained in the request, return the current state stored in the Realtime Database. Update the queryFirebase and queryDevice functions to return the state data of the washer.

index.js

const queryFirebase = async (deviceId) => {
  const snapshot = await firebaseRef.child(deviceId).once('value');
  const snapshotVal = snapshot.val();
  return {
    on: snapshotVal.OnOff.on,
    isPaused: snapshotVal.StartStop.isPaused,
    isRunning: snapshotVal.StartStop.isRunning,
  };
};

const queryDevice = async (deviceId) => {
  const data = await queryFirebase(deviceId);
  return {
    on: data.on,
    isPaused: data.isPaused,
    isRunning: data.isRunning,
    currentRunCycle: [{
      currentCycle: 'rinse',
      nextCycle: 'spin',
      lang: 'en',
    }],
    currentTotalRemainingTime: 1212,
    currentCycleRemainingTime: 301,
  };
};

Handle EXECUTE intent

The EXECUTE intent handles commands to update device state. The response returns the status of each command—for example, SUCCESS, ERROR, or PENDING—and the new device state.

In functions/index.js, edit the EXECUTE handler to process the list of traits that need updates and the set of target devices for each command:

index.js

app.onExecute(async (body) => {
  const {requestId} = body;
  // Execution results are grouped by status
  const result = {
    ids: [],
    status: 'SUCCESS',
    states: {
      online: true,
    },
  };

  const executePromises = [];
  const intent = body.inputs[0];
  for (const command of intent.payload.commands) {
    for (const device of command.devices) {
      for (const execution of command.execution) {
        executePromises.push(
            updateDevice(execution, device.id)
                .then((data) => {
                  result.ids.push(device.id);
                  Object.assign(result.states, data);
                })
                .catch(() => functions.logger.error('EXECUTE', device.id)));
      }
    }
  }

  await Promise.all(executePromises);
  return {
    requestId: requestId,
    payload: {
      commands: [result],
    },
  };
});

For each command and target device, update the values in the Realtime Database that correspond to the requested trait. Modify the updateDevice function to update the appropriate Firebase reference and return the updated device state.

index.js

const updateDevice = async (execution, deviceId) => {
  const {params, command} = execution;
  let state; let ref;
  switch (command) {
    case 'action.devices.commands.OnOff':
      state = {on: params.on};
      ref = firebaseRef.child(deviceId).child('OnOff');
      break;
    case 'action.devices.commands.StartStop':
      state = params.start
      ? {isRunning: true, isPaused: false}
      : {isRunning: false, isPaused: false};
      ref = firebaseRef.child(deviceId).child('StartStop');
      break;
    case 'action.devices.commands.PauseUnpause':
      const data = await queryDevice(deviceId);
      state = (data.isPaused === false && data.isRunning === false)
        ? {isRunning: false, isPaused: false}
        : {isRunning: !params.pause, isPaused: params.pause};
      ref = firebaseRef.child(deviceId).child('StartStop');
      break;
  }

  return ref.update(state)
      .then(() => state);
};

6. Test your Integration

After you implement all three intents, you can test that your integration controls the washer.

Deploy to Firebase

Deploy the updated cloud fulfillment using the Firebase CLI:

firebase deploy --only functions

Test the washer

Now you can see the value change when you try any of the following voice commands through your phone:

"Hey Google, turn on my washer."

"Hey Google, pause my washer."

"Hey Google, stop my washer."

You can also see the current state of your washer by asking questions.

"Hey Google, is my washer on?"

"Hey Google, is my washer running?"

"Hey Google, what cycle is my washer on?"

You can view these queries and commands in the logs that appear under your function in the Functions section of the Firebase Console. Learn more about Firebase logs in Write and view logs.

You can also find these queries and commands in the Google Cloud Console by navigating to Logging > Logs Explorer. Learn more about Google Cloud logging in Access event logs with Cloud Logging.

7. Report updates to Google

You have fully integrated your cloud service with the smart home intents, enabling users to control and query the current state of their devices. However, the implementation still lacks a way for your service to proactively send event information—such as changes to device presence or state—to the Assistant.

With Request Sync, you can trigger a new sync request when users add or remove devices, or when their device capabilities change. With Report State, your cloud service can proactively send a device's state to Home Graph when users physically change a device state—for example, turning on a light switch—or change the state using another service.

In this section, you will add code to call these methods from the frontend web app.

Enable the HomeGraph API

The HomeGraph API enables the storage and querying of devices and their states within a user's Home Graph. To use this API, you must first open the Google Cloud console and enable the HomeGraph API.

In the Google Cloud console, make sure to select the project that matches your integration <project-id>. Then, in the API Library screen for the HomeGraph API, click Enable.

Enable Report State

Writes to the Realtime Database trigger the reportstate function in the starter project. Update the reportstate function in functions/index.js to capture the data written to the database and post it to Home Graph via Report State.

index.js

exports.reportstate = functions.database.ref('{deviceId}').onWrite(
    async (change, context) => {
      functions.logger.info('Firebase write event triggered Report State');
      const snapshot = change.after.val();

      const requestBody = {
        requestId: 'ff36a3cc', /* Any unique ID */
        agentUserId: USER_ID,
        payload: {
          devices: {
            states: {
              /* Report the current state of our washer */
              [context.params.deviceId]: {
                on: snapshot.OnOff.on,
                isPaused: snapshot.StartStop.isPaused,
                isRunning: snapshot.StartStop.isRunning,
              },
            },
          },
        },
      };

      const res = await homegraph.devices.reportStateAndNotification({
        requestBody,
      });
      functions.logger.info('Report state response:', res.status, res.data);
    });

Enable Request Sync

Refreshing the icon in the frontend web UI triggers the requestsync function in the starter project. Implement the requestsync function in functions/index.js to call the HomeGraph API.

index.js

exports.requestsync = functions.https.onRequest(async (request, response) => {
  response.set('Access-Control-Allow-Origin', '*');
  functions.logger.info(`Request SYNC for user ${USER_ID}`);
  try {
    const res = await homegraph.devices.requestSync({
      requestBody: {
        agentUserId: USER_ID,
      },
    });
    functions.logger.info('Request sync response:', res.status, res.data);
    response.json(res.data);
  } catch (err) {
    functions.logger.error(err);
    response.status(500).send(`Error requesting sync: ${err}`);
  }
});

Deploy to Firebase

Deploy the updated code using the Firebase CLI:

firebase deploy --only functions

Test your implementation

Click the Refresh button in the web UI and verify that you see a sync request in the Firebase console log.

Next, adjust the attributes of the washer device in the frontend web UI and click Update. Verify that you can see the state change reported to Google in your Firebase console logs.

8. Congratulations

Congratulations! You successfully integrated the Assistant with a device cloud service using Cloud-to-cloud integrations.

Learn more

Here are some ideas you can implement to go deeper:

• Add modes and toggles to your device.
• Add more supported traits to your device.
• Explore local execution for smart home.
• Check out our GitHub sample to explore more.

You can also learn more about testing and submitting an integration for review, including the certification process to publish your integration to users.