# Smart Home CameraStream Trait Schema

`action.devices.traits.CameraStream` - This trait covers how to control the camerastream of a device.

This trait belongs to devices which have the capability to stream video feeds to smart displays, Chromecast-enabled devices, or smartphones. By and large, these are security cameras or baby cameras. But this trait also applies to more complex devices which have a camera on them (for example, video-conferencing devices or a vacuum robot with a camera on it).

> [!NOTE]
> **Note:** CameraStream is not currently supported by [local fulfillment](https://developers.home.google.com/local-home/overview).

## Device ATTRIBUTES

Devices with this trait may report the following
attributes as part of the [`SYNC`](https://developers.home.google.com/cloud-to-cloud/intents/sync) operation. To learn
more about handling `SYNC` intents, see
[Intent fulfillment](https://developers.home.google.com/cloud-to-cloud/integration/sync).

| Attributes | Type | Description |
|---|---|---|
| `cameraStreamSupportedProtocols` | *Array* | **Required.** Supported media types for the camera stream, ordered by preference. Typically, the first protocol in this array that is compatible with the target surface is requested. |
| `[*item, ...*]` | *String* | Media type. Supported values: `hls` :   HTTP Live Streaming `dash` :   Dynamic Adaptive Streaming over HTTP `smooth_stream` :   Smooth Streaming `progressive_mp4` :   Progressive MP4 (mostly used for clips) `webrtc` :   WebRTC |
| `cameraStreamNeedAuthToken` | *Boolean* | **Required.** Indicates if an auth token is required to stream the camera feed. > [!NOTE] > If `cameraStreamNeedAuthToken` is true, the `cameraStreamAuthToken` field is mandatory, and the token needs to be scoped to just the target device. <br /> |

#### Examples

*Camera with multiple protocols support, not requiring an auth token.*

```
{
  "cameraStreamSupportedProtocols": [
    "webrtc",
    "hls",
    "dash",
    "smooth_stream",
    "progressive_mp4"
  ],
  "cameraStreamNeedAuthToken": false
}
```

*Camera supporting a single protocol, requiring an auth token.*

```
{
  "cameraStreamSupportedProtocols": [
    "hls"
  ],
  "cameraStreamNeedAuthToken": true
}
```

> [!NOTE]
> **Note**: Auth tokens are only supported when using a custom Cast receiver. The generic Cast receiver doesn't support authentication.

## Supported streaming protocols

This trait supports the following streaming protocols:

- `webrtc`: [WebRTC](https://webrtc.org)
- `hls`: [HTTP Live Streaming](https://en.wikipedia.org/wiki/HTTP_Live_Streaming)
- `dash`: [Dynamic Adaptive Streaming over HTTP](https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP)
- `smooth_stream`: [Smooth Streaming](https://en.wikipedia.org/wiki/Adaptive_bitrate_streaming#Apple_HTTP_Live_Streaming)
- `progressive_mp4`: Progressive MP4 (mostly used for clips)

When streaming `hls`,`dash`,`smooth_stream`, and
`progressive_mp4` protocols to Cast devices (Chromecasts, smart displays, and
Chromecast-enabled smart TVs), a Cast Web Receiver is launched to process the stream and render it
on the device. It is recommended that the developer create a
[Custom Web Receiver](https://github.com/googlecast/CastCameraReceiver) to
enable access to debugging tools, customize player behavior, customize UI branding, and include
analytics. To enable the usage of the custom receiver and opt out of using the default receiver,
set the receiver app id that is generated when you
[register your Cast application](https://developers.home.google.com/cast/docs/registration)
in the `EXECUTE` response's `cameraStreamReceiverAppId`
field.

To learn more about Custom Web Receiver applications, visit the Developer's Site
[guide.](https://developers.home.google.com/cast/docs/web_receiver/basic)

## Device STATES

None.

## Device COMMANDS

Devices with this trait may respond to the following
commands as part of the [`EXECUTE`](https://developers.home.google.com/cloud-to-cloud/intents/execute) operation. To learn
more about handling `EXECUTE` intents, see
[Intent fulfillment](https://developers.home.google.com/cloud-to-cloud/integration/query-execute#handle_execute_intents).

### `action.devices.commands.GetCameraStream`

#### Parameters

| Parameters | Type | Description |
|---|---|---|
| `StreamToChromecast` | *Boolean* | **Required.** Whether the stream will be played on a Chromecast device. |
| `SupportedStreamProtocols` | *Array* | **Required.** Media types/formats supported by the desired destination. |
| `[*item, ...*]` | *String* | Media type. Supported values: `hls` :   HTTP Live Streaming `dash` :   Dynamic Adaptive Streaming over HTTP `smooth_stream` :   Smooth Streaming `progressive_mp4` :   Progressive MP4 (mostly used for clips) `webrtc` :   WebRTC |

#### Results

| Results | Type | Description |
|---|---|---|
| `cameraStreamAuthToken` | *String* | An auth token for the specific receiver to authorize access to the stream. `cameraStreamAuthToken` is mandatory if `cameraStreamNeedAuthToken` is true, and the token needs to be scoped to just the target device. |
| `cameraStreamProtocol` | *String* | **Required.** The media format that the stream url points to. It should be one of the protocols listed in the `SupportedStreamProtocols` command parameter. Supported values: `hls` :   HTTP Live Streaming `dash` :   Dynamic Adaptive Streaming over HTTP `smooth_stream` :   Smooth Streaming `progressive_mp4` :   Progressive MP4 (mostly used for clips) `webrtc` :   WebRTC |

#### Non-WebRTC results

| Results | Type | Description |
|---|---|---|
| `cameraStreamAccessUrl` | *String* | **Required.** URL endpoint for retrieving the real-time stream in the format specified by `cameraStreamProtocol`. |
| `cameraStreamReceiverAppId` | *String* | Cast receiver ID to process the camera stream when the `StreamToChromecast` parameter is true; default receiver will be used if not provided. |

#### WebRTC results

| Results | Type | Description |
|---|---|---|
| `cameraStreamSignalingUrl` | *String* | **Required.** URL endpoint for retrieving and exchanging camera and client [session description protocols](https://en.wikipedia.org/wiki/Session_Description_Protocol) (SDPs). The client should return the signaling URL which uses the `cameraStreamAuthToken` as the authentication token in the request header. |
| `cameraStreamOffer` | *String* | Offer session description protocol (SDP). If the [Offer SDP](https://developers.home.google.com/cloud-to-cloud/traits/camerastream#sample-signaling-request-and-response) is included as part of the Execution(WebRTC) result, then Google will provide the Answer SDP at the signaling endpoint. If it isn't included in the Execution intent, then Google will generate the Offer SDP and Post to the signaling endpoint, expecting an Answer SDP as a response. Please also see [here](https://developers.home.google.com/cloud-to-cloud/traits/camerastream#webrtc-results). <br /> |
| `cameraStreamIceServers` | *String* | Represents the Interactive Connectivity Establishment (ICE) servers using an encoded JSON string with the description of a [RTCIceServer](https://developer.mozilla.org/en-US/docs/Web/API/RTCIceServer). If you do not specify STUN (Session Traversal Utilities for NAT) servers, the platform defaults to Google's public STUN servers. TURN (Traversal Using Relays around NAT) servers are only required if you cannot guarantee the IPs / ICE candidates provided will be publicly accessible (e.g. via a media server, public host ICE candidate, relay ICE candidate, etc). |

#### Examples

*Show the front door camera (default Cast receiver).*

```
{
  "cameraStreamAccessUrl": "https://fluffysheep.com/baaaaa.mp4",
  "cameraStreamProtocol": "progressive_mp4"
}
```

*Show front door camera (custom Cast receiver).*

```
{
  "cameraStreamAccessUrl": "https://fluffysheep.com/baaaaa.mp4",
  "cameraStreamReceiverAppId": "1g2f89213hg",
  "cameraStreamAuthToken": "12657342190192783",
  "cameraStreamProtocol": "progressive_mp4"
}
```

*Show front door camera (WebRTC stream).*

```
{
  "cameraStreamIceServers": "[{\"urls\": \"stun:stun.l.partner.com:19302\"},{\"urls\":\"turn:192.158.29.39:3478?transport=udp\",\"credential\": \"JZEOEt2V3Qb0y27GRntt2u2PAYA=\",\"username\": \"28224511:1379330808\"},{\"urls\": \"turn:192.158.29.39:3478?transport=tcp\",\"credential\": \"JZEOEt2V3Qb0y27GRntt2u2PAYA=\",\"username\": \"28224511:1379330808\"}]",
  "cameraStreamSignalingUrl": "https://example.com/signaling/answer",
  "cameraStreamOffer": "o=- 4611731400430051336 2 IN IP4 127.0.0.1...",
  "cameraStreamProtocol": "webrtc"
}
```

## Device ERRORS

See the full list of [errors and exceptions](https://developers.home.google.com/cloud-to-cloud/intents/errors-exceptions).

## WebRTC protocol specifications

The benefits of using WebRTC are low latency and 1-way talk. WebRTC uses a `POST` method with a `POST` body and response in JSON format.

WebRTC is currently supported on the Google Nest Smart Display and Chromecast with Google TV
devices.

This section describes the requirements when using the WebRTC stream protocol.

| Data Type | Parameters/Definitions |
|---|---|
| Signaling request header | The header should meet these requirements: - **Authentication** : The authentication header should use the auth token from the `GetCameraStream` return value for `cameraStreamAuthToken` with token type `Bearer`. - **Content-Type** : `application/json`. |
| Signaling request parameters | The request can include these parameters: - `action`: String. The valid values are: - `offer`: Offer SDP message from provider. - `answer`: Answer SDP message from provider. - `end`: Close the current session. - `deviceId`: String. The device ID as reported in a SYNC or EXECUTE request. - `sdp`: String. Contains the Session Description Protocol message for the peer connection. The content is based on the value of the `action` parameter. If the `action` is "end", this parameter can be empty. |
| Signaling response parameters | The response can include these parameters: - `action`: String. Response value must be of type `answer`. - `sdp`: String. SDP message for the responding answer. |

### WebRTC Requirements and Recommendations

- Google currently supports 1-way (half duplex) communication.
- You must support bundling and rtcp-mux.
- You must use (D)TLS 1.2 or later.
- Trickle ICE is not supported. All ICE candidates must be gathered first before sending the SDP.
- It is strongly recommended that you include UDP/IPv4, TCP/IPv4, UDP/IPv6 and TCP/IPv6 ICE candidates to increase the probability of a successful connection.

#### Supported video resolutions:

- **Minimum:** 480p
- **Maximum:** 1080p

#### Supported video codecs:

- VP8
- H.264

#### Supported audio codecs:

- Opus (preferred codec)
- G.711/PCMU
- G.722

### Cross-Origin Resource Sharing

Cross-Origin Resource Sharing (CORS) is a mechanism that uses additional HTTP Headers to tell
browsers to allow a web application running at one origin to access selected resources from a
different origin. The server hosting `cameraStreamSignalingUrl` should respond with the following header:

```
Access-Control-Allow-Origin: https://www.gstatic.com
```

### Sample signaling request and response

The following example shows a request that Google sends to
your signaling service and the corresponding response to Google.
Request

```
Header:

Authentication: Bearer <cameraStreamAuthToken>
Content-Type: application/json

POST body:

// When camera offer SDP is provided in the execution response, Google provides an answer SDP.
{
  "action": "answer",
  "deviceId": "123",
  "sdp": "o=- 4611731400430051336 2 IN IP4 127.0.0.1..."
}

// When camera offer SDP is not provided in execution response, Google generates and provides an offer SDP.
{
  "action": "offer",
  "deviceId": "123",
  "sdp": "o=- 4611731400430051336 2 IN IP4 127.0.0.1..."
}

// Close the current stream session.
{
  "action": "end"
  "deviceId": "123"
}
```
Response

```
// Response to accept the answer SDP in the request.
Response Code : 200
{}

// Response to provide the answer SDP from the service provider.
Response Code : 200
{
  // When the camera offer SDP is not provided in the execution response,
  // Google provides the answer SDP via the signaling response.
  "action": "answer"
  "sdp": "o=- 4611731400430051336 2 IN IP4 127.0.0.1..."
}

// Response to close current session.
Response Code : 200
{}
```