Ambient Audio Streaming gRPC API reference

The Ambient Audio Streaming (AAS) 2.0 gRPC API enables partners to stream ambient audio recordings in real time and submit them for downstream processing by Dragon Copilot.

The gRPC transport is one of two streaming options (alongside WebSocket). It exposes three RPCs:

Authentication

All gRPC RPCs require a bearer token in the authorization metadata key.

Supported token types:

• S2S (Server-to-Server): Machine-to-machine token issued via MISE. After authentication, the service validates the calling application's identity against a configured allowlist.
• Entra ID User Token: User-delegated token issued by Microsoft Entra ID.
• EIS Bearer Token: JWT issued by the EHR Integration Service (EIS). See Token launch integration for details.

Required metadata

Conditionally required metadata

Recommended metadata

Service definition

service AudioStreamingService {
  rpc RetrieveConfiguration(RetrieveConfigurationRequest) returns (RetrieveConfigurationResponse);
  rpc RecordAmbient(stream RecordAmbientRequest) returns (stream RecordAmbientResponse);
  rpc StartProcessing(StartProcessingRequest) returns (StartProcessingResponse);
}

RPCs

RetrieveConfiguration

Type: Unary RPC

Retrieves the service configuration for a given product, partner, and customer. The response includes supported audio locales and recording duration limits. Call this before starting a recording session to determine available languages and duration constraints.

Request

Example request (C#):

var request = new RetrieveConfigurationRequest
{
    ProductId = "<product-guid>",
    PartnerId = "<partner-guid>",
    CustomerId = "<customer-guid>"
};

var response = await client.RetrieveConfigurationAsync(request);

Response

Example response:

{
  "encounterWarnSeconds": 2700,
  "encounterMaxSeconds": 4500,
  "supportedRecordingLocales": ["en-US", "fr-FR"],
  "supportedEncounterReportLocales": ["en-US", "fr-FR"]
}

Errors

RecordAmbient

Type: Bidirectional streaming

Streams an ambient audio recording to the server in three phases:

1. Open - The client sends a RecordingOpenRequest to initialize the session.
2. Stream - The client sends DataChunkRequest messages containing audio data. The server periodically sends DataStorageResponse messages indicating cumulative bytes stored.
3. Close - The client sends a RecordingCloseRequest to end the recording. The server responds with a RecordingCloseResponse confirming total bytes stored.

rpc RecordAmbient(stream RecordAmbientRequest) returns (stream RecordAmbientResponse);

Request messages

The client sends RecordAmbientRequest messages containing one of the following:

RecordingOpenRequest

Example:

{
  "recordingOpen": {
    "recordingId": "<recording-uuid>",
    "dataFormat": {
      "opus": {
        "sampleRateHz": 16000
      }
    },
    "ambientSessionData": {
      "productId": "<product-guid>",
      "partnerId": "<partner-guid>",
      "customerId": "<customer-guid>",
      "correlationId": "<correlation-guid>",
      "practitionerInfo": {
        "externalIdentifiers": [
          { "type": "fhirId", "identifier": "<practitioner-fhir-id>" }
        ],
        "name": {
          "givenName": "Jane",
          "familyName": "Smith",
          "suffix": "MD"
        }
      },
      "externalIdentifiers": [
        { "type": "userId", "identifier": "<external-user-id>" }
      ],
      "creationDate": "2026-02-20T14:30:00.000Z",
      "localeInfo": {
        "recordingLocales": ["en-US"],
        "encounterReportLocale": "en-US",
        "encounterUxLocale": "en-US"
      }
    },
    "actions": ["generate-draft"],
    "reason": "RECORDING_START_REASON_UI",
    "startingOffset": 0
  }
}

DataChunkRequest

Example:

{
  "dataChunk": {
    "dataStart": 0,
    "data": "<base64-encoded-audio-bytes>"
  }
}

RecordingCloseRequest

Example:

{
  "recordingClose": {
    "recordingId": "<recording-uuid>",
    "recordingLengthSeconds": 120,
    "reason": "RECORDING_STOP_REASON_UI"
  }
}

Response messages

The server sends RecordAmbientResponse messages containing one of the following:

DataStorageResponse example:

{
  "dataStored": {
    "dataStored": 32768
  }
}

RecordingCloseResponse example:

{
  "recordingCloses": {
    "dataStored": 65536
  }
}

Usage notes

• Send RecordingOpenRequest as the first message. Sending data before opening a session results in INVALID_ARGUMENT.
• data_start should reflect the byte offset from the beginning of the recording. Duplicate bytes (from reconnection) are silently discarded.
• Set starting_offset when resuming after interruption to the last confirmed data_stored value.
• previous_encounter_sessions is critical for pause/resume scenarios to connect sessions in the backend.
• Audio formats supported: PCM (signed 16-bit LE), Ogg Opus, and WebM Opus.

Errors

StartProcessing

Type: Unary RPC

Signals Dragon Copilot to begin processing a previously recorded ambient session. Processing happens asynchronously; the response confirms the request was accepted.

rpc StartProcessing(StartProcessingRequest) returns (StartProcessingResponse);

Request

Example request:

{
  "ambientSessionData": {
    "productId": "<product-guid>",
    "partnerId": "<partner-guid>",
    "customerId": "<customer-guid>",
    "correlationId": "<correlation-guid>",
    "practitionerInfo": {
      "externalIdentifiers": [
        { "type": "fhirId", "identifier": "<practitioner-fhir-id>" }
      ],
      "name": {
        "givenName": "Jane",
        "familyName": "Smith",
        "suffix": "MD"
      }
    },
    "externalIdentifiers": [
      { "type": "userId", "identifier": "<external-user-id>" }
    ],
    "localeInfo": {
      "recordingLocales": ["en-US"],
      "encounterReportLocale": "en-US",
      "encounterUxLocale": "en-US"
    }
  },
  "requestTime": "2026-02-20T14:35:00.000Z",
  "actions": ["generate-draft"],
  "recordingsToProcess": ["<recording-uuid-1>", "<recording-uuid-2>"]
}

Response

Success response:

{
  "streamingResponse": {
    "errorCode": 0,
    "errorMessage": "SUCCESS",
    "detailedErrorInformation": "SUCCESS"
  }
}

Error response:

{
  "streamingResponse": {
    "errorCode": 1,
    "errorMessage": "Error processing StartProcessing request",
    "detailedErrorInformation": ""
  }
}

Usage notes

• actions is required and must contain at least one action. An empty or null value results in INVALID_ARGUMENT.
• recordings_to_process is optional. When omitted, all recordings for the session (identified by correlation_id) are processed.
• Processing is asynchronous: a successful response confirms the request was accepted, not that processing is complete.
• Downstream failures are returned in the response body as a non-zero error_code, not as gRPC status exceptions.

Errors

Common types

AmbientSession

Session metadata provided when opening a recording or starting processing.

DataFormat

Audio encoding format. Specify exactly one of the following:

ExternalIdentifier

Best practices

1. Call RetrieveConfiguration before recording to confirm supported locales and duration limits.
2. Implement reconnection logic with exponential backoff for RecordAmbient streaming.
3. Track data_stored values to set starting_offset correctly when resuming after interruption.
4. Include previous_encounter_sessions when splitting recordings across multiple sessions.
5. Call StartProcessing after closing the recording stream. It is a separate unary call.
6. Monitor gRPC status codes to distinguish transient errors (RESOURCE_EXHAUSTED) from permanent failures (INVALID_ARGUMENT).