Dragon Copilot SDK for Android

Dragon Copilot SDK for Android enables you to build an embedded Dragon Copilot solution in your mobile apps. The SDK records and streams audio to Dragon Copilot for generative AI processing in a reliable and secure fashion.

System overview

This diagram gives you a high-level overview of the different system components and how they interact:

Android SDK high level overview

Concepts and terminology

This section outlines some basic concepts and terminology you need to be aware of when integrating Dragon Copilot SDK for Android.

  • Encounter

    An interaction between a patient and healthcare providers for the purpose of providing healthcare services or assessing the health status of a patient.

  • Recording

    An audio file created by the SDK during an encounter, capturing the interaction between the healthcare provider and the patient. Multiple recordings can be created for a single encounter, depending on the context and duration of the interaction.

  • Session

    A group of recordings associated with a specific encounter. These recordings collectively capture various aspects of the encounter and provide a comprehensive view of the patient-provider interaction.

  • Correlation ID

    The globally unique identifier for a session that's specific to the partner system and helps track and associate recordings with the corresponding encounter. The correlation ID must be associated with every recording of a session and is also associated with the AI output. The identifier must have the following characteristics:

    • Must be globally unique.

    • Must be opaque and not have any inherent meaning. It therefore doesn't contain any personal information (PII or PHI) and can be safely logged as plain text and shared between parties (customer, partner and Microsoft).

    • Can't be longer than 128 characters.

  • Payload

    A JSON file with an array of resources for a session. Each new recording triggers the creation of a new payload, allowing the partner system to access valuable insights from the encounter.

  • Resource

    The standards used to describe a category of AI-generated healthcare data. Each resource contains data elements necessary for communicating clinical concepts. For example, a clinical document section resource (such as HPI) can be imported into a note template, or a condition resource can supplement a problem list. Resources can be linked to other resources. For example, an order resource can be linked to a condition resource as the reason for placing the order.

The following diagram illustrates the relationships between a session, recordings, and payloads. Each additional recording for a session generates a new payload with revised or new resources.

relationships between a session, recordings and payloads

The following diagram shows the main events that trigger state changes in the system. After the stop recording event, processing starts. When processing is completed, the partner system is informed that a payload is available.

main events that trigger state changes

Resiliency

Streaming is the primary method of delivering audio to Dragon Copilot. However, Dragon Copilot SDK for Android also stores the audio locally as a backup in case of network interruptions. This approach allows users to continuously record even when network interruptions occur, and the upload finishes when the connection is restored. Your mobile app doesn't need to implement audio backup functionality because the SDK fully manages this feature.

The SDK saves backup audio files to local storage and encrypts them by using AES 256. It uses encryption keys that the SDK stores in platform-specific secure storage. These keys are unique for each recording and never leave the device, so the audio can only be decrypted on the respective device.

The following diagram shows the backup and encryption strategy that the SDK uses.

backup and encryption strategy

  • Last updated on