End-to-end AI customization manager workflow

Follow the steps below to validate that the AI customization manager tooling works end‑to‑end and that imported content (flowsheets and LDAs) is correct. Complete these steps before going live to ensure the best possible accuracy before first use. You can then create and run test cases to assess extraction accuracy against your flowsheet templates. Test cases are reusable, create a test once and run it at any time.

If a test case needs to be reviewed, use the Flowsheet analyzer to check for structural or configuration issues in the EHR that could affect row extraction. This process is iterative and can be repeated with each flowsheet template update from the EHR.

Testing goals

• Validate flowsheets and LDAs are imported correctly and tagged properly.
• Confirm that tools function as expected (for example, LDA viewer and Tune accuracy workflows).
• Ensure the correct environment is selected and aligned to the correct EHR instance.
• Identify structural or configuration issues early (for example, EHR rows not extracting).

Import validation & user acceptance test readiness

Validate that flowsheets and LDAs are imported and tagged correctly, and that AI customization manager tooling works as expected. All steps in this article occur in the AI customization manager.

1. Confirm user access

Verify that the Informatics Lead or Clinical Documentation Analyst has the correct permissions in Dragon admin center and can access the AI customization manager.

1. Sign in to Dragon admin center.
2. Select an environment.
3. Launch the AI customization manager.

If access fails, confirm you have the Dragon administrator role assigned; see: Role of a Dragon administrator.

2. Confirm imported flowsheet templates and LDAs align with the target EHR environment

Verify that flowsheet templates and LDA groups imported into each environment align with the correct EHR environment (for example, production).

• The user performing the export (EHR analyst or Informatics lead) should confirm which EHR environment the data originated from.
• Environment selection must be set correctly before the flowsheet templates are imported.

Common issues include selecting the wrong environment during import, or discovering missing or incorrect templates and LDAs after import.

3. Confirm LDA groups are imported and visible in the LDA viewer

Verify that LDA groups are listed in the LDA viewer.

• Expect to see approximately 20–100 LDA groups in the LDA viewer.
• LDAs and flowsheet templates are separate exports from the EHR; both must be imported.

4. Confirm ambient‑enabled flowsheet template metadata is correct

Verify that all ambient‑enabled templates have the service line, role and template type metadata assigned in the Flowsheet manager.

Metadata controls the behavior of the Tune accuracy tool. Incorrect assignments (for example, wrong service line, role, or template type) can affect test results.

5. Confirm ambient-enabled flowsheet templates are available to end‑user EHR profiles

Verify that flowsheet templates enabled for ambient recording align with the user's EHR department profile configuration.

• Confirm the flowsheet templates being ambient‑enabled are available to users in the EHR mobile app.
• Verify by checking EHR department profiles for the mobile EHR app.

6. Confirm ambient-enabled flowsheet template are unique for service line and role profiles

Verify that there are no duplicate Template Type assignments for the same Service Line plus Role combination:

• In the Flowsheet analyzer, review the Profiles tab to verify no unintentional duplicate template types are enabled.
• Check for multiple templates of the same Template Type in one profile.
• Document any unavoidable exceptions.

7. Confirm test cases are loaded in the Tune accuracy tool

Verify that onboarding test cases are listed in the Tune accuracy tool.

• Test cases are automatically deployed after flowsheet templates are ambient-enabled in the AI customization manager.
• You can populate expected results using Annotate with AI button on the Test cases tab of the Tune accuracy tool.

8. Confirm flowsheet templates appear in the Tune accuracy tool

Verify that ambient-enabled flowsheet templates appear when annotating a test case in Tune accuracy.

Open Tune accuracy and Create a new test case:

1. Enter any word, then select Next.
2. Select the template profile to run the test on.
3. Add an annotation to test.
4. Confirm that the enabled flowsheet template appears for selection.

FAQs

What are the considerations when enabling flowsheet templates for desktop and mobile apps?

Enabling desktop EHR flowsheet templates versus mobile EHR flowsheet templates directly impacts where and how users review their ambient observations.

For each ambient observation, Dragon Copilot sends the EHR the targeted template on which the observation should be displayed.

Mobile EHR flowsheet templates

• Users are required to review and edit ambient data using a mobile EHR template.
• This may be unfamiliar if the user typically works with the desktop EHR app.

Desktop EHR flowsheet templates

• Contain the full set of rows and row groupings users expect when reviewing documentation.
• Make it easier to identify missed documentation.
• Support clearer review prior to filing.
• Improve usability when reviewing observations on the recording Preview tab after pausing a recording.

What flowsheet templates should be ambient-enabled for a user?

Templates enabled for ambient recording should be unique per department profile.

When two versions of the same template (for example, Mobile EHR Vital Signs and Desktop EHR Vital Signs) are both enabled:

• Extracted observations might be split between nearly identical templates.
• The AI must choose which template each observation maps to.
• With similar templates, the AI might select the wrong template, creating inconsistent review experiences.

Can I test the difference between mobile EHR and desktop EHR flowsheet templates?

Yes, testing is supported and encouraged in non‑production environments.

Example: Testing a mobile EHR template

1. Enable the flowsheet template for mobile EHR Vital Signs.
2. Complete a vital signs ambient recording using the mobile EHR app.
3. Review and edit the ambient recording output using the desktop EHR app.