feat(ehr): contribution workflow piece #3891
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Ref: ENG-72 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
## Walkthrough
This change introduces a new multi-stage workflow for contributing EHR resource diff bundles, including new API endpoints, Lambda handlers, SQS queues, and supporting infrastructure. It implements logic for hydrating, preparing, and contributing resource diffs, updates job status handling, and extends bundle management to support resource-level operations. The update also refactors development and production code paths for better modularity.
## Changes
| Files / Paths | Change Summary |
|-----------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `packages/api/src/external/ehr/shared/command/bundle/contribute-resource-diff-bundle.ts`<br>`packages/api/src/external/ehr/shared/utils/bundle/types.ts`<br>`packages/api/src/routes/internal/ehr/patient.ts` | Adds a new function and API endpoint for contributing resource diff bundles, introduces a new params type, and updates route documentation and logic to use the new contribution workflow instead of setting entry status. |
| `packages/core/src/external/ehr/api/bundle/contribute-resource-diff-bundle.ts` | Adds a new API client function and parameter type for contributing resource diff bundles via HTTP, with error handling and retry logic. |
| `packages/core/src/external/ehr/api/job/create-resource-diff-bundles/set-entry-status.ts` | Removes the function and type for setting resource diff job entry status, as this is now handled differently. |
| `packages/core/src/external/ehr/athenahealth/command/get-resource-bundle-by-resource-id.ts`<br>`packages/core/src/external/ehr/canvas/command/get-resource-bundle-by-resource-id.ts`<br>`packages/core/src/external/ehr/command/get-resource-bundle-by-resource-id.ts` | Adds new functions and unified interface for retrieving FHIR resource bundles by resource ID from AthenaHealth and Canvas EHRs. |
| `packages/core/src/external/ehr/athenahealth/index.ts`<br>`packages/core/src/external/ehr/canvas/index.ts` | Adds support for resource-level bundle fetching, new supported resource types, and type guards for reference resources. Adds new methods to EHR API classes for retrieving bundles by resource ID. |
| `packages/core/src/external/ehr/bundle/bundle-shared.ts`<br>`packages/core/src/external/ehr/bundle/command/create-or-replace-bundle.ts`<br>`packages/core/src/external/ehr/bundle/command/fetch-bundle.ts` | Adds support for resource-level bundle keys with optional `resourceId`, new bundle type for resource diff data contributions, and updates bundle creation and fetching logic to handle resource IDs and mixed resource types. |
| `packages/core/src/external/ehr/job/create-resource-diff-bundles/steps/compute/ehr-compute-resource-diff-bundles-direct.ts`<br>`packages/core/src/external/ehr/job/create-resource-diff-bundles/steps/compute/ehr-compute-resource-diff-bundles-factory.ts`<br>`packages/core/src/external/ehr/job/create-resource-diff-bundles/steps/contribute/ehr-contribute-resource-diff-bundles-cloud.ts`<br>`packages/core/src/external/ehr/job/create-resource-diff-bundles/steps/contribute/ehr-contribute-resource-diff-bundles-direct.ts`<br>`packages/core/src/external/ehr/job/create-resource-diff-bundles/steps/contribute/ehr-contribute-resource-diff-bundles-factory.ts`<br>`packages/core/src/external/ehr/job/create-resource-diff-bundles/steps/contribute/ehr-contribute-resource-diff-bundles.ts` | Refactors resource diff bundle computation and contribution steps, introducing direct and cloud handlers, a factory for handler instantiation, and an interface for contribution. Adds logic for hydrating, preparing, and contributing bundles, with error handling and SQS integration.
8000
|
| `packages/core/src/external/ehr/job/create-resource-diff-bundles/steps/refresh/ehr-refresh-ehr-bundles-direct.ts`<br>`packages/core/src/external/ehr/job/create-resource-diff-bundles/steps/refresh/ehr-refresh-ehr-bundles-factory.ts` | Renames and refactors refresh bundle handler for direct execution, updates factory to use new class. |
| `packages/core/src/util/config.ts` | Adds a new configuration method for retrieving the contribute diff bundles queue URL from environment variables. |
| `packages/infra/lib/api-stack.ts`<br>`packages/infra/lib/ehr-nested-stack.ts` | Adds Lambda and SQS resources for resource diff bundle contribution to the infrastructure stack, wires up permissions, environment, and event sources. |
| `packages/lambdas/src/ehr/contribute-resource-diff-bundles.ts` | Introduces a new Lambda handler for processing SQS events and contributing resource diff bundles, with error/event capturing, environment config, and schema validation. |
| `packages/lambdas/src/ehr/compute-resource-diff-bundles.ts`<br>`packages/lambdas/src/ehr/refresh-ehr-bundles.ts` | Updates Lambda handlers to use new direct handler classes and removes Sentry serverless integration in favor of a new error/event capture utility. |
| `packages/core/src/external/fhir/shared/extensions/derived-from.ts`<br>`packages/core/src/external/fhir/shared/extensions/extension.ts` | Adds new FHIR extension helpers for predecessor and data source metadata on resources. |
| `packages/api/src/external/ehr/shared/utils/appointment.ts` | Reduces the `parallelPatients` constant from 200 to 100. |
| `packages/lambdas/src/shared/parse-body.ts`<br>`packages/lambdas/src/shared/sqs.ts` | Moves `parseBody` function from shared SQS module to a new shared parse-body module to centralize JSON parsing and schema validation with consistent error handling. Updates imports accordingly in Lambda handlers and other files. |
| `packages/lambdas/src/job/patient/run-job.ts`<br>`packages/lambdas/src/surescripts/receive-response.ts`<br>`packages/lambdas/src/surescripts/receive-verification.ts`<br>`packages/lambdas/src/surescripts/send-batch-request.ts`<br>`packages/lambdas/src/surescripts/send-patient-request.ts`<br>`packages/lambdas/src/surescripts/verify-request-in-history.ts` | Updates imports to use the new centralized `parseBody` function from the shared parse-body module instead of the shared SQS module. |
| `packages/lambdas/src/ehr/elation/link-patient.ts`<br>`packages/lambdas/src/ehr/healthie/link-patient.ts`<br>`packages/lambdas/src/ehr/sync-patient.ts` | Adds validation in Lambda handlers to ensure `WAIT_TIME_IN_MILLIS` environment variable parses to a valid number, throwing a `MetriportError` if invalid. |
| `packages/core/src/external/ehr/job/create-resource-diff-bundles/steps/compute/ehr-compute-resource-diff-bundles-cloud.ts`<br>`packages/core/src/external/ehr/job/create-resource-diff-bundles/steps/refresh/ehr-refresh-ehr-bundles-cloud.ts`<br>`packages/core/src/external/ehr/command/get-appointments/ehr-get-appointments-cloud.ts`<br>`packages/core/src/external/ehr/command/sync-patient/ehr-sync-patient-cloud.ts`<br>`packages/core/src/external/ehr/elation/command/link-patient/elation-link-patient-cloud.ts`<br>`packages/core/src/external/ehr/healthie/command/link-patient/healthie-link-patient-cloud.ts` | Simplifies constructors by removing optional region parameters and requiring clients (SQS or Lambda) with default instantiation using configured AWS region, eliminating conditional client creation inside constructors. |
## Sequence Diagram(s)
### Resource Diff Bundle Contribution Workflow
```mermaid
sequenceDiagram
participant Client
participant API
participant Lambda_Compute
participant SQS_Contribute
participant Lambda_Contribute
participant S3
participant EHR
participant DB
Client->>API: POST /internal/ehr/:ehrId/patient/:id/resource/diff/:jobId/contribute?resourceType=...
API->>Lambda_Compute: Trigger compute resource diff bundles
Lambda_Compute->>S3: Fetch RESOURCE_DIFF_EHR_ONLY bundle
Lambda_Compute->>EHR: Hydrate missing referenced resources
Lambda_Compute->>S3: Store hydrated resource diff data contribution bundle
Lambda_Compute->>SQS_Contribute: Send message with contribution payload
SQS_Contribute->>Lambda_Contribute: Triggered with payload
Lambda_Contribute->>S3: Fetch resource diff data contribution bundle
Lambda_Contribute->>API: Call contributeResourceDiffBundle endpoint
API->>DB: Store contributed resources and update job statusOld vs New Control Flow for Job Entry StatussequenceDiagram
participant API
participant Lambda_Compute
participant Lambda_Contribute
participant DB
rect rgb(240,240,240)
note left of API: Old Flow
API->>DB: Set job entry status directly
end
rect rgb(220,255,220)
note left of API: New Flow
API->>Lambda_Compute: Trigger compute/contribute
Lambda_Compute->>Lambda_Contribute: (via SQS) Contribute resource diff bundle
Lambda_Contribute->>DB: Update job entry status (success/failure)
end
Possibly related PRs
|
added 3 commits
May 23, 2025 13:30
9 tasks
added 10 commits
May 28, 2025 12:52
…port/metriport into eng-72-athena-data-contribution Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-72 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
…port/metriport into eng-72-athena-data-contribution Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-72 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-72 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-72 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-72 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-72 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-72 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-72 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
9 tasks
added 12 commits
May 29, 2025 06:15
…port/metriport into eng-72-athena-data-contribution Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-72 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
…port/metriport into eng-72-athena-data-contribution Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-199 Ref: #1040 Signed-off-by: Thomas Yope A3E2 s <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-199 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
…port/metriport into eng-72-athena-data-contribution Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
…port/metriport into eng-72-athena-data-contribution Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
…port/metriport into eng-72-athena-data-contribution Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-199 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-199 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-199 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-199 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
… eng-72-athena-data-contribution Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
leite08
reviewed
Jun 21, 2025
leite08
reviewed
Jun 21, 2025
| }, | ||
| { | ||
| numberOfParallelExecutions: parallelRequests, | ||
| delay: delayBetweenRequestBatches.asMilliseconds(), |
There was a problem hiding this comment.
Wouldn't be better to use min/max jitter? It adds some delay and spreads the load over time
There was a problem hiding this comment.
leite08
reviewed
Jun 21, 2025
leite08
reviewed
Jun 21, 2025
...create-resource-diff-bundles/steps/contribute/ehr-contribute-resource-diff-bundles-direct.ts
Show resolved
Hide resolved
leite08
reviewed
Jun 21, 2025
...create-resource-diff-bundles/steps/contribute/ehr-contribute-resource-diff-bundles-direct.ts
Outdated
Show resolved
Hide resolved
leite08
reviewed
Jun 21, 2025
...create-resource-diff-bundles/steps/contribute/ehr-contribute-resource-diff-bundles-direct.ts
Outdated
Show resolved
Hide resolved
leite08
reviewed
Jun 21, 2025
leite08
reviewed
Jun 21, 2025
leite08
reviewed
Jun 21, 2025
added 2 commits
June 22, 2025 05:26
Ref: ENG-72 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
… eng-72-athena-data-contribution Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
packages/shared/src/common/env-var.ts (1)
20-27: Specify radix parameter forparseIntto ensure consistent parsing.The implementation looks good and follows the established patterns in the file. However, consider specifying the radix parameter for
parseIntto ensure consistent decimal parsing regardless of input format.Apply this diff to improve parsing reliability:
- const int = parseInt(value); + const int = parseInt(value, 10);This ensures that values like "08" are always parsed as decimal 8 rather than potentially being interpreted as octal in some environments.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (7)
packages/lambdas/src/ehr/compute-resource-diff-bundles.ts(2 hunks)packages/lambdas/src/ehr/contribute-resource-diff-bundles.ts(1 hunks)packages/lambdas/src/ehr/elation/link-patient.ts(2 hunks)packages/lambdas/src/ehr/healthie/link-patient.ts(2 hunks)packages/lambdas/src/ehr/refresh-ehr-bundles.ts(2 hunks)packages/lambdas/src/ehr/sync-patient.ts(2 hunks)packages/shared/src/common/env-var.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (6)
- packages/lambdas/src/ehr/healthie/link-patient.ts
- packages/lambdas/src/ehr/elation/link-patient.ts
- packages/lambdas/src/ehr/sync-patient.ts
- packages/lambdas/src/ehr/refresh-ehr-bundles.ts
- packages/lambdas/src/ehr/compute-resource-diff-bundles.ts
- packages/lambdas/src/ehr/contribute-resource-diff-bundles.ts
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.ts`: - Use the Onion Pattern to organize a package's code in layers - Try to use immutable code and avoid sharing state across different functions, objects, and systems - Try...
**/*.ts: - Use the Onion Pattern to organize a package's code in layers
- Try to use immutable code and avoid sharing state across different functions, objects, and systems
- Try to build code that's idempotent whenever possible
- Prefer functional programming style functions: small, deterministic, 1 input, 1 output
- Minimize coupling / dependencies
- Avoid modifying objects received as parameter
- Only add comments to code to explain why something was done, not how it works
- Naming
- classes, enums:
PascalCase- constants, variables, functions:
camelCase- file names:
kebab-case- table and column names:
snake_case- Use meaningful names, so whoever is reading the code understands what it means
- Don’t use negative names, like
notEnabled, preferisDisabled- For numeric values, if the type doesn’t convey the unit, add the unit to the name
- Typescript
- Use types
- Prefer
constinstead oflet- Avoid
anyand casting fromanyto other types- Type predicates: only applicable to narrow down the type, not to force a complete type conversion
- Prefer deconstructing parameters for functions instead of multiple parameters that might be of
the same type- Don’t use
nullinside the app, only on code interacting with external interfaces/services,
like DB and HTTP; convert toundefinedbefore sending inwards into the code- Use
async/awaitinstead of.then()- Use the strict equality operator
===, don’t use abstract equality operator==- When calling a Promise-returning function asynchronously (i.e., not awaiting), use
.catch()to
handle errors (seeprocessAsyncErrorandemptyFunctiondepending on the case)- Date and Time
- Always use
buildDayjs()to createdayjsinstances- Prefer
dayjs.duration(...)to create duration consts and keep them asduration- Prefer Nullish Coalesce (??) than the OR operator (||) to provide a default value
- Avoid creating arrow functions
- Use truthy syntax instead of
in- i.e.,if (data.link)notif ('link' in data)- Error handling
- Pass the original error as the new one’s
causeso the stack trace is persisted- Error messages should have a static message - add dynamic data to MetriportError's
additionalInfoprop- Avoid sending multiple events to Sentry for a single error
- Global constants and variables
- Move literals to constants declared after imports when possible (avoid magic numbers)
- Avoid shared, global objects
- Avoid using
console.logandconsole.errorin packages other than utils, infra and shared,
and try to useout().loginstead- Avoid multi-line logs
- don't send objects as a second parameter to
console.log()orout().log()- don't create multi-line strings when using
JSON.stringify()- Use
eslintto enforce code style- Use
prettierto format code- max column length is 100 chars
- multi-line comments use
/** */- scripts: top-level comments go after the import
packages/shared/src/common/env-var.ts
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: Analyze (javascript)
added 2 commits
June 22, 2025 05:35
Ref: ENG-72 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
<
10000
a href="/metriport/metriport/pull/3891/commits/1cedab94edd213001468b6daa61c7515444f173d" class="Link--secondary">1cedab9
Ref: ENG-72 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
6 tasks
added 8 commits
June 23, 2025 15:28
Ref: ENG-72 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-72 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-72 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
… eng-72-athena-data-contribution Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-72 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Ref: ENG-72 Ref: #1040 Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
… eng-72-athena-data-contribution Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
… eng-72-athena-data-contribution Signed-off-by: Thomas Yopes <thomasyopes@Thomass-MBP.attlocal.net>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Ref: ENG-72
Issues:
Dependencies
Description
Testing
Release Plan
Summary by CodeRabbit
New Features
Improvements
Infrastructure
Bug Fixes
Refactor