RELEASE ENG-268 Enable ingestion on consolidate recreate #3917
Conversation
Ref eng-268 Signed-off-by: Rafael Leite <2132564+leite08@users.noreply.github.com>
…d-into-open-search_2 ENG-268 Enable ingestion on data pipeline
|
""" WalkthroughThe change activates previously commented-out code in the patient consolidated recreate command, enabling the ingestion of consolidated patient data into the search engine after bundle recreation. It also updates the ingestion queue to use FIFO with unique deduplication IDs, adjusts queue configuration to enable FIFO and remove batching window, and refines search query construction by removing redundant cleanup and making the lexical search field configurable. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant RecreateConsolidatedFunction
participant Ingestor
participant SearchEngine
User->>RecreateConsolidatedFunction: Trigger recreateConsolidated
RecreateConsolidatedFunction->>RecreateConsolidatedFunction: Recreate consolidated bundle
RecreateConsolidatedFunction->>Ingestor: makeIngestConsolidated()
RecreateConsolidatedFunction->>Ingestor: ingestConsolidatedIntoSearchEngine(cxId, patientId)
Ingestor->>SearchEngine: Ingest consolidated data
SearchEngine-->>Ingestor: Acknowledge ingestion
Ingestor-->>RecreateConsolidatedFunction: Complete ingestion
Possibly related PRs
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
npm error code ERR_SSL_WRONG_VERSION_NUMBER 📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (4)
✅ Files skipped from review due to trivial changes (1)
🧰 Additional context used📓 Path-based instructions (1)`**/*.ts`: - Use the Onion Pattern to organize a package's code in layers - Try to use immutable code and avoid sharing state across different functions, objects, and systems - Try...
⏰ Context from checks skipped due to timeout of 90000ms (1)
🔇 Additional comments (4)
✨ Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
packages/api/src/command/medical/patient/consolidated-recreate.ts (1)
59-65: Review the error message context and consider timing implications.The ingestion logic is correctly implemented with proper async error handling. However, there are a couple of considerations:
Error message context: The error message says "Post-DQ" but this function can be called with
isDq=false, making the message potentially misleading.Timing consideration: The ingestion is triggered immediately after consolidation recreation. Ensure that the consolidated data is fully available before ingestion begins.
Consider updating the error message to be more generic:
- .catch(processAsyncError("Post-DQ ingestConsolidatedIntoSearchEngine")); + .catch(processAsyncError("ingestConsolidatedIntoSearchEngine"));Also, verify that there are no timing issues between consolidation completion and ingestion initiation.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
packages/api/src/command/medical/patient/consolidated-recreate.ts(2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.ts`: - Use the Onion Pattern to organize a package's code in layers - Try to use immutable code and avoid sharing state across different functions, objects, and systems - Try...
**/*.ts: - Use the Onion Pattern to organize a package's code in layers
- Try to use immutable code and avoid sharing state across different functions, objects, and systems
- Try to build code that's idempotent whenever possible
- Prefer functional programming style functions: small, deterministic, 1 input, 1 output
- Minimize coupling / dependencies
- Avoid modifying objects received as parameter
- Only add comments to code to explain why something was done, not how it works
- Naming
- classes, enums:
PascalCase- constants, variables, functions:
camelCase- file names:
kebab-case- table and column names:
snake_case- Use meaningful names, so whoever is reading the code understands what it means
- Don’t use negative names, like
notEnabled, preferisDisabled- For numeric values, if the type doesn’t convey the unit, add the unit to the name
- Typescript
- Use types
- Prefer
constinstead oflet- Avoid
anyand casting fromanyto other types- Type predicates: only applicable to narrow down the type, not to force a complete type conversion
- Prefer deconstructing parameters for functions instead of multiple parameters that might be of
the same type- Don’t use
nullinside the app, only on code interacting with external interfaces/services,
like DB and HTTP; convert toundefinedbefore sending inwards into the code- Use
async/awaitinstead of.then()- Use the strict equality operator
===, don’t use abstract equality operator==- When calling a Promise-returning function asynchronously (i.e., not awaiting), use
.catch()to
handle errors (seeprocessAsyncErrorandemptyFunctiondepending on the case)- Date and Time
- Always use
buildDayjs()to createdayjsinstances- Prefer
dayjs.duration(...)to create duration consts and keep them asduration- Prefer Nullish Coalesce (??) than the OR operator (||) to provide a default value
- Avoid creating arrow functions
- Use truthy syntax instead of
in- i.e.,if (data.link)notif ('link' in data)- Error handling
- Pass the original error as the new one’s
causeso the stack trace is persisted- Error messages should have a static message - add dynamic data to MetriportError's
additionalInfoprop- Avoid sending multiple events to Sentry for a single error
- Global constants and variables
- Move literals to constants declared after imports when possible (avoid magic numbers)
- Avoid shared, global objects
- Avoid using
console.logandconsole.errorin packages other than utils, infra and shared,
and try to useout().loginstead- Avoid multi-line logs
- don't send objects as a second parameter to
console.log()orout().log()- don't create multi-line strings when using
JSON.stringify()- Use
eslintto enforce code style- Use
prettierto format code- max column length is 100 chars
- multi-line comments use
/** */- scripts: top-level comments go after the import
packages/api/src/command/medical/patient/consolidated-recreate.ts
🧠 Learnings (1)
packages/api/src/command/medical/patient/consolidated-recreate.ts (1)
Learnt from: leite08
PR: metriport/metriport#3814
File: packages/api/src/routes/internal/medical/patient-consolidated.ts:141-174
Timestamp: 2025-05-20T21:26:26.804Z
Learning: The functionality introduced in packages/api/src/routes/internal/medical/patient-consolidated.ts is planned to be refactored in downstream PR #3857, including improvements to error handling and validation.
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: check-pr / lint-build-test
- GitHub Check: Analyze (javascript)
🔇 Additional comments (1)
packages/api/src/command/medical/patient/consolidated-recreate.ts (1)
5-5: LGTM - Import correctly added for ingestion functionality.The import statement properly brings in the factory function needed for the ingestion logic below.
Ref eng-268 Signed-off-by: Rafael Leite <2132564+leite08@users.noreply.github.com>
Ref eng-268 Signed-off-by: Rafael Leite <2132564+leite08@users.noreply.github.com>
…d-into-open-search_3 ENG-268 Move ingestion queue to FIFO
Ref eng-268 Signed-off-by: Rafael Leite <2132564+leite08@users.noreply.github.com>
…d-into-open-search_4 ENG-268 Remove max batching window from ingestion queue
Issues:
Dependencies
Description
Testing
Release Plan
masterSummary by CodeRabbit
Summary by CodeRabbit