Documentdb push 20250619 #237
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Constraint ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Change streams does not support rebalancing at the moment Related work items: #4116549
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description ---- #### AI description (iteration 1) #### PR Classification Enhancement and bug fix to improve the stability and management of file-based cursors. #### PR Summary This pull request refactors and bolsters file-based cursor management by introducing shared memory support, improved cleanup, error handling, and configuration changes. It also adds new API endpoints and logging for better cursor lifecycle control. - **`cursor_store.c` and related files:** Added shared memory functions (e.g., `FileCursorShmemSize`, `InitializeFileCursorShmem`), cursor count regulation (with `IncrementCursorCount`/`DecrementCursorCount`), and cleanup routines to ensure stable cursor file operations. - **Test and expected output files:** Updated regression tests and expected outputs to cover scenarios like duplicate cursor creation, cursor deletion limits, and error reporting when the file cursor capacity is reached. - **API enhancements:** Introduced new UDFs (e.g., `command_delete_cursors`) and tenant client methods for killing and deleting cursors, improving session management. - **Configuration updates:** Renamed and added GUC parameters (e.g., `MaxAllowedCursorIntermediateFileSizeMB`, `MaxCursorFileCount`, and cursor expiry settings) to control file-based cursor behavior. - **Logging and metrics improvements:** Enhanced logging in `mdsd_logger.c` to report current cursor counts and related metrics for operational insight. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
…S Engine ### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) Yes, the API schema ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) 1.05 ### Description We are adding support for the Connection Status command, which exists in native Mongo, and allows users to see their authenticated user, authenticated roles, and user privileges. For more information, visit connectionStatus.md. This PR is the second of three PRs for this feature. The first one added extensible role privileges to our `users_info` command, allowing users to retrieve role privileges with `showPrivileges: true`. This second PR adds a callable `connection_status` command to our API, that utilizes the role privileges. The last PR will add a gateway change that calls this `connection_status` command. Related work items: #4287246
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description [OSS] fix 0.104-1 hotfix upgrade scripts ---- #### AI description (iteration 1) #### PR Classification Hotfix update to upgrade SQL upgrade scripts correcting version references. #### PR Summary This pull request updates SQL upgrade scripts to use the correct UDF and documentdb version numbers. - In `pgmongo/sql/pgmongo--1.103-0--1.103-1.sql`, the include directive was updated from version 0.105-0 to 0.104-1. - In `oss/pg_documentdb/sql/documentdb--0.104-0--0.104-1.sql`, the UDF include reference was changed to use version 0.104-1. - The file `oss/pg_documentdb/sql/documentdb--0.104-0--0.105-0.sql` was moved to `oss/pg_documentdb/sql/documentdb--0.104-1--0.105-0.sql` with no code changes. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Performance ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? Yes ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Currently, each round of the sproc that does create indexes does exactly 1 index build and bails. This means if a single client request queues 4 indexes and the job runs every 2 seconds, it'll take 8 seconds to build all 4 indexes. This is wasteful since we know all 4 indexes up front. Add a loop to the index build process so that multiple indexes can be built without waiting the 2 seconds. As part of this, there were a few places allocating in memory contexts that are unstable across executions. Fix this. Additionally there was a loop that could cause index builds to starve for ~20 mins if it picked the wrong index. Fix this loop too. Finally, Noticed that the version cache does not cache properly if the Major version is 0. Fix this (which can cause perf regressions in DocumentDB).
### Does this PR have any customer impact? yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) feature + refactoring ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) no ### Are you introducing any new config? If yes, do you have tests with and without them being set? no ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Finish the work on the query processing logic separation. This pull request is a design and architectural enhancement that finalizes query processing separation in the Rust gateway. **For more details see the author comment.** Related work items: #3666511
…efactor processing commands on DML and DDL ### Does this PR have any customer impact? no ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) parity ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) no ### Are you introducing any new config? If yes, do you have tests with and without them being set? no ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description 1. Implement unshardCollection command 2. Do the logical separation inside the `process.rs` Related work items: #4306635
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description When using mongodb-driver-sync 4.0.6, and performing a query, the namespace in the request looks like "<db>.$cmd" compared to other version where it is "admin.$cmd". To accommodate this version, we just check if the collection name equals $cmd which is similar to what we do in the other Gateway ---- #### AI description (iteration 1) #### PR Classification This PR implements an API update and bug fix to correctly handle OP_QUERY messages for Java driver 4.0.6. #### PR Summary The pull request refines the query parsing logic by extracting the collection name from the namespace instead of using the full collection path, ensuring that command queries are accurately identified and unsupported queries are logged. - In `oss/pg_documentdb_gw/src/protocol/reader.rs`, the code now extracts a tuple (_db, collection_name) from the namespace. - The condition to detect command messages is simplified to check if `collection_name` equals "$cmd". - An informational log is added to print the collection name when OP_QUERY is unsupported. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
visridha
approved these changes
Jun 19, 2025
safern
approved these changes
Jun 19, 2025
lichoil
approved these changes
Jun 19, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.