Releases: oceanbase/odc
Releases · oceanbase/odc
Release 4.3.4
New Features
- Quick start
- The team workspace is newly added with a workspace that contains Overview of Scheduled Jobs, Quick Start, Recently Accessed Databases, About Us, and Best Practices. It guides users to get started quickly and showcases the core services and advantages of ODC.
- The process guidance for empty objects is improved. When there is no data on a feature page, it guides users on how to continue operating, solving the problem of users not knowing how to use ODC upon their first login.
- The global search capability is improved. It supports searching in specified databases, projects, and data sources.
- The primary key capability is improved. The global search and database object page primary keys can locate specific database objects. SQL Console is newly added as an entry in the team workspace, facilitating users' quick access to the SQL console.
- Database list
- The database grouping capability is newly added. It supports grouping databases in resource trees, database lists in projects, database selection in SQL windows, and database selection when creating tickets, based on projects, data sources, clusters, tenants, database types, and environments. The grouping conditions can be persistently saved, facilitating users' definition of their own grouping preferences.
- The database search capability is improved. It supports searching for databases in resource trees, database lists in projects, database selection in SQL windows, and database selection when creating tickets, based on databases, data sources, clusters, and tenants, facilitating users' quick retrieval of target databases.
- The database remark capability is newly added. It supports managing remarks on the database list page in projects. Remark information can be displayed in resource trees, database lists in projects, SQL windows, and when creating tickets, based on mouse hovering over target databases, displaying database details including remarks to assist users in identifying databases with the same name.
- Configuration center
- The configuration center is upgraded. It is newly added with spatial configurations (personal space, team space) to constrain the configurations of modules within the spaces.
- The dependencies, application scenarios, and priorities of ODC configurations at different levels (spatial level, environment level, user level, feature level) are clearly defined to ensure logical consistency.
- Six configurations are newly added under spatial configurations:
- Maximum Entries per Query: The maximum number of result rows allowed in SQL query scenarios within the space.
- Default Entries per Query: The default number of result rows for SQL query scenarios within the space.
- Allow Schema Substitution for Import Tickets: The option to enable schema substitution when importing data.
- Automatically Generate Backup Rollback Plan for Database Changes: The default value for whether to automatically generate backup and rollback content for database changes.
- Job Description Prompt: The prompt text for guiding users on how to fill in the description when creating a ticket.
- Customize Data Source Access Key: Allows users to modify the data source key for more secure key encryption strategies.
- Configurations from the original [Settings] module are adjusted to user configurations. The [Row Limit for Queries] configuration is removed, replaced by the [Maximum Entries per Query] and [Default Entries per Query] configurations under spatial configurations. As this iteration only provides global search functionality, the [Enable Global Object Search] configuration is removed.
- Configuration item names of all modules are standardized. The original environment-level configuration item [Maximum Number of Rows in the Result Set] is adjusted to [Query Result Upper Limit in SQL Window].
- SQL development
- Support for the management of materialized views on the UI is newly added, supporting the creation, viewing, and management of materialized views on the UI.
- Support for batch downloading of scripts is newly added.
- Support for displaying object DDL statements upon mouse hovering over object names in the SQL window is newly added.
- Data archiving
- Support for multi-table associated archiving scenarios is newly added, allowing the definition of multi-table associated archiving conditions.
- Support for archiving tables with generated columns is newly added.
- A new data cleanup exception handling mechanism is newly added, providing users with three handling options: skip, update target database, and fail task. This solves the issue of poor fault tolerance, where exceptions during data cleanup directly terminate the operation.
- Partitioning plans
- A new partition creation method is newly added, which creates partitions with INT/CHAR partitioning fields in a sequential incremental manner, significantly reducing the cost of configuring partitioning strategies for INT/CHAR partitioning fields.
- Lock-free schema changes
- DDL changes in column deletion scenarios are supported.
- Collaborative development
- The number of affected rows by DML changes is displayed in database change tickets, facilitating users' assessment of change risks.
- Support for merging database and table permission application tickets is newly added. If the databases requested by the user for permission application are the same and hit the same approval flow, they are merged into one ticket.
- The approval flow hit rules for database and table permission application are upgraded. They are matched based on the risk levels identified by the risk identification rules in the security specifications.
- The authorization mechanism is improved. Users can manage the data sources and users they have created.
- Support for batch setting administrators for databases within projects is newly added.
- Task framework
- The task framework is upgraded. The new task framework:
- Abstracts the access methods into a unified resource module.
- Provides flexible resource control strategies with customizable resource implementations, balancing costs and startup speed.
- Optimizes the state machine module, changing from task-driven to single-node-driven, reducing database access pressure and lock competition.
- Optimizes the logic for several abnormal states, reducing the impact of concurrent access on task status.
- Persists task allocation information, allowing tracking of task scheduling times through the database, enabling effective alerts.
- Others
- The V4.2X series of OB databases is adapted to V4.2.5 BP3, and the V4.3.X series is adapted to V4.3.5 BP1.
- The data importing and exporting component is upgraded to V4.3.3 BP1.
- Logical database changes, partitioning plans, and shadow table tickets are supported to be re-initiated.
Bug Fixes
This release fixed 332 issues. For more details, refer to GitHub and Dima.
- Projects
- Tickets for requesting project permissions were not visible.
- Database admins could not participate in approvals.
- Error when selecting databases with the same name during table/view permission requests.
- SQL window
- Doris data source did not support duplicate keys.
- Viewing execution profiles failed on OceanBase V4.2.5.3.
- Query plan errors due to type mismatch:
Longcannot be cast toInteger. - Error when auto-suggesting view names in cross-database queries: “view does not exist.”
- Database change
- Unable to estimate affected rows for DML statements in OceanBase Oracle compatible mode.
- Partitioning plans
- Partitioning plan tasks failed when the target table did not exist.
- Data archiving and data cleanup
- Periodic tasks could not be deleted after the associated database was deleted.
- When archiving LOB data on OceanBase V4.3.5.1 with
supportLobLocator=false, source data was not deleted. - Archiving failed for TIMESTAMP value
'0000-00-00 00:00:00'. - Retry actions in archiving/cleanup tasks resulted in inconsistent subtask and execution statuses.
- Login integration
- OAuth2 login failed to load user roles.
- Result sets
- Unable to modify the default "Query Result Row Limit."
- Notifications
- Notification messages were sometimes sent multiple times.
- Missing post-cleanup success configuration option for data archiving tasks.
- Notifications were not sent when multi-database changes failed.
- Database objects
- Extra semicolon appeared when viewing functions or procedures.
- Import/export
- Failed to import stored procedures containing escape characters like
'\\'or'\'. - Improved CSV line ending handling across different data sources.
- Failed to import stored procedures containing escape characters like
- Online schema changes
kill sessionoperation may fail during online schema changes.
Release 4.3.3_bp2
New Features
- Archiving to object storage now supports post-archive cleanup and cleaning up temporary tables after archiving.
- Archiving to object storage now supports resuming from breakpoint after task timeout.
- Archiving to object storage now supports custom target filename configuration.
- Added compatibility for OceanBase Database V4.3.5.1.
Bug Fixes
- Fixed an issue where date format wasn't updated after executing SET nls_date_format='DD-MON-RR' in SQL window.
- Fixed missing operation records in the task details of SQL plans in the personal workspace.
- Fixed background task processes not being released after canceling data cleanup and archiving tasks.
- Fixed task descriptions of SQL plans not syncing with database change subtask descriptions.
- Fixed incorrect documentation links for partition creation configuration in partitioning plan tasks.
- Fixed throttling not taking effect when modifying settings after source table deletion during data cleanup.
- Fixed execution errors when re-initiating data mocking tasks.
- Fixed incorrect task status display after terminating data archiving tasks.
- Fixed missing project join entry when user isn't a member of any project.
- Fixed missing pending approval notifications for database/project/table permission requests.
Release 4.3.3_bp1
Bug fixes
- Fixed the error when viewing data archiving and data cleaning task details in personal space.
- Fixed the null pointer exception when executing SQL after upgrading to 4.3.3 with a custom environment.
- Fixed the null pointer exception when validating "limit the length of CHAR type fields" for tables with generated columns.
- Fixed the error when re-importing data exported from MySQL data source.
- Fixed the connection failure of obsharding data source.
- Fixed the issue where the risk level rule still triggers after the rule is deleted and a task is initiated.
- Fixed the issue where clicking next after triggering the approval rule by deleting a column cannot open the database change window.
- Fixed the occasional execution failure of SQL plans and the inability to initiate again.
Release 4.3.3
Version Changes
- Data Lifecycle Management
- Added archiving paths from Oracle to Object Storage.
- Added archiving paths from MySQL to Object Storage.
- Added archiving paths from OceanBase MySQL to Object Storage.
- Added archiving paths from OceanBase Oracle to Object Storage.
- Added archiving paths from PostgreSQL to Object Storage.
- Added support for editing history review with content comparison.
- Introduced dynamic target table definition to support storing historical data by day, month, or other time units.
- Added ability to delete archiving and cleanup tasks when completed or terminated.
- Optimized rollback logic to only revert data archived in the current task.
- Online Schema Change
- Added retry mechanism with enhanced retry logic for various failure scenarios.
- Added status display for online schema changes, allowing progress tracking of running tasks.
- Change Risk Management
- Introduced global project roles including Global Project Admin, Global Security Admin, and Global DBA.
- Added project archiving validation to check for unfinished tickets and periodic tasks before archiving.
- Added support for project deletion for archived projects.
- Implemented fine-grained view permission control with user-initiated permission requests.
- Expanded executable SQL types in SQL window to include
call,comment,set session, and more. - Extended SQL check support for native Oracle data sources.
- Added change approval workflow for native Oracle data sources.
- Added 2 new SQL check rules for standardizing
create likeandcreate astable creation statements.
- SQL Development
- Added GUI support for OceanBase external tables.
- Added support for displaying secondary partitions in OceanBase partitioned tables.
- Added support for editing stored procedures in OceanBase MySQL mode.
- Added support for PL debugging via OBProxy.
- Other Enhancements
- Added SAML single sign-on support.
- Added session kill capability for native Oracle data sources.
- Added compatibility for OceanBase V4.2.5 and V4.3.3.
- Added support for OBKV SQL mode.
- Enabled secure cookie mechanism for enhanced data transmission security.
- Added column width adjustment support for forms (ticket lists, database lists).
- Usability Improvements
- Added persistent project search criteria to reduce frequent searches.
- Maintained last used project context across user sessions.
- Standardized risk identification rule conditions using operators and English expressions to avoid ambiguity.
- Enhanced connection keep-alive logic with 3-minute database request intervals.
- Added project column in non-project ticket module for quick project identification.
- Extended "Create Again" functionality to all ticket types except logical database changes, partition plans, and shadow tables.
- Refined ticket management scope: admins and DBAs can manage all project tickets, while other roles can only manage self-initiated tickets. Also, all members can view tickets in their projects.
Bug Fixes
- Data Sources
- Fixed synchronization of system databases like
information_schemato projects in bastion host integration scenarios. - Resolved database synchronization suspension issues.
- Fixed synchronization of system databases like
- Tickets
- Fixed approval workflow triggering in personal workspace for data archiving tickets.
- Fixed status inconsistency in data archiving/cleanup task execution records.
- Fixed structure comparison task execution issues for non-current account users.
- Fixed Oracle table structure export failures with virtual columns.
- Fixed structure comparison failures for OceanBase MySQL tables with full-text index tokenizers.
- Fixed operation record viewing failures for periodic tasks with numerous subtasks.
- Fixed non-working configuration retention in export tasks.
- Change Management
- Fixed unauthorized view exports.
- Fixed partition plan disable issues affecting project archiving.
- SQL Development
- Fixed NPE exceptions in specific SQL check scenarios.
- Fixed PL drop requiring database change permissions.
- Fixed display issues for functions with year return type.
- Fixed PL creation and drop failures with @ in names.
- Fixed table detail viewing failures for Oracle tables with extended statistics (
DBMS_STATS.CREATE_EXTENDED_STATS). - Fixed ineffective row limit for Insert statements.
- Fixed null pointer exceptions when exporting array function result sets.
- Fixed missing run button for package subprocedures in Chrome 118.
- Fixed error when viewing subprocedures in package headers.
- Other
- Fixed last used project not opening on subsequent ODC access.
Release 4.3.2_bp3
Bug fixes
- Fixed an overflow issue with
tinyintduring native MySQL to native MySQL archiving. - Fixed a null pointer error in archiving caused by the current value of a numeric column being null when the previous column was empty.
- Fixed an issue where archiving failed due to null values in
timestampfields. - Fixed a null pointer error that occurred when executing SQL after upgrading from a lower version with custom environments to ODC 4.3.2-BP1.
Release 4.3.2_bp2
Bug fixes
- Fixed a mapping issue with time-type fields in MySQL to OBMySQL archiving tasks that caused task failures.
- Fixed a timeout issue in Oracle mode archiving for large tables caused by a short
HINTtimeout setting. - Fixed an issue where the condition matching strategy did not take effect in MySQL mode archiving.
- Fixed a slow query issue when setting composite vector filter conditions in OBMySQL mode archiving.
- Fixed an issue where the database change subtask status did not match expectations when "Ignore Errors and Continue" was selected in the partitioning plan.
- Fixed data loss caused by the
lock usercommand not taking effect during online schema changes.
Release 4.3.2_bp1
Bug fixes
- Fixed an issue where environment labels in projects were not updated after modifying them in data sources.
- Fixed an error when opening the SQL console after adding custom environment labels to a data source.
- Fixed a PL debugging error that occurred when there were both deleted and non-deleted databases with the same name under the same data source.
- Fixed an issue where the specified condition matching strategy in data archiving did not take effect.
- Fixed a timeout error during archiving in OBOracle mode.
- Fixed an issue where archiving failed due to inconsistent field order between source and target in OBMySQL mode.
- Fixed the overly narrow right-click menu in the object tree.
Release 4.3.2
This release focuses on enhancing three key areas:
- Logical Database Support: Introduced a new logical database feature, providing a fast, reliable, and user-friendly foundation for managing database and table partitions. This update includes logical database configuration, access control management, and the ability to extract logical tables, create new ones, and perform DDL changes.
- Expanded Data Archiving: Two new archiving paths have been added. Building on the previous version’s support for MySQL to OBMySQL archiving, this update introduces support for Oracle to OBOracle and PostgreSQL to OBMySQL archiving.
- SQL Development and Data Source Management Enhancements: Usability improvements have been made to the SQL development page, data sources, and forms, with over ten features optimized, including the navigation keys, resource tree objects, and SQL execution results.
For more details on this update, please refer to the version change list.
Feature Updates
- Logical Database
- Added support for configuring and removing logical databases.
- Automatically extract logical tables based on the topology distribution of physical tables to improve efficiency.
- Provided a new feature for creating logical tables, where users can follow prompts to configure and create logical tables.
- Added support for submitting logical database DDL change requests through the database change order feature.
- Added structural consistency checks for logical tables to detect physical table inconsistencies.
- Logical databases are now included in the existing database and table permission management system, ensuring secure access to logical database data.
- Data Archiving
- Added new support for archiving from Oracle to OceanBase Oracle.
- Added new support for archiving from Oracle to Oracle.
- Added new support for archiving from OceanBase Oracle to Oracle.
- Added new support for archiving from PostgreSQL to OceanBase MySQL.
- Data Cleanup
- Added support for data cleanup for Oracle databases.
- Added support for data cleanup for PostgreSQL databases.
- SQL Checks
- Added a new SQL row impact check specification for OceanBase MySQL and MySQL data sources, enabling the limitation of large SQL changes to ensure business stability.
- Partitioning Plan
- For date-type field partitions, added an option to ensure partition names and partition content ranges are consistent.
- Data Import
- The ODC desktop version now supports direct folder selection for data imports.
- SQL Window
- Added a feature where clicking the product logo on the SQL development page returns users to the project list page, providing a quick way to return to the homepage.
- Improved the interaction of the SQL window positioning key, ensuring that the targeted database is automatically displayed in the visible area.
- SQL executed in the SQL window now shows DB execution time directly on the result tab.
- Optimized sorting of resource tree objects; resource tree items (data sources, projects, databases, tables, views, etc.) are now sorted by name by default.
- Users can now initiate all types of tickets directly from databases in the SQL window resource tree.
- Data Source
- Databases under a data source can now be filtered quickly based on availability and whether they are assigned to a project.
- Improved interaction for data source connection initialization scripts, providing quick guides for common parameter configurations.
- Usability
- Optimized empty data state interactions across all features (data sources, projects, scripts, resource tree, etc.).
- Optimized data source selection interactions across all features (database maintenance, sensitive column scanning, adding new databases to projects).
- For commonly used forms (ticket lists, database lists, user lists), columns that do not display fully (ticket description, data source column, sorting rules, user roles, etc.) now display full content on hover.
- Added a GitHub issue link on the feedback page, allowing users to provide product feedback quickly via the open-source community.
- Others
- SSO integration now supports Azure AD.
- Updated the platform's login password policy. The latest policy requires a password length of 8-32 characters, containing at least three of the following four character types: numbers (0-9), uppercase letters (A-Z), lowercase letters (a-z), and special characters (all English special characters).
Bug Fixes
- Data Objects
- Fixed an error where SQL generated from formatted columns with large data types failed upon execution.
- Incorrect precision for
tinyinttype in MySQL mode has been fixed. - Resolved an issue where NULL values in CLOB or BLOB columns could not be edited via the GUI.
- Change Management
- Fixed incorrect precision for
tinyintcolumn types (MySQL data source) in SQL generated by schema comparison tasks. - Resolved an issue where partition names were misidentified as table names, leading to permission errors.
- Fixed a bug where
new_tableinrename table old_table to new_tablewas mistakenly flagged as needingnew_tablepermissions. - Addressed potential failures in data simulation tasks for MySQL 5.7.
- Resolved an issue where the manual execution button remained clickable after triggering manual execution.
- Fixed failure in creating or updating rules when there were too many risk identification rules.
- Fixed an issue where the page could not scroll when there were too many risk identification rules.
- Resolved an issue allowing scheduling intervals of less than ten minutes for periodic tasks.
- Fixed incorrect precision for
- Import and Export
- Fixed an issue where merging PL and table DDL into a single SQL file could cause import failures due to syntax errors.
- Resolved a bug where path issues could cause export failures on the Windows desktop version.
- Fixed an issue in OceanBase Oracle mode where exported result sets would be empty if table names were lowercase.
- Resolved export failures for native MySQL data on the Windows desktop version.
- Fixed errors when exporting tables with BigDecimal types.
- Addressed an issue where the result set export would fail when using
*andnvlfunctions in SQL. - Fixed export failures for result sets containing SDO-GEOMETRY data types.
- Data Archiving
- Fixed errors that occurred when modifying data archiving rate limiting parameters.
- Corrected inaccurate scan record counts in the details of executed archiving tasks.
- Fixed a bug in OceanBase Oracle mode where archiving operations would fail if users did not have DBA permissions.
- Data Source
- Fixed an issue where testing an OceanBase SYS tenant data source incorrectly succeeded when OceanBase Oracle type was selected.
- Others
- Resolved potential SSO login failures in environments with load balancing services.
- Fixed the issue where the startup parameter
ODC_APP_EXTRA_ARGSwas ineffective.
v4.3.1_bp1
Bug fixes
- Disabled the feature for synchronizing table structures with versions earlier than MySQL 5.7.
- Fixed an issue where the desktop version of ODC V4.3.1 could not be installed on Windows Home editions.
- Fixed a problem where alert notifications failed when the content type was set to
text/plain. - Fixed an issue where function calls in SQL were incorrectly intercepted by permission checks.
- Fixed a problem where having export permissions for only one table allowed exporting the entire database.
- Corrected the display issue of foreign key constraints'
ON DELETErules. - Fixed an issue where regular users received "no permission" errors when viewing task details after tasks were assigned a risk level.
- Fixed an issue where the "Specify Partition" option was not selected when re-initiating data archiving tasks based on the specified partition.
- Fixed a problem where project owners could not see the "Edit" and "Terminate" buttons when viewing task details.
- Fixed a potential data loss issue with online schema changes.
- Corrected an issue where the status icon for selectable databases remained in a loading state when requesting database permissions.
- Fixed a problem where databases with invalid statuses were not grayed out in the list of selectable databases for table permission application tasks.
- Fixed an issue where the context menu opened via right-click in the structure tree was too narrow, causing some options to be cut off.
Release 4.3.1
4.3.1 (2024-07-31)
Feature Updates
Risk Control Changes
- Added table-level permission control, allowing project members to have different operation permissions on different tables, including query, change, and export operations, enhancing collaborative control capabilities.
Session Management
- Support for closing sessions/queries has been extended to more scenarios where OBServer is not directly connected.
- When connected to OceanBase V4.2.3 and OBProxy V4.2.5 or higher versions, session management is performed using client session capability.
- In OceanBase V4.2.1 Oracle mode, session management is handled using anonymous blocks.
Analytical Instance Development
- Added real-time execution profiling, providing visual and interactive presentation of
sql_plan_monitor.- This feature requires data source version OceanBase V4.2.4 or higher.
- Supports analysis not only of completed SQL executions but also real-time analysis of ongoing SQL executions.
- Provides graphical, tabular, and text views of execution plans, intuitively displaying operator connections and step sequences.
- Global view offers sorting of Top 5 time-consuming operators and overall summary of execution phases, quickly pinpointing performance bottlenecks.
- Operator nodes include execution status and detailed information such as CPU, memory, disk, rows processed, and node attributes.
- For parallel execution nodes, supports sorting by DB time, IO memory, and rows processed, quickly identifying data skew. Supports analysis of both standalone and distributed execution plans.
- Newly designed integrated real-time diagnostic page for comprehensive execution profiling combining execution plans and end-to-end trace diagnostics.
SQL Development
- During SQL execution, supports viewing execution progress, including total number of executions, current execution count, and trace ID of currently executing SQL. Real-time viewing of completed execution results is also supported.
- Supports graphical format viewing of OceanBase's logical SQL execution plan.
Data Sources
- Fully compatible with OceanBase V4.2.4, OceanBase V4.3.1, OceanBase V4.3.2.
Usability Improvements
- Data cleanup tasks now support editing task configurations.
- Data source module supports batch import from MySQL, Oracle, and Doris data sources.
Bug Fixes
Data Lifecycle Management
- Table structure comparison performed even when structure synchronization is not enabled. #3014
Risk Control Changes
- Automatic authorization rules did not take effect for LoginSuccess events. #3003
Import/Export
- In the desktop mode, reinstalling ODC may lead to the unintended import of historical files generated during previous import tasks when initiating new tasks. #3006
SQL Check
- When SQL window rules are enabled, commit and rollback buttons in SQL window may become ineffective. #2985
SQL Development
- NPE may occur during PL debugging. #2930
- SQL error when modifying session variables for Oracle data sources. #2872
Mock Data
- Unable to terminate mock data tasks. #2850
Global Object Retrieval
- Object synchronization cannot be stopped. #2928
Tickets
- Unable to retrieve SQL check results when check result file does not exist locally. #2943
Auditing
- Actual length of
content_ip_addresscolumn values exceeds the column length limit. #2863
Other
- When deploying across multiple nodes using process mode to schedule tasks, all tasks may be scheduled to the same node. #2408