8000 Releases · dolthub/dolt · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Releases: dolthub/dolt

1.54.2

04 Jun 02:43
@github-actions github-actions
v1.54.2
8eadb58
Compare
Choose a tag to compare
1.54.2 Latest
Latest

Merged PRs

dolt

  • 9306: go: store/datas/pull: clone.go: Improve robust of Clone for certain remoteapi implementations when the remote Conjoins.
    The clone code works by listing the remote table files and downloading them into the local table file store. When the remote is a remoteapi implementation, like a DoltHub repository, this resulting in listing the remote table files and using URLs to fetch each of them.
    The URLs returned from these APIs can expire and they need to be refreshed. This refresh can happen in two ways:
    1. There is explicit support in the TableFileSource representation returned by the API to include a mechanism to refresh it. DoltHub uses this, and the Dolt client will make use of that support to refresh expired URLs.
    2. The heavy handed approach is to list the table files again and use the newly returned URLs.
      The Clone code has explicit support for doing (2), and it is necessary for remoteapi implementations with expiring URLs but without explicit RefreshTableFileUrl support. dolt itself, when running a remote as part of sql-server for example, does not implement RefreshTableFileUrl support, and so the re-list support is still necessary.
      This PR changes the Clone implementation so that, on a retry, it makes all the newly returned table file sources available for the next try, but it keeps the old sources around if they no longer come back from ListTableFiles. In this way, we get strictly more robust behavior than before.
      The downside is that, when the remote file is actually gone, the Clone code will continue attempting to download it until it reaches a terminal download failure. This change in behavior is not as disruptive as the current behavior, and so we make this new trade off for now.
  • 9299: Improved chunk tracking during clone
    Decrement the chunk counter when a file completes. Add a completion status message. Fix a potential race that @reltuk identified in review.
  • 9298: Last easy unskip according to Claude
  • 9296: Check for invalid ancestor merge spec
    fixes #9294
  • 9295: Various unskipped bats tests identified and fixed by Claude Code.
  • 9289: Fix load data tabs test
  • 9288: The where clause on a dolt diff command is always validated
    Even if there is an empty diff
  • 9287: Fix skip test for another load data issue
  • 9122: implement dolt_branch_status()
    This PR implements dolt_branch_status(<base_branch>, <feature_branch1>, <feature_branch2>, ...).
    The function compares base_branch against each of the feature branches and outputs a table that lists how many commits ahead and behind base branch.
    Additionally, this PR contains some cleanup/refactoring.
    addresses: #9090

go-mysql-server

  • 3010: Add query time to logger
    This is a proposed fix for Dolt issue: #8909
    I'm not certain if we want to just add this field or replace connect time. I think this implementation is safer.
  • 3009: New skipped tests for group_concat
    The ORDER BY clauses in aggregates like group_concat do not work correctly with subqueries. Lots of varied tests demonstrating this defect.
  • 3004: Add support for IS NULL expressions with records
    The implementation in GMS for IS NULL checks if the value is NULL or not. For record and composite types in Postgres, IS NULL needs to check if each value in the record or composite type is NULL to determine if the value is NULL. See dolthub/doltgresql#1520 for more details.
    This change creates a new shared type, named TupleValue, in GMS that is used for record values (and eventually, composite type values). I also considered giving Doltgres it's own, custom IsNull implementation, but the analyzer does specific checks and optimizations when it looks for *expression.IsNull instances, which would stop working correctly.
  • 3002: fix table functions in stored procedures
  • 3001: Support for injected expressions with ORDER BY clauses
    Depends on 416

vitess

  • 416: New flavor of injected expression
  • 415: Fix implementation for REVOKE ALL...
    This PR fixes REVOKE ALL [PRIVILEGES], GRANT OPTION ... so that it parse to the same thing as REVOKE ALL PRIVILEGES as they are equivalent in MySQL. The GRANT privilege is already part of all privileges, specifying it is just for clarity.
    Additionally, this PR adds syntax support for the IF EXISTS option and IGNORE UNKNOWN USER option that is part of REVOKE statements.
    MySQL Docs: https://dev.mysql.com/doc/refman/8.4/en/revoke.html
    Syntax for: #9228

Closed Issues

  • 8909: Dolt start time of queries in logs is all pinned to the connection open start time
  • 7984: Implement dolt_help set of system tables
  • 8345: dolt does not reload all statistics on server restart
  • 9240: Clone Status reports more chunks being downloaded than it says exist
  • 9294: HEAD^0 causes panic

Contributors

reltuk
Assets 10
Loading

1.54.1

30 May 23:48
@github-actions github-actions
v1.54.1
85acc26
Compare
Choose a tag to compare
1.54.1

Merged PRs

dolt

  • 9286: Fix skip test for dolt diff -r sql and views
  • 9285: Fixed a load data test for mismatched columns
  • 9278: Two additional unskipped bats tests: one that said flakey, one in git status
    One says it was skipped because it was flakey in CI. Let's see if it's now fine.
  • 9276: Unskip keyless add/drop column test
    This is now implemented
  • 9275: Fix merge test to properly test for a constraint violation
    Identified and fixed by Claude
  • 9274: Skipped bats test for dolt status that now works.
    Identified by Claude.
  • 9273: Fix input validation of primary keys
  • 9272: Added a new constant for string columns created during schema import.
    Length is 200 instead of 1023.
    Fixes: #8972
  • 9269: manifest safety checks
    Recent user report of a manifest landing that was missing it's format. This will at least prevent the generation of such a file in the future.
  • 9267: Remove the --new-format flag from dolt init
    This flag really makes no sense any more
    This is a claude change. $0.80, two prompts.
  • 9264: Unskip test for export quotes
    Companion GMS PR: dolthub/go-mysql-server#2995
  • 9256: remote cleanup
    When removing remote definitions, we now also remove the remote references from the branch specifications.
    Fixes: #9169

go-mysql-server

  • 3002: fix table functions in stored procedures
  • 3000: make update join case insensitive
    fixes: #9268
  • 2997: Properly escape special characters in comments
    Mostly fixes #8509 (does not handle Control+Z)
  • 2996: Revert "Merge pull request #2894 from dolthub/max/nil-proc-db"
    This reverts commit ce4535ec2da6600a552279a5abf46eb652b7eec0, reversing changes made to f38274720545e418e94e16b664302d96098d515e.
  • 2995: Fixed enum escaping issue
    This fixes a skipped bats tests in table export.
    Fixes Dolt issue: #2197
  • 2993: Added undeclared variable error message
    Fixes #8539
  • 2894: [planbuilder] nil database doesn't panic building procedure call

Closed Issues

  • 9268: Update with subselect in WHERE clause silently fails to update
  • 8509: SHOW CREATE TABLE doesn't properly escape quotes in comments
  • 8972: varchar types inferred on schema import are set to size 1023. 18 occurrences overflow row size limit
  • 2197: SQL export for set types with embedded single quotes broken
  • 8697: Support MariaDB to Dolt replication
  • 9169: Dolt maintains reference to removed remote
Loading

1.54.0

28 May 20:31
@github-actions github-actions
v1.54.0
33cf010
Compare
Choose a tag to compare
1.54.0

This release adds the commit_order column to the dolt_log system table and dolt_log() table function. This is a backwards incompatible change because select *-style queries against that table or table function will return rows of a different size and require the user to update their parsing code. Per Dolt's versioning policy, this requires a minor version bump.

Merged PRs

dolt

  • 9262: Fix replace count test
    Claude insists that this behavior matches MySQL. It consulted the docs and ran tests against MySQL.
    I can confirm the tested behavior matches MariaDB which I have installed: 
    MariaDB [(none)]> CREATE DATABASE test_replace;
ERROR 1007 (HY000): Can't create database 'test_replace'; database exists
MariaDB [(none)]> DROP DATABASE IF EXISTS
->       test_replace;
Query OK, 1 row affected (0.057 sec)
MariaDB [(none)]> USE test_replace;
ERROR 1049 (42000): Unknown database 'test_replace'
MariaDB [(none)]> CREATE DATABASE test_replace;
Query OK, 1 row affected (0.003 sec)
MariaDB [(none)]> USE test_replace;
Database changed
MariaDB [test_replace]> CREATE TABLE test(pk BIGINT
->       PRIMARY KEY, v BIGINT);
Query OK, 0 rows affected (0.030 sec)
MariaDB [test_replace]> REPLACE INTO test
->       VALUES (1, 1);
Query OK, 1 row affected (0.001 sec)
MariaDB [test_replace]> REPLACE INTO test VALUES (1,
->       2);
Query OK, 2 rows affected (0.001 sec)
  • 9261: Unskip no test symbols in binary test
  • 9260: Ignore CLAUDE.md for now
  • 9254: Add support for MariaDB's GTID format
    Fixes: #9241
    Fixes: #8697
    Testing for this is currently manual. To add automated tests for MariaDB integration, we'll need some larger changes to the binlog replication testing framework to support launching a MariaDB server. We should do that work eventually, especially if we hear more customer requests for MariaDB replication integration.
  • 9251: Allow dumping wrapper values to SQL
    This prevents an issue where commands like dolt dump would fail when trying to export a table as SQL statements if the table has a BLOB column, since it would fail to unwrap the text wrapper value to access the underlying bytes.
  • 9250: Implement commit_order in dolt_log system table and table function
    This PR implements: #9145. This is implemented using Claude Code.
    A customer asked for the commit order in the dolt_log system table. We have that information as we generate the commit log in the height variable. This exposes that variable as a column populated for each commit. 
    $ dolt sql -q "select * from dolt_log limit 3"
+----------------------------------+--------------------+-----------------------------------+---------------------+--------------------------------------+--------------+
| commit_hash                      | committer          | email                             | date                | message                              | commit_order |
+----------------------------------+--------------------+-----------------------------------+---------------------+--------------------------------------+--------------+
| iifq0hvp5fis9ld2lnhorqr5l697ph38 | post-no-preference | post.no.preference@protonmail.com | 2025-05-23 06:34:26 | volatility_history 2025-05-22 update | 1427         |
| 62n14k63lu5jcc3a3qr18q8ku19e8g8q | post-no-preference | post.no.preference@protonmail.com | 2025-05-23 06:34:15 | option_chain 2025-05-22 update       | 1426         |
| 2paehofana33lbbbjnqa18v3nlsvbgbk | post-no-preference | post.no.preference@protonmail.com | 2025-05-22 06:34:38 | volatility_history 2025-05-21 update | 1425         |
+----------------------------------+--------------------+-----------------------------------+---------------------+--------------------------------------+--------------+
  • 9247: Made dolt diff --stat -r sql an error
    Fixes #7817
    From Claude: 
    ⏺ Perfect! The changes are working correctly. Let me summarize what I've
implemented:
Summary
I successfully implemented the fix for GitHub issue #7817. The issue was
that dolt diff --stat -r sql was throwing a generic "unsupported" error,
but it should return a more specific error message explaining that SQL
format diffs are only for schema or data changes.
Changes Made:
1. Updated the error message in
/Users/timsehn/dolthub/git/dolt/go/cmd/dolt/commands/diff_output.go:
- Changed from: "diff stats are not supported for sql output"
- Changed to: "SQL format diffs only rendered for schema or data
changes"
2. Updated the test in
/Users/timsehn/dolthub/git/dolt/integration-tests/bats/sql-diff.bats:
- Updated the expected error message in the test case to match the new
error message
Reasoning:
The --stat flag shows statistics about diffs (like number of rows
changed), which cannot be meaningfully represented as SQL statements. SQL
format should only be used for actual schema or data changes that can be
expressed as executable SQL commands. The new error message clearly
communicates this limitation to users.
The implementation correctly returns an error when someone tries to use
dolt diff --stat -r sql, and the tests confirm that:
1. The specific error case now returns the improved error message
2. Normal SQL diff functionality continues to work as expected
> /cost
⎿ Total cost:            $3.08
Total duration (API):  11m 9.9s
Total duration (wall): 25m 15.4s
Total code changes:    2 lines added, 3 lines removed
Token usage by model:
claude-3-5-haiku:  79.9k input, 1.9k output, 0 cache read, 0 cache
write
claude-sonnet:  23.2k input, 15.3k output, 4.1m cache read,
396.2k cache write
  • 9246: --all option for dolt log
    Issue #8200
    Adds a --all option for dolt log. You can also specify tables after the all option, with dolt lot --all table1 table2.
    I still want to work on:
    • The code's a bit messy still, I think a helper function might be nice, especially to set the existingTables map.
    • Maybe make it work with --not to exclude branches?
    • I'm sort of wondering if the code handling the generation of the "query" we make could be improved. It might not be worth it, but for example if you have a table and branch with the same name, we reinsert that value into the params slice. We also reuse a bunch of code for the code handling the first non-revision argument.
  • 9244: update maven for ORM tests
  • 9237: Sql shell warnings
    Fix issue #8875
    This pr adds support for warnings in the sql shell. There is now both a summary and detailed list of warnings.
    Running select 1/0; will produce, after the table: 
    Warning (Code 1365): Division by 0```
You can disable/enable the ending list with \w and \W, respectively.
  • 9235: Log errors on a binlog connection
    When a SQL error is sent from the replication source to the replica, the connection is closed out, but the error wasn't getting logged. This made it difficult to debug why the replication connection was erroring out.

go-mysql-server

  • 2993: Added undeclared variable error message
    Fixes #8539
  • 2992: Hacky extension point for aggregate function determination
    Also refactors building aggregates
  • 2989: fix: Fix create new server in example.
    I found that the parameters did not match when creating the server in the example, so I fixed it.
  • 2988: Implement REVOKE ALL PRIVILEGES, GRANT OPTION ...
    This PR adds tests for REVOKE [IF EXISTS] ALL [PRIVILEGES], GRANT OPTION ... and implements the IGNORE UNKNOWN USER option.
    There are changes to REVOKE PROXY, but it seems like that is unsupported so there are no tests.
    Companion pr: dolthub/vitess#415
  • 2986: add returning clause case for assign exec index rule
    Test for it is added in doltgresql PR
  • 2985: fix default value for non null enum columns
    In MySQL non null enum columns default to the very first enum instead of throwing a missing default error.
    fixes: #8617
  • 2984: Add lock_warnings system variable.
    Adds a session variable for locking warnings, allowing you to run background queries without clearing the warnings from the last user-generated command.

vitess

  • 415: Fix implementation for REVOKE ALL...
    This PR fixes REVOKE ALL [PRIVILEGES], GRANT OPTION ... so that it parse to the same thing as REVOKE ALL PRIVILEGES as they are equivalent in MySQL. The GRANT privilege is already part of all privileges, specifying it is just for clarity.
    Additionally, this PR adds syntax support for the IF EXISTS option and IGNORE UNKNOWN USER option that is part of REVOKE statements.
    MySQL Docs: https://dev.mysql.com/doc/refman/8.4/en/...
Read more
Loading

1.53.6

21 May 00:12
@github-actions github-actions
v1.53.6
6c8288c
Compare
Choose a tag to compare
1.53.6

Merged PRs

dolt

  • 9236: go: sqle/statspro: prollyStats: FlushFromMem: Fix long-held lock when flushing stats to disk.
    For sql-server instances with many databases, many branches or many tables and indexes, flushing stats could cause queries to block until stats were finished flushing. StatsController attempted to rate limited how much I/O bandwidth would be used to flush stats, but this is currently counter-productive because all queries are blocked from analyzing successfully until stats are flushed.
    This changes it back to flush stats to disk as quickly as possible when we need to do it. Computing stats is already rate limited, so for large flushes this should not happen too often.
    We will continue to improve things here so that stats writes have less impact on server resource utilization and do not block user-facing work as they proceed.
  • 9229: When performing a three-way merge, Use a patchBuffer to build the new primary index instead of a MutableMap.
    MutableMaps are designed for caching point modifications to a table. But during merge, the new primary index is computed sequentially. There's no benefit to using a MutableMap here.
    MutableMaps are built on top of the ApplyMutations function, which takes a sequential stream of modifications (called a PatchBuffer) and applies them to a chunker. This has the added benefit of being parallelizable: the patches are produced in one goroutine and consumed in another.
    Instead of using the MutableMap, we can extract the underlying PatchBuffer and use it directly. This should be both more performant, and more correct as it avoids a failure case with schema merges where the MutableMap flushes changes to disk and writes a chunk containing rows with different schemas.
  • 9215: go/store/nbs: table_index.go: Fix uint32 overflow in findPrefix when working with large table files.

go-mysql-server

  • 2985: fix default value for non null enum columns
    In MySQL non null enum columns default to the very first enum instead of throwing a missing default error.
    fixes: #8617
  • 2983: fix explain plan on server engine
    fixes: #8882
  • 2982: Bug fix: use ExtendedType serialization if present
    The grouping key wasn't getting created correctly for DoltgresTypes, since they implement sql.StringType, but aren't supported in GMS' types.ConvertToString function. Tests for grouping with DoltgresTypes added in Doltgres.
  • 2979: fix indexing for GROUP BYs and WINDOWs in INSERT and REPLACE statements in TRIGGERS
    Using aggregation and window functions inside a select statement inside an insert source inside a trigger was causing problems. For example, a trigger defined like so: 
    create trigger trig before insert on t1
for each row
begin
insert into t2
select
max(id),
first_value(id) over (partition by id order by id),
...
from
t3;
end;
    The issue involved the Projections over the Group Bys. The scope for the group bys already contained the trigger's columns and are indexed uniquely, so we shouldn't include the trigger/parent scope.

Closed Issues

  • 8617: INSERT to ENUM column with NOT NULL errors
  • 8882: Error in DESCRIBE PLAN SELECT COUNT(*) FROM table on running server
  • 5317: dolt FixedWidthTableWriter panic when calling stored proc
  • 5530: PREPARE / EXECUTE in stored procedures is buggy
  • 7068: first call of a stored procedure that uses session user variable always return NULL
  • 9213: Connection concurrency issues with DDL

Performance

Read Tests MySQL Dolt Multiple
covering_index_scan 1.86 0.67 0.36
groupby_scan 13.46 17.95 1.33
index_join 1.47 2.35 1.6
index_join_scan 1.42 1.44 1.01
index_scan 34.33 30.81 0.9
oltp_point_select 0.18 0.26 1.44
oltp_read_only 3.43 5.18 1.51
select_random_points 0.33 0.6 1.82
select_random_ranges 0.36 0.62 1.72
table_scan 34.95 32.53 0.93
types_table_scan 75.82 127.81 1.69
reads_mean_multiplier 1.3
Write Tests MySQL Dolt Multiple
oltp_delete_insert 8.9 6.32 0.71
oltp_insert 4.1 3.07 0.75
oltp_read_write 8.9 11.45 1.29
oltp_update_index 4.18 3.19 0.76
oltp_update_non_index 4.18 3.07 0.73
oltp_write_only 5.67 6.32 1.11
types_delete_insert 8.28 6.67 0.81
writes_mean_multiplier 0.88
TPC-C TPS Tests MySQL Dolt Multiple
tpcc-scale-factor-1 98.1 40.36 2.43
tpcc_tps_multiplier 2.43
Overall Mean Multiple 1.54
Loading

1.53.5

16 May 20:15
@github-actions github-actions
v1.53.5
218f9c2
Compare
Choose a tag to compare
1.53.5

Merged PRs

dolt

  • 9204: feat(import,csv,psv): Add support for importing CSV and PSV files without header rows

    Summary

    • Add --no-header flag to treat the first row in CSV/PSV files as data instead of column names
    • Add --columns option to specify column names when importing files without headers
    • Fix nil pointer panic when importing from stdin with --create-table
      In short, this feature makes Dolt more compatible with MySQL/SQLite workflows and provides users with more flexibility when importing data.

    Problem

    Previously, Dolt always expected the first row of CSV/PSV files to contain column names. This differs from MySQL and SQLite which support importing files where the first row contains data. Users migrating from these systems or working with headerless data files couldn't import them without modifying their files.
    Additionally, when users attempted to import data from stdin using --create-table, they would encounter a nil pointer panic instead of receiving a error message.

    Solution

    The implementation adds:
    1. A new --no-header flag that treats the first row as data instead of column headers
    2. A complementary --columns option to specify column names when headers aren't present
    3. Proper validation to ensure correct flag combinations
    4. Comprehensive error handling for stdin imports with clear error messages
    5. Integration tests for both CSV and PSV files

    Testing

    • Added integration tests for both CSV and PSV files that verify:
    • Importing files with --no-header and --columns options
    • Error cases when required options are missing
    • Original behavior is maintained when not using --no-header
    • Behavior of --columns with and without --no-header
    • Edge cases like stdin imports

go-mysql-server

  • 2981: Unwrap wrapper values used in JSON aggregation functions, and un-skip accidentally-skipped tests for this behavior.
    We were accidentally skipping most of the tests in TestJsonScripts. An error in the test harness meant that skipping one test in this suite would also skip all additional tests.
    A few of the skipped tests were for JSON aggregation functions. The recent "Adaptive Encoding / Wrapper Values" optimization wasn't working properly with these functions because the wrapped values provided to these functions weren't being unwrapped before being inserted into JSON documents. These tests would have caught that issue, but didn't because they were disabled.
    This PR fixes the issue and also re-enables the test.
  • 2979: fix indexing for GROUP BYs and WINDOWs in INSERT and REPLACE statements in TRIGGERS
    Using aggregation and window functions inside a select statement inside an insert source inside a trigger was causing problems. For example, a trigger defined like so: 
    create trigger trig before insert on t1
for each row
begin
insert into t2
select
max(id),
first_value(id) over (partition by id order by id),
...
from
t3;
end;
    The issue involved the Projections over the Group Bys. The scope for the group bys already contained the trigger's columns and are indexed uniquely, so we shouldn't include the trigger/parent scope.

Closed Issues

  • 9222: JSON_OBJECT error on longtext columns - "unsupported type: *val.TextStorage"
  • 7831: Allow importing CSVs without column names.

Performance

Read Tests MySQL Dolt Multiple
covering_index_scan 2.0 0.65 0.32
groupby_scan 13.46 17.95 1.33
index_join 1.47 2.39 1.63
index_join_scan 1.42 1.5 1.06
index_scan 34.33 30.26 0.88
oltp_point_select 0.18 0.26 1.44
oltp_read_only 3.43 5.28 1.54
select_random_points 0.33 0.59 1.79
select_random_ranges 0.37 0.61 1.65
table_scan 34.33 32.53 0.95
types_table_scan 75.82 125.52 1.66
reads_mean_multiplier 1.3
Write Tests MySQL Dolt Multiple
oltp_delete_insert 8.9 6.32 0.71
oltp_insert 4.1 3.07 0.75
oltp_read_write 8.74 11.45 1.31
oltp_update_index 4.18 3.19 0.76
oltp_update_non_index 4.18 3.07 0.73
oltp_write_only 5.67 6.32 1.11
types_delete_insert 8.28 6.67 0.81
writes_mean_multiplier 0.88
TPC-C TPS Tests MySQL Dolt Multiple
tpcc-scale-factor-1 97.61 39.19 2.49
tpcc_tps_multiplier 2.49
Overall Mean Multiple 1.56
Loading

1.53.4

15 May 23:54
@github-actions github-actions
v1.53.4
feeaf2c
Compare
Choose a tag to compare
1.53.4

Merged PRs

dolt

  • 9220: Bump undici from 5.28.5 to 5.29.0 in /.github/actions/ses-email-action
    Bumps undici from 5.28.5 to 5.29.0.
    Release notes

    Sourced from undici's releases.

    v5.29.0

    What's Changed

    Full Changelog: nodejs/undici@v5.28.5...v5.29.0

    Commits

    [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=undici&package-manager=npm_and_yarn&previous-version=5.28.5&new-version=5.29.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
    Dependabot commands and options
    You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/dolthub/dolt/network/alerts).
  • 9219: Adding dolt_update_column_tag() stored procedure
    Adds the dolt_update_column_tag() stored procedure to update a column's tag. This is an advanced operation that most users will not need to use. This blog explains more about column tags and column tag conflicts.
    Example usage: 
    CALL dolt_update_column_tag('myTable', 'col1', 4321);
    Follow ups:
    • Add docs for the new dolt_schema_update_tag() stored procedure
    • The dolt schema update-tag CLI command should be updated to use this stored procedure.

go-mysql-server

  • 2981: Unwrap wrapper values used in JSON aggregation functions, and un-skip accidentally-skipped tests for this behavior.
    We were accidentally skipping most of the tests in TestJsonScripts. An error in the test harness meant that skipping one test in this suite would also skip all additional tests.
    A few of the skipped tests were for JSON aggregation functions. The recent "Adaptive Encoding / Wrapper Values" optimization wasn't working properly with these functions because the wrapped values provided to these functions weren't being unwrapped before being inserted into JSON documents. These tests would have caught that issue, but didn't because they were disabled.
    This PR fixes the issue and also re-enables the test.
  • 2980: Unwrap inputs to JSON_OBJECT.
    The inputs to the JSON_OBJECT function are expected to be strings for the keys, and the expected types for the values. So we need to check whether the inputs are wrapper values and unwrap them.
    The safest thing to do is unwrap the value when the document is created, so this PR does that. In theory, allowed wrapped values to be used as document values and unwrapping them at the use site could improve performance for certain queries that create JSON objects in memory but then only read some of the fields. But that's likely an infrequent enough situation that it's not worth the added complexity.
  • 2978: Bug fix: ensure an active transaction is set before preparing a query
    Also adds support for running transaction tests with prepared statements.
    Fixes: #9213

Closed Issues

  • 9222: JSON_OBJECT error on longtext columns - "unsupported type: *val.TextStorage"
  • 9143: index out of range panic on CTE
  • 9213: Connection concurrency issues with DDL
Loading

1.53.3

15 May 00:35
@github-actions github-actions
v1.53.3
169766c
Compare
Choose a tag to compare
1.53.3

Merged PRs

dolt

go-mysql-server

  • 2978: Bug fix: ensure an active transaction is set before preparing a query
    Also adds support for running transaction tests with prepared statements.
    Fixes: #9213
  • 2977: Bug fix for offset without limit
    Used by Doltgres (not legal syntax in MySQL dialect)
  • 2974: fixval.TextStorage panic
    fixes: #9202

vitess

  • 414: add support for with <resource_option> for ALTER USER ... statements
    fixes: #7841
  • 388: syntax for FLUSH TABLE
    MyDumper expects this syntax to at least parse.

Closed Issues

  • 9213: Connection concurrency issues with DDL
  • 8840: "address" values are unnecessarily deserialized/reserialized.
Loading

1.53.2

13 May 00:20
@github-actions github-actions
v1.53.2
b56618e
Compare
Choose a tag to compare
1.53.2

Merged PRs

dolt

  • 9208: When a table is deleted on one branch and has its data modified on the other, record this as a schema conflict.
    Currently, attempting to merge when a table has been drop on one branch and had its data modified on the other produces an immediate error that aborts the merge. This error does not necessarily have enough context to determine the affected table, which makes resolving the merge difficult.
    Given that we already handle a simultaneous drop + schema modification (by treating it as a schema conflict), we should be able to handle the simpler case of data modifications as well.
    Since dropping a table is technically a schema change, this should now get reported as a schema conflict, and pause the merge, allowing the user to query dolt_schema_conflicts for more information.
    After this PR, conflicts involving a dropped table will present the same as other schema conflicts: the user still won't be able to use dolt conflicts resolve to resolve them because of #6616, but the user can query dolt_schema_conflicts to see the cause of the conflict.

go-mysql-server

vitess

  • 402: add variable to colval
  • 388: syntax for FLUSH TABLE
    MyDumper expects this syntax to at least parse.

Closed Issues

  • 7841: Add support for ALTER USER ... WITH <resource option>
  • 9201: Selecting from view using AS OF errors if the view doesn't exist on the current branch
  • 9202: Querying longtext with WHERE IN results in panic: interface conversion: interface {} is *val.TextStorage, not string
  • 9194: Round Trip tables with bit(...) type and DEFAULT value

Performance

Read Tests MySQL Dolt Multiple
covering_index_scan 1.86 0.67 0.36
groupby_scan 13.22 17.95 1.36
index_join 1.47 2.35 1.6
index_join_scan 1.44 1.44 1.0
index_scan 34.33 30.26 0.88
oltp_point_select 0.18 0.26 1.44
oltp_read_only 3.43 5.18 1.51
select_random_points 0.33 0.6 1.82
select_random_ranges 0.36 0.62 1.72
table_scan 34.33 32.53 0.95
types_table_scan 75.82 123.28 1.63
reads_mean_multiplier 1.3
Write Tests MySQL Dolt Multiple
oltp_delete_insert 8.9 6.32 0.71
oltp_insert 4.03 3.07 0.76
oltp_read_write 8.74 11.65 1.33
oltp_update_index 4.1 3.19 0.78
oltp_update_non_index 4.18 3.07 0.73
oltp_write_only 5.67 6.32 1.11
types_delete_insert 8.28 6.67 0.81
writes_mean_multiplier 0.89
TPC-C TPS Tests MySQL Dolt Multiple
tpcc-scale-factor-1 97.65 39.52 2.47
tpcc_tps_multiplier 2.47
Overall Mean Multiple 1.55
Loading

1.53.1

08 May 20:33
@github-actions github-actions
v1.53.1
426950a
Compare
Choose a tag to compare
1.53.1

Merged PRs

dolt

  • 9193: go: store/datas/pull: puller.go: Fix incorrect location for Close call on PullChunkTracker.
    When a pull or push failed with an error in a HasMany call or on the initiating read of a set of chunks from the ChunkFetcher, the initiating goroutine used to close the PullChunkTracker. However, the writes to the PullChunkTracker are issued on the thread which reads and processes the chunks themselves. This meant that it was possible for the PullChunkTracker to be Closed while that thread was still issuing writes against it.
    Making the chunk-reading thread responsible for the Close is more correct.
    This fixes #9190.
  • 9192: go: statspro: Make it possible for callers of StatsController.Stop() to block on the worker thread shutdown.
    Add pre-finalize blocking on the worker thread shutdown to dolt_gc safepoint controllers.
    Add blocking on the worker thread shutdown to dolt_stats_stop(), so that a caller knows that stats have actually stopped by the time it returns.
  • 9186: Fix --branch global option always flagging error "no valid repository"
    Fix issue #6979
    We define noValidRepository to ensure the command has a valid target, but in that definition we call mrEnv.GetEnv() on the wrong string, so it's always null with the --branch flag. We should call it just on the name of the database, excluding the branch.
    A string with the database and branch is also needed later, so we need to save both variables.
  • 9181: go: sqle/dprocedures: fetch,pull,push: Rebase the remote database before we interact with it.
    We need to rebase a remote in order to see its latest changes.
    Fixes #9164.
  • 9179: refactor: replace map iteration with maps.Copy
    Optimize code using a more modern writing style.

go-mysql-server

  • 2967: Unwrap wrapped values before computing their hash.
    (This PR changes the types in one of the tables used in tests. This improves the test coverage for TEXT columns, especially when they're used in conjunction with other tables with VARCHAR columns. None of the existing tests were testing the original type: this should be strictly increasing our test coverage.)
    The following plans involve computing a hash of rows to store in an in-memory hash set:
    • Intersect, Except, and Distinct
    • HashLookup
    • InSubquery
    • HashSubquery
    • FullJoinIter
    • ConcatJoin
    • Recursive CTE
    • UpdateJoinIter
      We weren't previously unwrapping wrapped values before computing hashes. The default hash implementation used the struct's %v representation to compute the hash, which has two problems:
    • The hash of the %v representation of a wrapper struct is not the same as the hash of the value that the wrapper is semantically equivalent to.
    • The hash of the %v representation of a wrapper struct depends on internal state, such as whether the wrapped has already been unwrapped once before (and cached the unwrapped value in an internal buffer)
      The simplest fix is to unwrap values before computing a row hash in the HashOf function.
      However, this fix comes at a cost: it now requires the engine to unwrap all values if they get used in any of the above plans. This will hurt performance for any of the above plans if they don't actually need to unwrap the value. For example, an UpdateJoinIter on a table with a TEXT column will now load that column from disk, even if its value is never used.
      A better fix might be to use the Hash() function that is already defined on the sql.Wrapper interface. For all existing Wrapper implementations, this returns the Dolt content address of the value, and is the same regardless of whether or not that address has previously been resolved. However, this would still return a different hash than an equivalent string. If we wanted them to return the same hash, Dolt would need to define a custom hash for strings that computes the Dolt content address of the string if it were to be stored as a Dolt chunk. This would likely be slower than Go's builtin hash for strings, although the performance might be comparable? This would likely result in worse performance for plans that don't use TEXT columns.
  • 2966: Fix float to bigint conversion result
    Fixes: #9178
    Test to be added to dolt engine tests
  • 2958: Correctly map enum values when altering an enum column.
    When altering the possible values of an enum column, the mapping between enum strings and their indexes may change. In order to alter the column correctly, we need to map the old indexes onto the new ones.
    This PR fixes two bugs in that mapping process:
    • Previously, we were converting the row to the new schema before setting the enums to their new index. This means that if the old row had an index that didn't exist in the new schema, this conversion would fail. By setting the enums to their new index before we convert the row to the new schema, we avoid this problem.
    • Enum indexes are 1-indexed, with the 0 value representing an "invalid" value. One way to get an invalid enum in MySQL is an INSERT IGNORE statement where the provided value for the enum is out of range. Dolt also allows setting the column to 0 directly, although MySQL does not. Previously, we were not able to alter enum columns if they contained a 0 value, but now we can.

Closed Issues

  • 9180: Unintuitive error messages when using global options --branch and --use-branch on invalid branches
  • 6979: --branch global option doesn't work by itself
  • 9190: Panic when trying to push wikipedia db
  • 9166: More panic: interface conversion: interface {} is *val.TextStorage, not string
  • 9178: Problem modifying a column to change from float to bigint
  • 9164: push/pull synchronization fails with file remote in multi-node setup

Performance

Read Tests MySQL Dolt Multiple
covering_index_scan 1.89 0.67 0.35
groupby_scan 13.22 17.95 1.36
index_join 1.47 2.39 1.63
index_join_scan 1.44 1.44 1.0
index_scan 34.33 30.26 0.88
oltp_point_select 0.18 0.26 1.44
oltp_read_only 3.43 5.18 1.51
select_random_points 0.33 0.59 1.79
select_random_ranges 0.37 0.62 1.68
table_scan 34.33 32.53 0.95
types_table_scan 75.82 125.52 1.66
reads_mean_multiplier 1.3
Write Tests MySQL Dolt Multiple
oltp_delete_insert 8.9 6.32 0.71
oltp_insert 4.03 3.13 0.78
oltp_read_write 8.9 11.45 1.29
oltp_update_index 4.1 3.19 0.78
oltp_update_non_index 4.18 3.07 0.73
oltp_write_only 5.67 6.32 1.11
types_delete_insert 8.28 6.67 0.81
writes_mean_multiplier 0.89
TPC-C TPS Tests MySQL Dolt Multiple
tpcc-scale-factor-1 98.01 39.33 2.49
tpcc_tps_multiplier 2.49
Overall Mean Multiple 1.56
Loading

1.53.0

05 May 18:23
@github-actions github-actions
v1.53.0
eff4680
Compare
Choose a tag to compare
1.53.0

There's no known breaking changes in this release. However, we did rewrite the Stored Procedure engine. As far as we know they changes only improve correctness but it's a big change so we made a minor version bump.

Merged PRs

dolt

  • 9171: dolt status user contribution
  • 9167: test for temporary tables in stored procedures
    The stored procedures rewrite fixed an issue where temporary tables aren't able to be created and selected within a stored procedure; this PR adds tests to specifically test for that case.
    fixes: #8762
  • 9165: go: keymutex: Improve cleanup and refcnt implementation so that we can keep the mutexes map clean in more cases.
  • 9155: fix status command after removing origin from remote
    Fix #9130 and #9069. 
    $ dolt clone timsehn/docs
$ cd docs
$ dolt remote -v
origin https://doltremoteapi.dolthub.com/timsehn/docs
$ dolt remote rm origin
$ dolt status
On branch main
Your branch is up to date with 'origin/main'.
nothing to commit, working tree clean
  • 9120: Consistently use pointer receiver for ByteArray
    Similar to #9100
    For structs that are exclusively used in interfaces, there's no reason to not use pointer receivers instead of value receivers. Using value receivers can cause the struct to be copied every time the method is invoked.
    Pointer receivers are also more correct here: copying the struct means that changes to the ImmutableValue field (such as setting its Buf field in ImmutableValue.GetBytes) won't be saved when the method exits, defeating the caching of values loaded from storage.

go-mysql-server

  • 2964: Unwrap inputs to REGEXP_LIKE, REPLACE, and RPAD/LPAD functions.
    These are a couple more functions whose inputs aren't being unwrapped. If the inputs come from a column with a TEXT type, then they won't be converted to strings and the functions will fail. This PR fixes this by making sure that the inputs are converted.
    We missed this previously because although we have tests for these functions, none of them tested getting the inputs from TEXT columns of a table.
  • 2963: From unixtime
    Merged main from dolthub/go-mysql-server#2910
  • 2962: fix infinite recursion with recursive stored procedures
    fixes: #8763
  • 2961: fix prepare statements and user vars in stored procedures
    This PR adds tests for user variables within Stored Procedures.
    Additionally, this PR has changes so that we don't double parse PREPARE ... FROM ... statements, which also allows them to be used in the updated Stored Procedures call.
    fixes: #8911
  • 2959: Doltgres literal support
    This makes literal expression implement the sqlparser.Injectable interface so they can be used in Doltgres AST conversion.
    Also exports fields so literals can be edited in place (required for how Doltgres processes these in a couple places).
    This is an alternative to dolthub/go-mysql-server#2946.
  • 2955: Add support for UPDATE ... RETURNING
    Related to: dolthub/doltgresql#1421
  • 2851: rewriting stored procedures
    This PR reimplements Stored Procedures so that queries are run individually, while maintaining context through the use of a stack.
    companion pr: dolthub/vitess#402

vitess

  • 413: Add support for RETURNING to UPDATE and INSERT nodes
  • 402: add variable to colval

Closed Issues

  • 8911: can't assign local variables to user variables
  • 8763: Stored procedures: stored procedure with recursive call does not terminate
  • 9130: Dolt and Git differ if status is run after deleting the origin remote.
  • 8762: Stored procedure: temp table created in procedure not visible
  • 9149: CREATE TEMPORARY TABLE during a transaction causes a commit
  • 2957: Unexpected syntax error when using AFTER clause in ALTER TABLE statement

Performance

Read Tests MySQL Dolt Multiple
covering_index_scan 1.96 0.65 0.33
groupby_scan 13.46 17.63 1.31
index_join 1.44 2.35 1.63
index_join_scan 1.42 1.44 1.01
index_scan 34.33 30.26 0.88
oltp_point_select 0.18 0.26 1.44
oltp_read_only 3.36 5.18 1.54
select_random_points 0.33 0.59 1.79
select_random_ranges 0.36 0.62 1.72
table_scan 34.33 32.53 0.95
types_table_scan 75.82 123.28 1.63
reads_mean_multiplier 1.29
Write Tests MySQL Dolt Multiple
oltp_delete_insert 8.9 6.32 0.71
oltp_insert 4.03 3.13 0.78
oltp_read_write 8.74 11.45 1.31
oltp_update_index 4.1 3.19 0.78
oltp_update_non_index 4.18 3.07 0.73
oltp_write_only 5.67 6.32 1.11
types_delete_insert 8.28 6.67 0.81
writes_mean_multiplier 0.89
TPC-C TPS Tests MySQL Dolt Multiple
tpcc-scale-factor-1 97.55 39.68 2.46
tpcc_tps_multiplier 2.46
Overall Mean Multiple 1.55
Loading
0