Warnings on exit when 8000 DBI handle is global variable and there is assignment from subroutine #23306
Comments
|
This is a problem with DBI, which apparently needs to take more care to close down its connections in the right order during global destru 8000 ction. |
|
Why the same DBI version with perl 5.36 doesn't print the warning? |
|
@karenetheridge suggestions on improvement welcome! |
|
On Mon, May 19, 2025 at 05:06:56AM -0700, vvv2542 wrote:
The code below prints warnings on exit:
``` perl
#!/usr/bin/env perl
use strict;
use warnings;
use DBI;
my $dbh = DBI->connect("dbi:Pg:dbname=db1", "user1", "password");
print "DONE\n";
exit 0;
sub sub1 {
$dbh = "";
}
Note that, by referring to $dbh within the sub, you're creating a closure.
This can delay or alter the point at which $dbh is destroyed.
Whether this is then showing up an issue with DBI, or with perl, or is
simply a "don't do this", I can't say off-hand.
…--
This is a great day for France!
-- Nixon at Charles De Gaulle's funeral
|
|
I've encountered a similar situation in my own code when moving from 5.38 to 5.40 What is interesting is that the errors are not reproducible. So far, I've been unable to reproduce the error by running the work load again. Not sure why yet. Two nodes, running the same code, against similar databases, won't necessarily encounter the error when doing the same set of work. Again, not sure why. Given the data and work, if one node hits the error, everything running that same code should as well. |
My testing on FreeBSD 14.2 with PostgreSQL 16 gives the same error If I remove |
|
During global destruction, the order in which resources are freed can be pretty random, so depending on the order in which things can happen, something can expect something to still exist which surprisingly does not. Is there some graceful disconnecting or teardowns that you can do in an END{} block, or in the destructor for a particular object? |
|
I my case adding END{} solves the problem. But I don't understand why in different scripts helps $dbh->disconnect, or $dbh=undef, or both. The solution for a particular script is unpredictable. But the problem is always reproducible. |
|
What is this |
|
Try END {
$dbh->disconnect;
$dbh = undef;
} |
|
That resolves my test issue. In my case, I think it's a race condition, since the same test does not fail on all hosts every time. |
I would recommend changing that to
No it's not a race condition, it's just unorderedness like @karenetheridge explained. |
|
I created perl5-dbi/dbi#171 |
dlangille
added a commit
to FreshPorts/ports
that referenced
this issue
Jun 1, 2025
After moving from perl 5.38 to perl 5.40, DBI errors started to show up.
DBI db handle 0x86ba253f2a0 has 1 uncleared child handles during global destruction.
dbih_clearcom (dbh 0x86ba253f2a0, com 0x86ba1ba90c0, imp DBD::Pg::db):
FLAGS 0x580111: COMSET Warn PrintError PrintWarn
PARENT DBI::dr=HASH(0x86ba2540480)
KIDS 1 (0 Active)
It is not clear why or how to clean it up.
re: perl5-dbi/dbi#171
Perl/perl5#23306
In my attempts to solve this problem, I've added many of these:
$sth->finish();
And some of thse:
undef $dbh;
And this to database.pm
$dbh_pg->{AutoInactiveDestroy} = 1;
Also added in this release:
* column alignments
dlangille
added a commit
to FreshPorts/ports
that referenced
this issue
Jun 1, 2025
After moving from perl 5.38 to perl 5.40, DBI errors started to show up.
DBI db handle 0x86ba253f2a0 has 1 uncleared child handles during global destruction.
dbih_clearcom (dbh 0x86ba253f2a0, com 0x86ba1ba90c0, imp DBD::Pg::db):
FLAGS 0x580111: COMSET Warn PrintError PrintWarn
PARENT DBI::dr=HASH(0x86ba2540480)
KIDS 1 (0 Active)
It is not clear why or how to clean it up.
re: perl5-dbi/dbi#171
Perl/perl5#23306
In my attempts to solve this problem, I've added many of these:
$sth->finish();
And a few of these (load_xml_into_db.pl, load_xml_into_db_git.pl):
END {
$dbh->disconnect();
}
Also added in this release:
* Jail/upgrade-jail-via-replace.sh
- re https://news.freshports.org/2025/03/27/updating-a-jail-by-replacing-it/
* archive-messages.sh
- searched only one directory deep, so I can store failed results in subdirectories
|
I had a package like this: Adding the This changes my approach to cleaning up this error under Perl 5.40 - previously, I was looking mostly at |
|
IMHO it is the DBD that should guarantee the correct order of destruction, not the DBI. See e.g.: https://github.com/perl5-dbi/DBD-Unify/blob/master/dbdimp.ic#L902 |
|
DBD::Pg hasn't had this problem with any version of Perl through Perl 5.38, but it's somehow DBI or the DBDs which are at fault here? Okaaaaaaayyy.... |
|
The same problem with DBD::mysql also starting from perl 5.40. I still think that the reason is the change in perl's behavior. |
|
Has someone tried to bisect the change in behavior? Right now we don't even know which development release was involved. |
|
FWIW: I couldn't reproduce this using the OP test script and following combination, all built from source:
I'll do a 5.40.2 build, just in case it was something in a maint release. |
Nope, still can't reproduce with 5.40.2 built from source. Those were gcc builds, will try with clang, since that's what the OP's perl was built with.
|
|
I couldn't reproduce with clang on this VM either. No warnings from threaded or non-threaded builds. Not sure if it's a build environment difference or what. |
|
@richardleach wrote:
Very interesting! Thanks for doing all that testing, @richardleach. @vvv2542 : Can you provide additional details on your builds and host configuration? Can you reproduce this error in a Docker container and share the Dockerfile? That would be unimpeachable, I think. |
|
I can reproduce it only on FreeBSD:
|
|
You need to tell You can Edit: That previous comment sounds confused about |
Non-success. |
(On FreeBSD use One of the challenges of bisecting a test program (as we're doing here) is writing something that unambiguously exits |
|
@thesamesam Exactly. Locating the failing commit indicates that you've succeeded. |
|
Using this: I found this: Looking through the logs, I find this: Does this help? |
No, that range doesn't make sense. You mentioned you observed it in 5.38, so the correct range would be |
|
The problem appeared in 5.40. 5.38 works fine in the same environment. |
The problem never occurred when I was using 5.38. I first noticed the problem after installing 5.40 - now the problem happens randomly and is not repeatable with my code - however, it is repeatable every time with the small example shown at #23306 (comment) |
Sorry I misread. Then you need to bisect |
|
Latest test we're running is: with |
|
Are you using MacOS 15 (Darwin 24) or higher. |
|
On irc, it was pointed out that we turned of Darwin locale collation in 9e3a653 due to a bug in Apple. Apple took its locale handling code from FreeBSD many fixes ago, and has not kept up with FreBSD fixes. I have at least one bug report submitted to FreeBSD with no sign of their being interested in fixing it/them. |
I'm guessing this response belongs in #23368 instead? |
|
Yes. sorry |
|
Now testing with: |
|
By request via IRC: |
|
Updated the script to not use ~ After hitting: |
|
Adjusted that script to log both STDIN and STDOUT: Proceeding again, this time with: |
|
@dlangille wrote:
That doesn't seem right... |
Right now, we are struggling to get a fully complete run of bisect, so we're concentrating on a single run of a version we know to be bad. Once we get that, we'll expand the range. Thank you for pointing that out. Please continue doing that. It is appreciated. |
|
The most recent run, on perl 5.40.2, same as what's in my test host gives. Next goal: Compare perl configuration values for both host and the one created by bisect. |
|
perl, as built by the FreeBSD project. I also build my own packages and can easily change the configuration options for perl. |
|
the bisect perl: |
|
This is a diff of the two: |
|
Now testing with |
|
Still failed to produce the problem. Next strategy, test by adding For those following along at home, the current goal is to reproduce the error with a |
|
Still no failure. Tried adding in got an eror, did not save it, something about not finding trace.h(?) - no trying again without |
This isn't my usual interest, but Ii will ask, is the shell %ENV state outside of the bisect runner script, and the shell %ENV state of a process started from the bisect runner script, along with the PP %ENV state state on perl proc startup outside of the bisect runner script, and the PP %ENV state state on a perl proc startup from inside the bisect runner script, are all 4 things 100% byte wise identical? Something like Update: make an strace log of a defective perl+DBI process run and a can't reproduce it perl+DBI process run and compare the 2 strace logs. strace solves 75% of my daily IT problems in 90 seconds or less. |
Is #23306 (comment) at all relevant to the above? |
That also did not fail. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
perl 5.40 prints warnings on exit when DBI handle is global variable and there is assignment from subroutine.
Module:
DBI 1.647
Description
The code below prints warnings on exit:
Result:
The script completed without warnings with perl 5.36 and the same version of DBI. Not tested with 5.38.
No warnings if remove $dbh assignment in sub1() which never called.
Steps to Reproduce
Execute the script with perl 5.40.
Expected behavior
No warnings.
Perl configuration
The text was updated successfully, but these errors were encountered: