[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Page MenuHomePhabricator
Create Task
Maniphest T247865

Fatal error: Call to fetch_object() on boolean (in DatabaseMysqli->mysqlFetchObject)
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

Error

MediaWiki version: 1.35.0-wmf.23

message
Fatal error: Call to a member function fetch_object() on boolean
exception.trace
#0 /srv/mediawiki/php-1.35.0-wmf.23/includes/libs/rdbms/database/DatabaseMysqlBase.php(255): Wikimedia\Rdbms\DatabaseMysqli->mysqlFetchObject(boolean)
#1 /srv/mediawiki/php-1.35.0-wmf.23/includes/libs/rdbms/database/DatabaseMysqlBase.php(1091): Wikimedia\Rdbms\DatabaseMysqlBase->fetchObject(boolean)
#2 /srv/mediawiki/php-1.35.0-wmf.23/includes/libs/rdbms/loadbalancer/LoadBalancer.php(2065): Wikimedia\Rdbms\DatabaseMysqlBase->serverIsReadOnly()
#3 /srv/mediawiki/php-1.35.0-wmf.23/includes/libs/objectcache/BagOStuff.php(148): Wikimedia\Rdbms\LoadBalancer->Wikimedia\Rdbms\{closure}(integer)
#4 /srv/mediawiki/php-1.35.0-wmf.23/includes/libs/rdbms/loadbalancer/LoadBalancer.php(2069): BagOStuff->getWithSetCallback(string, integer, Closure)
#5 /srv/mediawiki/php-1.35.0-wmf.23/includes/libs/rdbms/loadbalancer/LoadBalancer.php(966): Wikimedia\Rdbms\LoadBalancer->isMasterConnectionReadOnly(Wikimedia\Rdbms\DatabaseMysqli, integer)
#6 /srv/mediawiki/php-1.35.0-wmf.23/includes/libs/rdbms/loadbalancer/LoadBalancer.php(901): Wikimedia\Rdbms\LoadBalancer->getServerConnection(integer, string, integer)
#7 /srv/mediawiki/php-1.35.0-wmf.23/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1031): Wikimedia\Rdbms\LoadBalancer->getConnection(integer, array, string, integer)
#8 /srv/mediawiki/php-1.35.0-wmf.23/includes/externalstore/ExternalStoreDB.php(168): Wikimedia\Rdbms\LoadBalancer->getConnectionRef(integer, array, string, integer)
#9 /srv/mediawiki/php-1.35.0-wmf.23/includes/externalstore/ExternalStoreDB.php(311): ExternalStoreDB->getReplica(string)
#10 /srv/mediawiki/php-1.35.0-wmf.23/includes/externalstore/ExternalStoreDB.php(66): ExternalStoreDB->fetchBlob(string, string, boolean)
#11 /srv/mediawiki/php-1.35.0-wmf.23/includes/externalstore/ExternalStoreAccess.php(52): ExternalStoreDB->fetchFromURL(string)
#12 /srv/mediawiki/php-1.35.0-wmf.23/includes/Storage/SqlBlobStore.php(496): ExternalStoreAccess->fetchFromURL(string, array)
#13 /srv/mediawiki/php-1.35.0-wmf.23/includes/libs/objectcache/wancache/WANObjectCache.php(1471): MediaWiki\Storage\SqlBlobStore->MediaWiki\Storage\{closure}(boolean, integer, array, NULL, array)
#14 /srv/mediawiki/php-1.35.0-wmf.23/includes/libs/objectcache/wancache/WANObjectCache.php(1314): WANObjectCache->fetchOrRegenerate(string, integer, Closure, array, array)
#15 /srv/mediawiki/php-1.35.0-wmf.23/includes/Storage/SqlBlobStore.php(500): WANObjectCache->getWithSetCallback(string, integer, Closure, array)
#16 /srv/mediawiki/php-1.35.0-wmf.23/includes/Storage/SqlBlobStore.php(419): MediaWiki\Storage\SqlBlobStore->expandBlob(string, array, string)
#17 /srv/mediawiki/php-1.35.0-wmf.23/includes/Storage/SqlBlobStore.php(286): MediaWiki\Storage\SqlBlobStore->fetchBlobs(array, integer)
#18 /srv/mediawiki/php-1.35.0-wmf.23/includes/libs/objectcache/wancache/WANObjectCache.php(1471): MediaWiki\Storage\SqlBlobStore->MediaWiki\Storage\{closure}(boolean, integer, array, NULL, array)
#19 /srv/mediawiki/php-1.35.0-wmf.23/includes/libs/objectcache/wancache/WANObjectCache.php(1314): WANObjectCache->fetchOrRegenerate(string, integer, Closure, array, array)
#20 /srv/mediawiki/php-1.35.0-wmf.23/includes/Storage/SqlBlobStore.php(291): WANObjectCache->getWithSetCallback(string, integer, Closure, array)
#21 /srv/mediawiki/php-1.35.0-wmf.23/maintenance/fetchText.php(81): MediaWiki\Storage\SqlBlobStore->getBlob(string)
#22 /srv/mediawiki/php-1.35.0-wmf.23/maintenance/doMaintenance.php(99): FetchText->execute()
#23 /srv/mediawiki/php-1.35.0-wmf.23/maintenance/fetchText.php(100): require_once(string)
#24 /srv/mediawiki/multiversion/MWScript.php(101): require_once(string)
#25 {main}

Impact

We see that some maintenance scripts have failed.

The code path that leads to the issue however does not seem to be specific to dumps or maintenance in any way. I would imagine it is a general bug in the Rdbms lib that could affect other web requests as well.

Notes

Details

Request ID
63ce333480f33f0c6b979399
Request URL
n/a
SubjectRepoBranchLines +/-
rdbms: Don't silence errors in DatabaseMysqlBase::serverIsReadOnly()mediawiki/coremaster+1 -1
Customize query in gerrit

Event Timeline

Krinkle created this task.Mar 17 2020, 4:21 PM
Restricted Application added a project: Platform Engineering. · View Herald TranscriptMar 17 2020, 4:21 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Krinkle added a comment.Mar 17 2020, 4:23 PM
DatabaseMysqli.php
	/**
	 * @param mysqli_result $res
	 * @return stdClass|bool
	 */
	protected function mysqlFetchObject( $res ) {
		$object = $res->fetch_object();
DatabaseMysqlBase.php
	/**
	 * @param IResultWrapper|resource $res
	 */
	public function fetchObject( $res ) {
		$row = $this->mysqlFetchObject( ResultWrapper::unwrap( $res ) );

Then, we see that fetchObject is sometimes called conditionally after a query() and sometimes unconditionally.

DatabaseMysqlBase.php
	// unconditionally 
	public function serverIsReadOnly() {

		$flags = self::QUERY_IGNORE_DBO_TRX | self::QUERY_SILENCE_ERRORS;
		$res = $this->query( "SELECT @@GLOBAL.read_only AS Value", __METHOD__, $flags );
		$row = $this->fetchObject( $res );

	/// unconditionally

		$res = $this->query( 'SHOW SLAVE STATUS', __METHOD__, $flags );
		$row = $res ? $res->fetchObject() : false;

That looks suspicious but I am not familiar with the values that are being queried by serverIsReadOnly, nor the intended behaviour of QUERY_SILENCE_ERRORS to know whether it is supposed to be possible for query() to return false in this case.

Krinkle moved this task from Untriaged to Mar 2020 on the Wikimedia-production-error board.Mar 17 2020, 4:23 PM
Anomie moved this task from Inbox to Triage Meeting Inbox on the Platform Engineering board.Mar 17 2020, 9:27 PM
Anomie subscribed.

It'd be interesting to know what the error is, that query doesn't seem like it should fail. The error should still have been logged somewhere, can you check?

QUERY_SILENCE_ERRORS is indeed intended to make query() return false instead of throw in certain cases. So either serverIsReadOnly() should check for false and do something sensible, or it should not pass QUERY_SILENCE_ERRORS and allow an exception to be thrown if the query somehow errors.

Anomie claimed this task.Mar 24 2020, 8:19 PM
Anomie edited projects, added Platform Team Workboards (Clinic Duty Team); removed Platform Engineering.
Anomie moved this task from Inbox to Ready (WIP:5) on the Platform Team Workboards (Clinic Duty Team) board.
Anomie triaged this task as Medium priority.Mar 24 2020, 8:50 PM
Anomie moved this task from Ready (WIP:5) to Waiting for Review on the Platform Team Workboards (Clinic Duty Team) board.Mar 25 2020, 7:11 PM
gerritbot added a comment.Mar 25 2020, 7:11 PM

Change 583416 had a related patch set uploaded (by Anomie; owner: Anomie):
[mediawiki/core@master] rdbms: Don't silence errors in DatabaseMysqlBase::serverIsReadOnly()

https://gerrit.wikimedia.org/r/583416

gerritbot added a project: Patch-For-Review.Mar 25 2020, 7:11 PM
gerritbot added a comment.Mar 26 2020, 6:41 AM

Change 583416 merged by jenkins-bot:
[mediawiki/core@master] rdbms: Don't silence errors in DatabaseMysqlBase::serverIsReadOnly()

https://gerrit.wikimedia.org/r/583416

ReleaseTaggerBot added a project: MW-1.35-notes (1.35.0-wmf.26; 2020-03-31).Mar 26 2020, 7:00 AM
Maintenance_bot removed a project: Patch-For-Review.Mar 26 2020, 7:10 AM
Anomie closed this task as Resolved.Mar 26 2020, 2:27 PM
Anomie moved this task from Waiting for Review to Done on the Platform Team Workboards (Clinic Duty Team) board.
Aklapper removed a subscriber: Anomie.Oct 16 2020, 5:41 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits