Page MenuHomePhabricator
Create Task
Maniphest T355089

CI broken for ORES/PageTriage. Insert returned unacceptable warning: Data truncated for column 'oresc_probability' at row 1
Closed, ResolvedPublicBUG REPORT
Actions

Description

Steps to replicate the issue (include links if applicable):

  • Upload a patch to the ORES or PageTriage repositories via Gerrit
  • Wait for CI to finish

What happens?:

12:06:08 INFO:quibble.commands:Shutting down backends: <ExternalWebserver http://127.0.0.1:9413 /workspace/src> <Xvfb :94> <ChromeWebDriver :94>
12:06:08 INFO:backend.ChromeWebDriver:Terminating ChromeWebDriver
12:06:08 INFO:backend.Xvfb:Terminating Xvfb
12:06:08 INFO:quibble.commands:Shutting down backends: <MySQL /workspace/db/quibble-mysql-x90vwkuc/socket>
12:06:08 INFO:backend.MySQL:Terminating MySQL
12:06:08 [quibble-vendor-mysql-php74-noselenium-docker] $ /bin/bash -xe /tmp/jenkins14156388653785436904.sh
12:06:11 + set -eu
12:06:11 + ERROR_FILES=(mw-dberror.log mw-error.log)
12:06:11 + echo 'Asserting empty files: mw-dberror.log mw-error.log'
12:06:11 Asserting empty files: mw-dberror.log mw-error.log
12:06:11 ++ cd /srv/jenkins/workspace/quibble-vendor-mysql-php74-noselenium-docker/log
12:06:11 ++ ls mw-dberror.log mw-error.log
12:06:11 ++ :
12:06:11 + log_files=mw-dberror.log
12:06:11 + '[' '!' mw-dberror.log ']'
12:06:11 + echo 'Dumping file(s) mw-dberror.log'
12:06:11 Dumping file(s) mw-dberror.log
12:06:11 + set +e
12:06:11 + cd /srv/jenkins/workspace/quibble-vendor-mysql-php74-noselenium-docker/log
12:06:11 + grep --color . mw-dberror.log mw-error.log
12:06:11 mw-dberror.log:Mon Jan 15 20:06:08 UTC 2024	5973089a7101	wikidb-unittest_	Error 1265 from ORES\Storage\SqlScoreStorage::storeScores, Insert returned unacceptable warning: Data truncated for column 'oresc_probability' at row 1 INSERT IGNORE INTO `unittest_ores_classification` (oresc_rev,oresc_model,oresc_class,oresc_probability,oresc_is_predicted) VALUES (17,5,1,'0.2405552732802',0),(17,7,1,'0.74879388161087',1) localhost:/workspace/db/quibble-mysql-x90vwkuc/socket
12:06:11 mw-dberror.log:#0 /workspace/src/includes/libs/rdbms/database/Database.php(1161): Wikimedia\Rdbms\Database->getQueryExceptionAndLog()
12:06:11 mw-dberror.log:#1 /workspace/src/includes/libs/rdbms/database/DatabaseMySQL.php(274): Wikimedia\Rdbms\Database->reportQueryError()
12:06:11 mw-dberror.log:#2 /workspace/src/includes/libs/rdbms/database/Database.php(1483): Wikimedia\Rdbms\DatabaseMySQL->checkInsertWarnings()
12:06:11 mw-dberror.log:#3 /workspace/src/includes/libs/rdbms/database/DBConnRef.php(119): Wikimedia\Rdbms\Database->insert()
12:06:11 mw-dberror.log:#4 /workspace/src/includes/libs/rdbms/database/DBConnRef.php(407): Wikimedia\Rdbms\DBConnRef->__call()
12:06:11 mw-dberror.log:#5 /workspace/src/extensions/ORES/includes/Storage/SqlScoreStorage.php(101): Wikimedia\Rdbms\DBConnRef->insert()
12:06:11 mw-dberror.log:#6 /workspace/src/extensions/ORES/includes/Services/FetchScoreJob.php(98): ORES\Storage\SqlScoreStorage->storeScores()
12:06:11 mw-dberror.log:#7 /workspace/src/extensions/ORES/tests/phpunit/includes/Services/FetchScoreJobTest.php(131): ORES\Services\FetchScoreJob->run()
12:06:11 mw-dberror.log:#8 /workspace/src/vendor/phpunit/phpunit/src/Framework/TestCase.php(1548): ORES\Tests\FetchScoreJobTest->testRun()
12:06:11 mw-dberror.log:#9 /workspace/src/vendor/phpunit/phpunit/src/Framework/TestCase.php(1154): PHPUnit\Framework\TestCase->runTest()
12:06:11 mw-dberror.log:#10 /workspace/src/vendor/phpunit/phpunit/src/Framework/TestResult.php(728): PHPUnit\Framework\TestCase->runBare()
12:06:11 mw-dberror.log:#11 /workspace/src/vendor/phpunit/phpunit/src/Framework/TestCase.php(904): PHPUnit\Framework\TestResult->run()
12:06:11 mw-dberror.log:#12 /workspace/src/vendor/phpunit/phpunit/src/Framework/TestSuite.php(675): PHPUnit\Framework\TestCase->run()
12:06:11 mw-dberror.log:#13 /workspace/src/vendor/phpunit/phpunit/src/Framework/TestSuite.php(675): PHPUnit\Framework\TestSuite->run()
12:06:11 mw-dberror.log:#14 /workspace/src/vendor/phpunit/phpunit/src/Framework/TestSuite.php(675): PHPUnit\Framework\TestSuite->run()
12:06:11 mw-dberror.log:#15 /workspace/src/vendor/phpunit/phpunit/src/Framework/TestSuite.php(675): PHPUnit\Framework\TestSuite->run()
12:06:11 mw-dberror.log:#16 /workspace/src/vendor/phpunit/phpunit/src/TextUI/TestRunner.php(653): PHPUnit\Framework\TestSuite->run()
12:06:11 mw-dberror.log:#17 /workspace/src/vendor/phpunit/phpunit/src/TextUI/Command.php(144): PHPUnit\TextUI\TestRunner->run()
12:06:11 mw-dberror.log:#18 /workspace/src/vendor/phpunit/phpunit/src/TextUI/Command.php(97): PHPUnit\TextUI\Command->run()
12:06:11 mw-dberror.log:#19 phpvfscomposer:///workspace/src/vendor/phpunit/phpunit/phpunit(97): PHPUnit\TextUI\Command::main()
12:06:11 mw-dberror.log:#20 /workspace/src/vendor/bin/phpunit(118): include(string)
12:06:11 mw-dberror.log:#21 {main}
12:06:11 + set -e
12:06:11 + echo -e 'MediaWiki emitted some errors. Check output above.'
12:06:11 MediaWiki emitted some errors. Check output above.
12:06:11 + exit 1
12:06:11 Build step 'Execute shell' marked build as failure

What should have happened instead?:
CI passes

Software version (skip for WMF-hosted wikis like Wikipedia):

Other information (browser name/version, screenshots, etc.):
Unit tests pass in local dev environment.

Related Objects

Event Timeline

Novem_Linguae created this task.Jan 15 2024, 8:14 PM
Restricted Application added a project: Machine-Learning-Team. · View Herald TranscriptJan 15 2024, 8:14 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Novem_Linguae edited projects, added ci-test-error (WMF-deployed Build Failure); removed ci-test-error.Jan 15 2024, 10:32 PM
matmarex subscribed.Jan 15 2024, 11:16 PM

Error message is new, from rMW551ec29ea676: Check warnings from INSERT during tests (https://gerrit.wikimedia.org/r/c/mediawiki/core/+/979189)

oresc_probability column is defined as NUMERIC(3, 3), but you're trying to insert 0.74879388161087 (this is not random, but a literal value defined in the test code), which presumably gets truncated to 0.748, causing this warning.

Novem_Linguae added a comment.EditedJan 15 2024, 11:37 PM

Thanks matmarex for the quick response. A search for decimals longer than 3 digits used in PHPUnit tests in this codebase:

https://codesearch.wmcloud.org/deployed/?q=%5C.%5Cd%7B4%2C%7D&files=Test%5C.php&excludeFiles=&repos=mediawiki%2Fextensions%2FORES

I guess one fix would be to manually truncate these to 3 digits.

Another fix might be to round() in whatever method is doing that SQL database write.

Third possible fix would be marking the test skipped.

Novem_Linguae added a comment.Jan 16 2024, 12:10 AM

Any tips for triggering this error on localhost so I can test solutions? I tried the following and am not getting the error:

  • git pull the latest version of core, containing the merged patch rMW551ec29ea676: Check warnings from INSERT during tests mentioned above
  • in LocalSettings.php, I set $wgDBStrictWarnings = true;
  • in a MediaWiki Docker instance with MariaDB, I ran docker compose exec mediawiki composer phpunit:entrypoint -- extensions/ORES/tests/phpunit
  • tests pass, no errors
Novem_Linguae added a comment.EditedJan 16 2024, 12:22 AM

includes/Storage/SqlScoreStorage.php is catching the error. Hmm.

try {
	$this->dbProvider->getPrimaryDatabase()->insert(
		'ores_classification',
		$dbData,
		__METHOD__,
		[ 'IGNORE' ]
	);
} catch ( DBError $exception ) {
	$this->logger->error(
		'Inserting new data into the datbase has failed:' . $exception->getMessage()
	);
	return;
}
tstarling subscribed.Jan 16 2024, 6:16 AM

I would suggest rounding with sprintf("%.3f", $probability ). If you use round(), there will be another implicit rounding operation which takes place when the number is converted to a string by addQuotes(). Using sprintf() rounds to the nearest number (it doesn't just truncate), and it also ensures that the string representation has the correct number of digits.

Novem_Linguae added a comment.Jan 16 2024, 8:40 AM

I can now test this in localhost. I added throw $exception; to the catch block.

Thanks Tim. I took a stab at writing a patch that uses your sprintf suggestion, but I ended up breaking a bunch of tests. I don't know this repo very well. I may try a different approach.

fix-ci.patch2 KBDownload
Novem_Linguae added a comment.EditedJan 16 2024, 9:07 AM

I think I figured it out :) Gerritbot should post the patch in a moment.

https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ORES/+/990949

ReleaseTaggerBot added a project: MW-1.42-notes (1.42.0-wmf.15; 2024-01-23).Jan 16 2024, 11:00 AM
Novem_Linguae closed this task as Resolved.Jan 16 2024, 11:03 AM
Novem_Linguae claimed this task.
Novem_Linguae mentioned this in T333534: Add PageTriage to gated extensions.Feb 6 2024, 7:47 PM
calbon moved this task from Unsorted to 2023-2024 Q3 Done on the Machine-Learning-Team board.Feb 13 2024, 3:12 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits