Page MenuHomePhabricator
Create Task
Maniphest T355216

Keep a track of HTTP status codes from PhotoDNA
Closed, ResolvedPublic1 Estimated Story Points
Actions

Description

We currently only log HTTP status codes to logstash via the mediamoderation channel. We want to lower the level of the logging to info and debug, to avoid logstash spam. However, we still want to see the count of HTTP status codes returned by PhotoDNA.

Acceptance criteria
  • Send HTTP status codes returned by PhotoDNA to grafana for display
  • Make logs sent to logstash about the status of each scan debug and info level to avoid logstash spam

Related Objects

Event Timeline

Dreamy_Jazz created this task.Jan 17 2024, 11:31 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 17 2024, 11:31 AM
Dreamy_Jazz claimed this task.Jan 17 2024, 11:31 AM
Dreamy_Jazz set the point value for this task to 2.
Dreamy_Jazz moved this task from Priority Backlog to In Progress on the Trust and Safety Product Sprint (Sprint Northumbrian smallpipes (8th Jan.‘24 - 19th Jan.'24)) board.
gerritbot added a comment.Jan 17 2024, 12:13 PM

Change 991315 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/MediaModeration@master] Log to statsd HTTP status codes and reduce logstash log levels

https://gerrit.wikimedia.org/r/991315

gerritbot added a project: Patch-For-Review.Jan 17 2024, 12:13 PM
Dreamy_Jazz moved this task from In Progress to Needs review on the Trust and Safety Product Sprint (Sprint Northumbrian smallpipes (8th Jan.‘24 - 19th Jan.'24)) board.Jan 17 2024, 12:16 PM
Dreamy_Jazz changed the point value for this task from 2 to 1.Jan 17 2024, 12:18 PM
Dreamy_Jazz mentioned this in T355243: MediaModeration maintenance script scanFilesInScanTable.php indirectly calls $wgImageMagickConvertCommand.Jan 18 2024, 2:54 PM
gerritbot added a comment.Jan 18 2024, 5:19 PM

Change 991555 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/MediaModeration@wmf/1.42.0-wmf.14] Log to statsd HTTP status codes and reduce logstash log levels

https://gerrit.wikimedia.org/r/991555

gerritbot added a comment.Jan 18 2024, 5:20 PM

Change 991315 merged by jenkins-bot:

[mediawiki/extensions/MediaModeration@master] Log to statsd HTTP status codes and reduce logstash log levels

https://gerrit.wikimedia.org/r/991315

ReleaseTaggerBot added a project: MW-1.42-notes (1.42.0-wmf.15; 2024-01-23).Jan 18 2024, 7:00 PM
Dreamy_Jazz updated the task description. (Show Details)Jan 18 2024, 8:31 PM
gerritbot added a comment.Jan 18 2024, 9:05 PM

Change 991555 merged by jenkins-bot:

[mediawiki/extensions/MediaModeration@wmf/1.42.0-wmf.14] Log to statsd HTTP status codes and reduce logstash log levels

https://gerrit.wikimedia.org/r/991555

Stashbot added a comment.Jan 18 2024, 9:05 PM

Mentioned in SAL (#wikimedia-operations) [2024-01-18T21:05:17Z] <dreamyjazz@deploy2002> Started scap: Backport for [[gerrit:991555|Log to statsd HTTP status codes and reduce logstash log levels (T355216)]]

Stashbot added a comment.Jan 18 2024, 9:08 PM

Mentioned in SAL (#wikimedia-operations) [2024-01-18T21:08:24Z] <dreamyjazz@deploy2002> dreamyjazz: Backport for [[gerrit:991555|Log to statsd HTTP status codes and reduce logstash log levels (T355216)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Stashbot added a comment.Jan 18 2024, 9:14 PM

Mentioned in SAL (#wikimedia-operations) [2024-01-18T21:14:19Z] <dreamyjazz@deploy2002> Finished scap: Backport for [[gerrit:991555|Log to statsd HTTP status codes and reduce logstash log levels (T355216)]] (duration: 09m 00s)

Maintenance_bot removed a project: Patch-For-Review.Jan 18 2024, 9:30 PM
ReleaseTaggerBot edited projects, added MW-1.42-notes (1.42.0-wmf.14; 2024-01-16); removed MW-1.42-notes (1.42.0-wmf.15; 2024-01-23).Jan 18 2024, 10:00 PM
Dreamy_Jazz moved this task from Needs review to Needs QA on the Trust and Safety Product Sprint (Sprint Northumbrian smallpipes (8th Jan.‘24 - 19th Jan.'24)) board.Jan 24 2024, 1:22 PM

QA for this ticket would have to be on production. Setting this up on a local wiki, beta or patch demo is likely not possible

To check this has worked on production, you would do the following steps:

  1. Open the Grafana dashboard
  2. Verify that after the 18th, no items with the name 'Error' appear in the graph named Request successes and errors and instead HTTP status codes 500 appear.
  3. If you have access to production logstash, open https://logstash.wikimedia.org/goto/c83f45f16f4cf9b80fb9427819d1ba68 and verify no events are present after the 18th. If you do not have access, then I have been able to verify that this step passed.
Djackson-ctr moved this task from Needs QA to Done on the Trust and Safety Product Sprint (Sprint Northumbrian smallpipes (8th Jan.‘24 - 19th Jan.'24)) board.Jan 25 2024, 2:28 AM
Djackson-ctr subscribed.

I have verified the new code has been implemented and is functioning and displaying as expected... Thank you for the QA Steps @Dreamy_Jazz.

Dreamy_Jazz closed this task as Resolved.Jan 25 2024, 10:25 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits