Re: Critical Denial of Service bugs in Discover
On Thu, Mar 3, 2022 at 8:41 AM Aleix Pol wrote: > (dropping the distros list) > > @sysadmin have you been able to look into any tools we devs can have to > make sure this situation doesn't repeat in the future? > Hi Aleix, To be honest i've been struggling to think of ways that we could detect this on the server side prior to it becoming a massive issue. By the time an issue is evident server side it is usually much too late. The main tools i'd usually recommend would be the standard tools you would use for monitoring the network activity of any application - such as Wireshark. Is there something you were thinking of specifically in terms of us being able to provide? Thanks, Ben > > Aleix > > On Thu, Feb 10, 2022 at 1:10 PM Aleix Pol wrote: > >> On Thu, Feb 10, 2022 at 11:05 AM Ben Cooksley wrote: >> > >> > >> > >> > On Thu, Feb 10, 2022 at 8:20 AM Aleix Pol wrote: >> >> >> >> [Snip] >> >> >> >> We still haven't discussed here is how to prevent this problem from >> >> happening again. >> >> >> >> If we don't have information about what is happening, we cannot fix >> problems. >> > >> > >> > Part of the issue here is that the problem only came to Sysadmin >> attention very recently, when the system ran out of disk space as a result >> of growing log files. >> > It was at that point we realised we had a serious problem. >> > >> > Prior to that the system load hadn't climbed to dangerous levels (> >> number of CPU cores) and Apache was keeping up with the traffic, so none of >> our other monitoring was tripped. >> > >> > If you have any thoughts on what sort of information you are thinking >> of that would be helpful. >> >> We could have plots of the amount of queries we get with a KNewStuff/* >> user-agent over time and their distribution. >> >> > It would definitely be helpful though to know when new software is >> going to be released that will be interacting with the servers as we will >> then be able to monitor for abnormalities. >> >> We make big announcements of every Plasma release... (?) >> >> >> Is there anything that could be done in this front? The issue here >> >> could have been addressed months ago, we just never knew it was >> >> happening. >> > >> > >> > One possibility that did occur to me today would be for us to integrate >> some kind of killswitch that our applications would check on first >> initialisation of functionality that talks to KDE.org servers. >> > This would allow us to disable the functionality in question on user >> systems. >> > >> > The check would only be done on first initialization to keep load low, >> while still ensuring all users eventually are affected by the killswitch >> (as they will eventually need to logout/reboot for some reason or another). >> > >> > The killswitch would probably work best if it had some kind of version >> check in it so we could specify which versions are disabled. >> > That would allow for subsequent updates - once delivered by >> distributions - to restore the functionality (while leaving it disabled for >> those who haven't updated). >> >> The file we are serving here effectively is the kill switch to all of >> KNewStuff. >> >> Aleix >> >
KDE CI: Frameworks » kio » kf5-qt5 SUSEQt5.15 - Build # 1414 - Failure!
BUILD FAILURE
Build URL
https://build.kde.org/job/Frameworks/job/kio/job/kf5-qt5%20SUSEQt5.15/1414/
Project:
kf5-qt5 SUSEQt5.15
Date of build:
Thu, 03 Mar 2022 11:09:30 +
Build duration:
43 sec and counting
CONSOLE OUTPUT
Started by an SCM changeRunning in Durability level: PERFORMANCE_OPTIMIZED[Pipeline] Start of Pipeline[Pipeline] nodeStill waiting to schedule taskWaiting for next available executor on ‘SUSEQt5.15’Running on Docker Swarm-d71eaea88533 in /home/jenkins/workspace/Frameworks/kio/kf5-qt5 SUSEQt5.15[Pipeline] {[Pipeline] timestamps[Pipeline] {[Pipeline] catchError[Pipeline] {[Pipeline] stage[Pipeline] { (Checkout Sources)[Pipeline] checkout[2022-03-03T11:09:52.439Z] The recommended git tool is: NONE[2022-03-03T11:09:53.484Z] No credentials specified[2022-03-03T11:09:53.487Z] Cloning the remote Git repository[2022-03-03T11:09:53.501Z] Cloning repository https://invent.kde.org/frameworks/kio.git[2022-03-03T11:09:53.556Z] > git init /home/jenkins/workspace/Frameworks/kio/kf5-qt5 SUSEQt5.15 # timeout=10[2022-03-03T11:09:53.580Z] Fetching upstream changes from https://invent.kde.org/frameworks/kio.git[2022-03-03T11:09:53.581Z] > git --version # timeout=10[2022-03-03T11:09:53.595Z] > git --version # 'git version 2.35.1'[2022-03-03T11:09:53.596Z] > git fetch --tags --force --progress -- https://invent.kde.org/frameworks/kio.git +refs/heads/*:refs/remotes/origin/* # timeout=120[2022-03-03T11:10:07.891Z] Avoid second fetch[2022-03-03T11:10:07.921Z] Checking out Revision fa08838141f18959e545f08c9fcc41cffd447555 (origin/master)[2022-03-03T11:10:07.852Z] > git config remote.origin.url https://invent.kde.org/frameworks/kio.git # timeout=10[2022-03-03T11:10:07.872Z] > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10[2022-03-03T11:10:07.896Z] > git rev-parse origin/master^{commit} # timeout=10[2022-03-03T11:10:07.928Z] > git config core.sparsecheckout # timeout=10[2022-03-03T11:10:07.934Z] > git checkout -f fa08838141f18959e545f08c9fcc41cffd447555 # timeout=10[2022-03-03T11:10:11.597Z] Commit message: "Add Qt6 Android CI"[2022-03-03T11:10:11.601Z] > git rev-list --no-walk 2ffcc2cab351b8d64682ccfb7f2c0c1cfa81edbe # timeout=10[Pipeline] checkout[2022-03-03T11:10:11.749Z] The recommended git tool is: NONE[2022-03-03T11:10:11.757Z] No credentials specified[2022-03-03T11:10:11.760Z] Cloning the remote Git repository[2022-03-03T11:10:11.762Z] Cloning repository https://invent.kde.org/sysadmin/ci-tooling.git[2022-03-03T11:10:11.762Z] > git init /home/jenkins/workspace/Frameworks/kio/kf5-qt5 SUSEQt5.15/ci-tooling # timeout=10[2022-03-03T11:10:11.770Z] Fetching upstream changes from https://invent.kde.org/sysadmin/ci-tooling.git[2022-03-03T11:10:11.771Z] > git --version # timeout=10[2022-03-03T11:10:11.783Z] > git --version # 'git version 2.35.1'[2022-03-03T11:10:11.784Z] > git fetch --tags --force --progress -- https://invent.kde.org/sysadmin/ci-tooling.git +refs/heads/*:refs/remotes/origin/* # timeout=10[2022-03-03T11:10:13.115Z] Avoid second fetch[2022-03-03T11:10:13.126Z] Checking out Revision b4fb37c3190807b2b3223e4967638477f54f70bc (origin/master)[2022-03-03T11:10:13.153Z] Commit message: "Initial Android Qt6/KF6 CI image"[Pipeline] checkout[2022-03-03T11:10:13.160Z] The recommended git tool is: NONE[2022-03-03T11:10:13.166Z] No credentials specified[2022-03-03T11:10:13.169Z] Cloning the remote Git repository[2022-03-03T11:10:13.493Z] ERROR: Error cloning remote repo 'origin'[2022-03-03T11:10:13.493Z] hudson.plugins.git.GitException: Command "git fetch --tags --force --progress -- https://invent.kde.org/sysadmin/repo-metadata.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:[2022-03-03T11:10:13.493Z] stdout: [2022-03-03T11:10:13.493Z] stderr: fatal: unable to access 'https://invent.kde.org/sysadmin/repo-metadata.git/': The requested URL returned error: 500[2022-03-03T11:10:13.493Z] [2022-03-03T11:10:13.493Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2450)[2022-03-03T11:10:13.493Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2051)[2022-03-03T11:10:13.493Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:84)[2022-03-03T11:10:13.493Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:573)[2022-03-03T11:10:13.493Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:802)[2022-03-03T11:10:13.493Z] at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)[2022-03-03T11:10:13.493Z] at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154)[2022-03-03T11:10:13.493Z] at hudson.remoting.UserRequest.perform(UserRequest.java:211)[2022-03-03T11:10:13.493Z] at hudson.rem
KDE CI: Frameworks » kholidays » kf5-qt5 SUSEQt5.15 - Build # 136 - Failure!
BUILD FAILURE
Build URL
https://build.kde.org/job/Frameworks/job/kholidays/job/kf5-qt5%20SUSEQt5.15/136/
Project:
kf5-qt5 SUSEQt5.15
Date of build:
Thu, 03 Mar 2022 11:09:40 +
Build duration:
32 sec and counting
CONSOLE OUTPUT
Started by an SCM changeRunning in Durability level: PERFORMANCE_OPTIMIZED[Pipeline] Start of Pipeline[Pipeline] nodeStill waiting to schedule taskWaiting for next available executor on ‘SUSEQt5.15’Running on Docker Swarm-04a148b81353 in /home/jenkins/workspace/Frameworks/kholidays/kf5-qt5 SUSEQt5.15[Pipeline] {[Pipeline] timestamps[Pipeline] {[Pipeline] catchError[Pipeline] {[Pipeline] stage[Pipeline] { (Checkout Sources)[Pipeline] checkout[2022-03-03T11:10:05.055Z] The recommended git tool is: NONE[2022-03-03T11:10:06.818Z] No credentials specified[2022-03-03T11:10:06.824Z] Cloning the remote Git repository[2022-03-03T11:10:06.856Z] Cloning repository https://invent.kde.org/frameworks/kholidays.git[2022-03-03T11:10:06.933Z] > git init /home/jenkins/workspace/Frameworks/kholidays/kf5-qt5 SUSEQt5.15 # timeout=10[2022-03-03T11:10:06.978Z] Fetching upstream changes from https://invent.kde.org/frameworks/kholidays.git[2022-03-03T11:10:06.978Z] > git --version # timeout=10[2022-03-03T11:10:06.990Z] > git --version # 'git version 2.35.1'[2022-03-03T11:10:06.991Z] > git fetch --tags --force --progress -- https://invent.kde.org/frameworks/kholidays.git +refs/heads/*:refs/remotes/origin/* # timeout=120[2022-03-03T11:10:08.700Z] Avoid second fetch[2022-03-03T11:10:08.730Z] Checking out Revision bbe08c4c7bd569841fcf04d3d727fc97053a1a07 (origin/master)[2022-03-03T11:10:08.683Z] > git config remote.origin.url https://invent.kde.org/frameworks/kholidays.git # timeout=10[2022-03-03T11:10:08.689Z] > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10[2022-03-03T11:10:08.705Z] > git rev-parse origin/master^{commit} # timeout=10[2022-03-03T11:10:08.738Z] > git config core.sparsecheckout # timeout=10[2022-03-03T11:10:08.754Z] > git checkout -f bbe08c4c7bd569841fcf04d3d727fc97053a1a07 # timeout=10[2022-03-03T11:10:12.786Z] Commit message: "Add Qt6 Android CI"[2022-03-03T11:10:12.792Z] > git rev-list --no-walk c584c3ae5aea63d908f066554e63d405937b5919 # timeout=10[Pipeline] checkout[2022-03-03T11:10:12.859Z] The recommended git tool is: NONE[2022-03-03T11:10:12.872Z] No credentials specified[2022-03-03T11:10:12.879Z] Cloning the remote Git repository[2022-03-03T11:10:13.466Z] ERROR: Error cloning remote repo 'origin'[2022-03-03T11:10:13.466Z] hudson.plugins.git.GitException: Command "git fetch --tags --force --progress -- https://invent.kde.org/sysadmin/ci-tooling.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:[2022-03-03T11:10:13.466Z] stdout: [2022-03-03T11:10:13.466Z] stderr: error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500[2022-03-03T11:10:13.466Z] fatal: expected flush after ref listing[2022-03-03T11:10:13.466Z] [2022-03-03T11:10:13.466Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2450)[2022-03-03T11:10:13.466Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2051)[2022-03-03T11:10:13.466Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:84)[2022-03-03T11:10:13.466Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:573)[2022-03-03T11:10:13.466Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:802)[2022-03-03T11:10:13.466Z] at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)[2022-03-03T11:10:13.466Z] at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154)[2022-03-03T11:10:13.466Z] at hudson.remoting.UserRequest.perform(UserRequest.java:211)[2022-03-03T11:10:13.466Z] at hudson.remoting.UserRequest.perform(UserRequest.java:54)[2022-03-03T11:10:13.466Z] at hudson.remoting.Request$2.run(Request.java:375)[2022-03-03T11:10:13.466Z] at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73)[2022-03-03T11:10:13.466Z] at java.util.concurrent.FutureTask.run(FutureTask.java:264)[2022-03-03T11:10:13.466Z] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)[2022-03-03T11:10:13.466Z] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)[2022-03-03T11:10:13.466Z] at java.lang.Thread.run(Thread.java:829)[2022-03-03T11:10:13.467Z] Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to Docker Swarm-04a148b81353[2022-03-03T11:10:13.467Z] at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)[2022-03-03T11:10:13.467Z] at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)[2022-03-
KDE CI: Frameworks » kholidays » kf5-qt5 SUSEQt5.15 - Build # 137 - Fixed!
BUILD SUCCESS Build URL https://build.kde.org/job/Frameworks/job/kholidays/job/kf5-qt5%20SUSEQt5.15/137/ Project: kf5-qt5 SUSEQt5.15 Date of build: Thu, 03 Mar 2022 11:10:14 + Build duration: 2 min 34 sec and counting BUILD ARTIFACTS acc/KF5Holidays-5.92.0.xml JUnit Tests Name: (root) Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s)Name: projectroot Failed: 0 test(s), Passed: 5 test(s), Skipped: 0 test(s), Total: 5 test(s) Cobertura Report Project Coverage Summary Name PackagesFilesClassesLinesConditionalsCobertura Coverage Report80% (4/5)88% (15/17)88% (15/17)68% (1585/2327)49% (774/1594)Coverage Breakdown by Package Name FilesClassesLinesConditionalsautotests100% (5/5)100% (5/5)99% (385/387)57% (125/218)src100% (7/7)100% (7/7)71% (582/824)48% (336/701)src.declarative0% (0/2)0% (0/2)0% (0/28)0% (0/8)src.parsers100% (2/2)100% (2/2)43% (316/727)34% (142/414)src.parsers.plan2100% (1/1)100% (1/1)84% (302/361)68% (171/253)
KDE CI: Frameworks » kio » kf5-qt5 FreeBSDQt5.15 - Build # 1431 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kio/job/kf5-qt5%20FreeBSDQt5.15/1431/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 11:09:30 + Build duration: 5 min 13 sec and counting JUnit Tests Name: projectroot Failed: 4 test(s), Passed: 56 test(s), Skipped: 0 test(s), Total: 60 test(s)Failed: projectroot.autotests.kiocore_jobtestFailed: projectroot.autotests.kiocore_ktcpsockettestFailed: projectroot.autotests.kiowidgets_kdirlistertestFailed: projectroot.autotests.kiowidgets_kdirmodeltestName: projectroot.autotests Failed: 0 test(s), Passed: 6 test(s), Skipped: 0 test(s), Total: 6 test(s)Name: projectroot.src.ioslaves.trash Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s)Name: projectroot.src.kpasswdserver Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s)
KDE CI: Frameworks » kcalendarcore » kf5-qt5 FreeBSDQt5.15 - Build # 195 - Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kcalendarcore/job/kf5-qt5%20FreeBSDQt5.15/195/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 11:10:41 + Build duration: 7 min 4 sec and counting JUnit Tests Name: projectroot Failed: 1 test(s), Passed: 494 test(s), Skipped: 0 test(s), Total: 495 test(s)Failed: projectroot.autotests.testdateserialization
KDE CI: Frameworks » kcalendarcore » kf5-qt5 SUSEQt5.15 - Build # 200 - Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kcalendarcore/job/kf5-qt5%20SUSEQt5.15/200/ Project: kf5-qt5 SUSEQt5.15 Date of build: Thu, 03 Mar 2022 11:10:41 + Build duration: 7 min 1 sec and counting BUILD ARTIFACTS acc/KF5CalendarCore-5.92.0.xml JUnit Tests Name: (root) Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s)Name: projectroot Failed: 1 test(s), Passed: 494 test(s), Skipped: 0 test(s), Total: 495 test(s)Failed: projectroot.autotests.testdateserialization Cobertura Report Project Coverage Summary Name PackagesFilesClassesLinesConditionalsCobertura Coverage Report100% (2/2)94% (88/94)94% (88/94)73% (10721/14587)52% (5661/10988)Coverage Breakdown by Package Name FilesClassesLinesConditionalsautotests100% (41/41)100% (41/41)97% (3729/3855)51% (1836/3604)src89% (47/53)89% (47/53)65% (6992/10732)52% (3825/7384)
KDE CI: Frameworks » kio » kf5-qt5 SUSEQt5.15 - Build # 1415 - Fixed!
BUILD SUCCESS Build URL https://build.kde.org/job/Frameworks/job/kio/job/kf5-qt5%20SUSEQt5.15/1415/ Project: kf5-qt5 SUSEQt5.15 Date of build: Thu, 03 Mar 2022 11:10:14 + Build duration: 11 min and counting BUILD ARTIFACTS acc/KF5KIO-5.92.0.xml JUnit Tests Name: (root) Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s)Name: projectroot Failed: 0 test(s), Passed: 61 test(s), Skipped: 0 test(s), Total: 61 test(s)Name: projectroot.autotests Failed: 0 test(s), Passed: 6 test(s), Skipped: 0 test(s), Total: 6 test(s)Name: projectroot.src.ioslaves.trash Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s)Name: projectroot.src.kpasswdserver Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s) Cobertura Report Project Coverage Summary Name PackagesFilesClassesLinesConditionalsCobertura Coverage Report61% (25/41)69% (298/429)69% (298/429)58% (39323/67995)42% (21770/52050)Coverage Breakdown by Package Name FilesClassesLinesConditionalsautotests95% (61/64)95% (61/64)91% (11408/12474)46% (7159/15602)autotests.http100% (5/5)100% (5/5)99% (527/528)58% (167/290)autotests.kcookiejar100% (1/1)100% (1/1)94% (173/185)63% (70/112)src100% (1/1)100% (1/1)89% (8/9)71% (10/14)src.core88% (107/121)88% (107/121)61% (9367/15288)52% (4629/8876)src.core.kssl100% (1/1)100% (1/1)38% (33/86)50% (2/4)src.filewidgets79% (31/39)79% (31/39)58% (5405/9396)44% (2324/5325)src.gui100% (11/11)100% (11/11)73% (867/1188)58% (458/790)src.gui.systemd50% (2/4)50% (2/4)4% (7/178)1% (1/108)src.ioslaves.file100% (7/7)100% (7/7)56% (785/1400)42% (518/1227)src.ioslaves.file.kauth0% (0/2)0% (0/2)0% (0/187)0% (0/83)src.ioslaves.ftp100% (2/2)100% (2/2)40% (551/1380)30% (435/1430)src.ioslaves.help0% (0/5)0% (0/5)0% (0/253)0% (0/138)src.ioslaves.http88% (7/8)88% (7/8)43% (1873/4354)37% (1359/3719)src.ioslaves.http.kcookiejar40% (2/5)40% (2/5)49% (664/1364)56% (588/1053)src.ioslaves.remote100% (2/2)100% (2/2)25% (57/224)7
KDE CI: Frameworks » kio » kf5-qt5 FreeBSDQt5.15 - Build # 1432 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kio/job/kf5-qt5%20FreeBSDQt5.15/1432/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 11:14:51 + Build duration: 7 min 40 sec and counting JUnit Tests Name: projectroot Failed: 3 test(s), Passed: 57 test(s), Skipped: 0 test(s), Total: 60 test(s)Failed: projectroot.autotests.kiocore_ktcpsockettestFailed: projectroot.autotests.kiofilewidgets_kfilewidgettestFailed: projectroot.autotests.kiowidgets_kdirlistertestName: projectroot.autotests Failed: 0 test(s), Passed: 6 test(s), Skipped: 0 test(s), Total: 6 test(s)Name: projectroot.src.ioslaves.trash Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s)Name: projectroot.src.kpasswdserver Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s)
KDE CI: Frameworks » kcalendarcore » kf5-qt5 FreeBSDQt5.15 - Build # 196 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kcalendarcore/job/kf5-qt5%20FreeBSDQt5.15/196/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 11:18:02 + Build duration: 6 min 7 sec and counting JUnit Tests Name: projectroot Failed: 1 test(s), Passed: 494 test(s), Skipped: 0 test(s), Total: 495 test(s)Failed: projectroot.autotests.testdateserialization
KDE CI: Frameworks » kcalendarcore » kf5-qt5 WindowsMSVCQt5.15 - Build # 184 - Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kcalendarcore/job/kf5-qt5%20WindowsMSVCQt5.15/184/ Project: kf5-qt5 WindowsMSVCQt5.15 Date of build: Thu, 03 Mar 2022 11:10:41 + Build duration: 34 min and counting JUnit Tests Name: projectroot Failed: 1 test(s), Passed: 493 test(s), Skipped: 0 test(s), Total: 494 test(s)Failed: projectroot.autotests.testdateserialization
Re: Critical Denial of Service bugs in Discover
I'd say wireshark is too low level for what the problem is here. We are talking about having too many HTTP requests for specific URLs. I can think two main measures: - Trigger an alarm (an e-mail notification?) if there's a specific UserAgent that has a specific portion of the queries we have in a specific day in the services we care about. - Offer plots to see how queries by UserAgent evolve over the last couple of months (or couple of years). Aleix On Thu, Mar 3, 2022 at 9:59 AM Ben Cooksley wrote: > On Thu, Mar 3, 2022 at 8:41 AM Aleix Pol wrote: > >> (dropping the distros list) >> >> @sysadmin have you been able to look into any tools we devs can have to >> make sure this situation doesn't repeat in the future? >> > > Hi Aleix, > > To be honest i've been struggling to think of ways that we could detect > this on the server side prior to it becoming a massive issue. > By the time an issue is evident server side it is usually much too late. > > The main tools i'd usually recommend would be the standard tools you would > use for monitoring the network activity of any application - such as > Wireshark. > > Is there something you were thinking of specifically in terms of us being > able to provide? > > Thanks, > Ben > > >> >> Aleix >> >> On Thu, Feb 10, 2022 at 1:10 PM Aleix Pol wrote: >> >>> On Thu, Feb 10, 2022 at 11:05 AM Ben Cooksley wrote: >>> > >>> > >>> > >>> > On Thu, Feb 10, 2022 at 8:20 AM Aleix Pol wrote: >>> >> >>> >> [Snip] >>> >> >>> >> We still haven't discussed here is how to prevent this problem from >>> >> happening again. >>> >> >>> >> If we don't have information about what is happening, we cannot fix >>> problems. >>> > >>> > >>> > Part of the issue here is that the problem only came to Sysadmin >>> attention very recently, when the system ran out of disk space as a result >>> of growing log files. >>> > It was at that point we realised we had a serious problem. >>> > >>> > Prior to that the system load hadn't climbed to dangerous levels (> >>> number of CPU cores) and Apache was keeping up with the traffic, so none of >>> our other monitoring was tripped. >>> > >>> > If you have any thoughts on what sort of information you are thinking >>> of that would be helpful. >>> >>> We could have plots of the amount of queries we get with a KNewStuff/* >>> user-agent over time and their distribution. >>> >>> > It would definitely be helpful though to know when new software is >>> going to be released that will be interacting with the servers as we will >>> then be able to monitor for abnormalities. >>> >>> We make big announcements of every Plasma release... (?) >>> >>> >> Is there anything that could be done in this front? The issue here >>> >> could have been addressed months ago, we just never knew it was >>> >> happening. >>> > >>> > >>> > One possibility that did occur to me today would be for us to >>> integrate some kind of killswitch that our applications would check on >>> first initialisation of functionality that talks to KDE.org servers. >>> > This would allow us to disable the functionality in question on user >>> systems. >>> > >>> > The check would only be done on first initialization to keep load low, >>> while still ensuring all users eventually are affected by the killswitch >>> (as they will eventually need to logout/reboot for some reason or another). >>> > >>> > The killswitch would probably work best if it had some kind of version >>> check in it so we could specify which versions are disabled. >>> > That would allow for subsequent updates - once delivered by >>> distributions - to restore the functionality (while leaving it disabled for >>> those who haven't updated). >>> >>> The file we are serving here effectively is the kill switch to all of >>> KNewStuff. >>> >>> Aleix >>> >>
KDE CI: Frameworks » kwindowsystem » kf5-qt5 FreeBSDQt5.15 - Build # 168 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kwindowsystem/job/kf5-qt5%20FreeBSDQt5.15/168/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 17:07:31 + Build duration: 2 min 22 sec and counting JUnit Tests Name: projectroot Failed: 1 test(s), Passed: 12 test(s), Skipped: 0 test(s), Total: 13 test(s)Failed: projectroot.autotests.kwindowsystem_kwindowsystemplatformwaylandtest
KDE CI: Frameworks » kcrash » kf5-qt5 FreeBSDQt5.15 - Build # 123 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kcrash/job/kf5-qt5%20FreeBSDQt5.15/123/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 17:09:51 + Build duration: 31 sec and counting JUnit Tests Name: projectroot Failed: 1 test(s), Passed: 2 test(s), Skipped: 0 test(s), Total: 3 test(s)Failed: projectroot.autotests.metadatatest
KDE CI: Frameworks » kcrash » kf5-qt5 FreeBSDQt5.15 - Build # 124 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kcrash/job/kf5-qt5%20FreeBSDQt5.15/124/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 17:10:23 + Build duration: 32 sec and counting JUnit Tests Name: projectroot Failed: 1 test(s), Passed: 2 test(s), Skipped: 0 test(s), Total: 3 test(s)Failed: projectroot.autotests.metadatatest
KDE CI: Frameworks » kwindowsystem » kf5-qt5 FreeBSDQt5.15 - Build # 169 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kwindowsystem/job/kf5-qt5%20FreeBSDQt5.15/169/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 17:09:55 + Build duration: 1 min 15 sec and counting JUnit Tests Name: projectroot Failed: 1 test(s), Passed: 12 test(s), Skipped: 0 test(s), Total: 13 test(s)Failed: projectroot.autotests.kwindowsystem_kwindowsystemplatformwaylandtest
KDE CI: Frameworks » kirigami » kf5-qt5 FreeBSDQt5.15 - Build # 790 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kirigami/job/kf5-qt5%20FreeBSDQt5.15/790/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 21:07:13 + Build duration: 1 min 10 sec and counting JUnit Tests Name: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515 Failed: 16 test(s), Passed: 0 test(s), Skipped: 0 test(s), Total: 16 test(s)Failed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.pagepool/tst_layers.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.pagepool/tst_pagepool.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_actiontoolbar.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_avatar.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_icon.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_keynavigation.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_listskeynavigation.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_mnemonicdata.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_pagerouter.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_pagerow.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_routerwindow.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_theme.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_filterMouseEvents.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_invokables.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_onWheel.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_scrolling.qml
KDE CI: Frameworks » kirigami » kf5-qt5 WindowsMSVCQt5.15 - Build # 688 - Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kirigami/job/kf5-qt5%20WindowsMSVCQt5.15/688/ Project: kf5-qt5 WindowsMSVCQt5.15 Date of build: Thu, 03 Mar 2022 21:07:13 + Build duration: 1 min 49 sec and counting JUnit Tests Name: projectrootC_.CI.Job_Build Failed: 1 test(s), Passed: 15 test(s), Skipped: 0 test(s), Total: 16 test(s)Failed: projectrootC_.CI.Job_Build.autotests.pagepool/tst_layers.qml
KDE CI: Frameworks » kirigami » kf5-qt5 FreeBSDQt5.15 - Build # 791 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kirigami/job/kf5-qt5%20FreeBSDQt5.15/791/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 21:28:33 + Build duration: 42 sec and counting JUnit Tests Name: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515 Failed: 16 test(s), Passed: 0 test(s), Skipped: 0 test(s), Total: 16 test(s)Failed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.pagepool/tst_layers.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.pagepool/tst_pagepool.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_actiontoolbar.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_avatar.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_icon.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_keynavigation.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_listskeynavigation.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_mnemonicdata.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_pagerouter.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_pagerow.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_routerwindow.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_theme.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_filterMouseEvents.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_invokables.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_onWheel.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_scrolling.qml
KDE CI: Frameworks » kcalendarcore » kf5-qt5 FreeBSDQt5.15 - Build # 197 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kcalendarcore/job/kf5-qt5%20FreeBSDQt5.15/197/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 21:51:28 + Build duration: 1 min 50 sec and counting JUnit Tests Name: projectroot Failed: 1 test(s), Passed: 494 test(s), Skipped: 0 test(s), Total: 495 test(s)Failed: projectroot.autotests.testdateserialization
KDE CI: Frameworks » kcalendarcore » kf5-qt5 SUSEQt5.15 - Build # 201 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kcalendarcore/job/kf5-qt5%20SUSEQt5.15/201/ Project: kf5-qt5 SUSEQt5.15 Date of build: Thu, 03 Mar 2022 21:51:28 + Build duration: 3 min 0 sec and counting BUILD ARTIFACTS acc/KF5CalendarCore-5.92.0.xml JUnit Tests Name: (root) Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s)Name: projectroot Failed: 1 test(s), Passed: 494 test(s), Skipped: 0 test(s), Total: 495 test(s)Failed: projectroot.autotests.testdateserialization Cobertura Report Project Coverage Summary Name PackagesFilesClassesLinesConditionalsCobertura Coverage Report100% (2/2)94% (88/94)94% (88/94)74% (10764/14630)52% (5709/11052)Coverage Breakdown by Package Name FilesClassesLinesConditionalsautotests100% (41/41)100% (41/41)97% (3770/3896)51% (1853/3638)src89% (47/53)89% (47/53)65% (6994/10734)52% (3856/7414)
KDE CI: Frameworks » kirigami » kf5-qt5 FreeBSDQt5.15 - Build # 792 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kirigami/job/kf5-qt5%20FreeBSDQt5.15/792/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 22:57:53 + Build duration: 3 min 36 sec and counting JUnit Tests Name: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515 Failed: 16 test(s), Passed: 0 test(s), Skipped: 0 test(s), Total: 16 test(s)Failed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.pagepool/tst_layers.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.pagepool/tst_pagepool.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_actiontoolbar.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_avatar.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_icon.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_keynavigation.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_listskeynavigation.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_mnemonicdata.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_pagerouter.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_pagerow.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_routerwindow.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_theme.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_filterMouseEvents.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_invokables.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_onWheel.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_scrolling.qml
