BOTS! The idea that Russia committed an act of automated cyber warfare in critical elections has spread thanks to the work of researchers from credible institutions. However, despite the major universities that stand behind the researchers, we, as data journalists, see their work as seriously flawed.

Evidence clearly shows that Russia used the Internet and social media in an effort to manipulate the 2016 US elections and the outcome of Brexit. We are not denying this.

We just don’t believe it was the work of bots.

A look into the data shows why.

The evidence

When we searched for press articles with the terms »social bots US election« and »social bots brexit«, we found that from the 48 top Google news results, 45 refer to the work of three research groups:

ABS-CBNBBCBuzzfeedCBCCNETEngadgetFinancial TimesIEEE SPECTRUMIndependentNPRNewScientistThe AtlanticThe TelegraphThe New York TimesThe Washington PostForbesMOTHERBOARDPHYS.ORGWIREDCNBCFast CompanyMIT Technology ReviewPew Research CenterThe ConversationThe SunUS NewsThe GuardianBloombergBusinessinsiderMashableTechCrunchThe HillThe Sydney Morning HeraldThe TimesSocial media 'bots' from Russia distorting global politics: studyTrump's Twitter debate lead was 'swelled by bots'Pro-Clinton bots 'fought back but outnumbered in second debate'Bots used to bias online political chats'Researchers Are Upset That Twitter Is Dismissing Their Work On Election InterferencePolitical bots spread misinformation during U.S. campaignBots and trolls are swaying opinions on Facebook, TwitterPropaganda bots dominate social networks in some countriesSocial Bots of BrexitIs social media empowering Dutch populism?How Political Campaigns Weaponize Social Media BotsBrexit: Pro-Leave Twitter bots played ‘strategic role’ in EU referendum result, says Oxford University studyBeware the Brexit bots: The Twitter spam out to swing your voteDid Social Media Ruin Election 2016?Political bots are poisoning democracy – so, off with their headsHow Twitter Bots Are Shaping the ElectionFacebook and Twitter are being used to manipulate public opinion – reportAutomated Pro-Trump Bots Overwhelmed Pro-Clinton Messages, Researchers SayOn Twitter, a Battle Among Political BotsExclusive: Labour election campaign boosted by fake Twitter accountsPropaganda flowed heavily into battleground states around election, study saysThe Political Twitter Bots Will Rage This Election DaySenator Slams Twitter's Disclosure On Russian Meddling As 'Inadequate On Every Level'Election 2016 Belongs to the Twitter BotsSoftware 'bots' distort Trump support on Twitter: studyHow Twitter Is Being Gamed to Feed Misinformation‘Something fishy’ is going on with Trump’s Twitter account, researchers sayAs many as 48 million Twitter accounts aren't people, says studyOn Twitter, No One Knows You’re A Trump BotHow the Bot-y Politic Influenced This ElectionFirst Evidence That Social Bots Play a Major Role in Spreading Fake NewsBots in the TwittersphereFake Tweets, real consequences for the electionHow Twitter bots affected the US presidential campaignBOTS 'SWAY' ELECTION - Fake tweets by social media robots could swing US Presidential electionTwitter Bots Favored Trump Leading Up to ElectionHow Twitter bots played a role in electing Donald TrumpTwitter Bots Helped Trump and Brexit Win, Economic Study SaysSuspected Russian Twitter accounts spread Islamophobia and Sharia law fears before the Brexit voteRussia used a network of 150,000 Twitter accounts to meddle in BrexitRussian bots dropped 45,000 garbage tweets on us during the Brexit referendumStudy: Russian Twitter bots sent 45k Brexit tweets close to voteRussian bid to influence Brexit vote detailed in new US Senate reportTwitter bots may have affected voters on Brexit, U.S. presidential race: studyTwitter bots helped Trump and Brexit win, economic study saysRussia used Twitter bots and trolls ‘to disrupt’ Brexit voteWhere does the evidence for countless social bots come from?OxfordBerkeley/SwanseaClick the media linksand check the articlesINFOGRAPHICS GROUP • www.info.graphics • CC BYSouthern California/IndianaNumber of times the article was a reference in the media
Where does the evidence for countless social bots come from?15 timesSouthern California/Indiana9 timesBerkeley/Swansea27 timesOxford "The Computational Propaganda Project of Oxford University"INFOGRAPHICS GROUP • www.info.graphics • CC BY Click the media linksand check the articlesNumber of times the article was a reference in the mediaABS-CBNBBCBuzzfeedCBCCNETEngadgetFinancial TimesIEEE SPECTRUMIndependentNPRNewScientistThe AtlanticThe TelegraphThe New York TimesThe Washington PostForbesMOTHERBOARDPHYS.ORGWIREDCNBCFast CompanyMIT Technology ReviewPew Research CenterThe ConversationThe SunUS NewsThe GuardianBloombergBusinessinsiderMashableTechCrunchThe HillThe Sydney Morning HeraldThe TimesSocial media 'bots' from Russia distorting global politics: studyTrump's Twitter debate lead was 'swelled by bots'Pro-Clinton bots 'fought back but outnumbered in second debate'Bots used to bias online political chats'Researchers Are Upset That Twitter Is Dismissing Their Work On Election InterferencePolitical bots spread misinformation during U.S. campaignBots and trolls are swaying opinions on Facebook, TwitterPropaganda bots dominate social networks in some countriesSocial Bots of BrexitIs social media empowering Dutch populism?How Political Campaigns Weaponize Social Media BotsBrexit: Pro-Leave Twitter bots played ‘strategic role’ in EU referendum result, says Oxford University studyBeware the Brexit bots: The Twitter spam out to swing your voteDid Social Media Ruin Election 2016?Political bots are poisoning democracy – so, off with their headsHow Twitter Bots Are Shaping the ElectionFacebook and Twitter are being used to manipulate public opinion – reportAutomated Pro-Trump Bots Overwhelmed Pro-Clinton Messages, Researchers SayOn Twitter, a Battle Among Political BotsExclusive: Labour election campaign boosted by fake Twitter accountsPropaganda flowed heavily into battleground states around election, study saysThe Political Twitter Bots Will Rage This Election DaySenator Slams Twitter's Disclosure On Russian Meddling As 'Inadequate On Every Level'Election 2016 Belongs to the Twitter BotsSoftware 'bots' distort Trump support on Twitter: studyHow Twitter Is Being Gamed to Feed Misinformation‘Something fishy’ is going on with Trump’s Twitter account, researchers sayAs many as 48 million Twitter accounts aren't people, says studyOn Twitter, No One Knows You’re A Trump BotHow the Bot-y Politic Influenced This ElectionFirst Evidence That Social Bots Play a Major Role in Spreading Fake NewsBots in the TwittersphereFake Tweets, real consequences for the electionHow Twitter bots affected the US presidential campaignBOTS 'SWAY' ELECTION - Fake tweets by social media robots could swing US Presidential electionTwitter Bots Favored Trump Leading Up to ElectionHow Twitter bots played a role in electing Donald TrumpTwitter Bots Helped Trump and Brexit Win, Economic Study SaysSuspected Russian Twitter accounts spread Islamophobia and Sharia law fears before the Brexit voteRussia used a network of 150,000 Twitter accounts to meddle in BrexitRussian bots dropped 45,000 garbage tweets on us during the Brexit referendumStudy: Russian Twitter bots sent 45k Brexit tweets close to voteRussian bid to influence Brexit vote detailed in new US Senate reportTwitter bots may have affected voters on Brexit, U.S. presidential race: studyTwitter bots helped Trump and Brexit win, economic study saysRussia used Twitter bots and trolls ‘to disrupt’ Brexit vote

These are widely recognized and respected institutions, but the devil is in the details: Which definition did the researchers use to determine what qualifies an account as a social bot? How did the researchers find social bots? And how did they prove that those accounts were in fact bots?

Let’s dig in.

Oxford

In the press, »Bots and Automation over Twitter during the U.S. Election« was by far the most quoted publication. This study looked at hashtags typically associated with the US election (#maga, #imwithher, etc.) to categorize political tweets.

But one important scientific element is missing from the study: It was only published on the project’s website. Such articles are usually published in official academic journals and have to first be peer reviewed, which requires a review of the method and results by other scientists.

We assume that this study was never published in a journal because it would not have passed the peer review due to its definition of bot behavior:

We define a high level of automation as accounts that post at least 50 times a day [...].

(emphasis by us)

In short, according to this research team, any account that tweets more than 50 times a day is a bot. This assumption was never proven.

Scientific research should be reproducible – so the most logical step for us was to reproduce what the Oxford team did. We took a list of accounts that are, in a verified manner, not bots: accounts that are marked as »verified« by Twitter itself.

We scraped the tweet frequency of 300,000 verified accounts. What we found? Many accounts that are verified by Twitter would be social bots according to this research. (You can find our raw data here.)

4,475 of these accounts tweeted more than 50 times a day. Though a small percentage – 1.48% – this is absolutely and proportionately more than Oxford found. Following their method, we found 4.1 million tweets that would match the criteria of »bot behavior«.

So, which accounts would be defined as »social bots« according to the Oxford method? The top 10 accounts with the most followers are:

English-language accountsGerman-language accounts
  1. nytimes (108.3)
  2. CNN (156.6)
  3. NBA (115)
  4. NFL (81.1)
  5. TheEconomist (92.2)
  6. Reuters (318.4)
  7. WSJ (93.3)
  8. TIME (78.9)
  9. MTV (51.7)
  10. Forbes (81.6)
  1. zeitonline (65.0)
  2. BILD (106.8)
  3. SZ (59.2)
  4. welt (186.0)
  5. sternde (75.0)
  6. sportschau (53.3)
  7. ntvde (74.2)
  8. faznet (65.6)
  9. WELTnews (81.7)
  10. Tagesspiegel (100.4)
Number of tweets per day are in parentheses.
You can find the Excel sheet here

The most influential of those false positive social bots were media outlets – who of course shared their reporting on Twitter. Other notable »social bot« accounts include Starbucks (151.7), McDonalds (127.1), AskPlayStation (275.6), British Airways (280.8), journalist Glenn Greenwald (50.2) and author Cory Doctorow (142.2).

Additionally, Oxford did not publish their data, making it impossible to verify their research now.

Southern California/Indiana

Professor Ferrara and his team used machine learning to create a tool that was first called »BotorNot« and has since been renamed »Botometer«.

To train the algorithm, they collected data from 15,000 known bot accounts and 16,000 human accounts, including creation date, amount of tweets per hour, emoticon usage, etc. The analysis used 1,150 features per account that were fed into the machine learning algorithm in order to distinguish between these two groups. According to their own statements, »Botometer« has a 95% success rate at correctly identifying whether an account belongs to the group »people« or the group »bots«.

Whether the system works well outside of the test group was either not tested or not disclosed. Unfortunately the research team published neither their algorithm nor their data so this cannot be verified. However, they created a website that allows anyone to use their tool.

In April 2018, we put the list of US Congress members through the »Botometer« to test its reliability:

INFOGRAPHICS GROUPSource: Botometer®, April 2018US Congress on Twitter: Bots or not?Botometer, with a claim of a 95% success rate, classified about half of US Congress members on Twitter as Bots.@RoyBluntwas rated most bot-like@SenatorWicker and@SusanWBrookswere rated leastbot-like70%60%30%40%50%@DanaRohrabacher@RepBost@RepMoolenaar@SenStabenow@HASCRepublicans@RepCarbajal@RepRickLarsen@chelliepingree@JacksonLeeTX18@RepGutierrez@RepSeanMaloney@repjoecrowley@SenatorWicker@SusanWBrooks@HouseBudgetDems@RepRWilliams@HouseVetAffairs@JaredHuffman@LacyClayMO1@InhofePress@BennieGThompson@JoaquinCastrotx@RandPaul@MikeCrapo@BettyMcCollum04@RepGeneGreen@RepBarragan@RepDannyDavis@EnergyDems@RepJoseSerrano@RepBillFlores@RepJeffDuncan@PatrickMcHenry@RepMcSally@RepBrendanBoyle@RepTedBudd@RepAlLawsonJr@RepNewhouse@RepBrianFitz@RepValDemings@RepBonnie@RepSmucker@RepBuddyCarter@dinatitus@RepFredUpton@RepSteveChabot@RepEBJ@Jim_Jordan@RepHalRogers@SenBennetCO@RepHensarling@RepBradWenstrup@RepKevinYoder@USRepLong@RepJohnYarmuth@RepJimRenacci@RepMarkMeadows@justinamash@RepRutherfordFL@RepMullin@RepScottPeters@JECDems@SenSanders@SenCoryGardner@JECRepublicans@NitaLowey@repjimcooper@senorrinhatch@EnergyCommerce@RepMaloney@RepMaxineWaters@RepStephMurphy@GerryConnolly@RepMattGaetz@RepRalphNorman@RodneyDavis@KYComer@RepMcCaul@RepSeanDuffy@SASCMajority@KenCalvert@RepMikeJohnson@RepTipton@SenatorCardin@McConnellPress@RepMikeTurner@SenBlumenthal@WaysandMeansGOP@RepBrady@RepWalorski@SenJohnThune@rep_stevewomack@RepBrianHiggins@RobinLynneKelly@clairecmc@ChrisVanHollen@RepDavid@SenatorLankford@sendavidperdue@CongBoyle@RepEsty@gracenapolitano@BradSherman@HouseAdmnGOP@RepJoeWilson@BillCassidy@Call_Me_Dutch@RepCartwright@RepLindaSanchez@CongPalazzo@DonaldNorcross@RepClayHiggins@RepLukeMesser@HurdOnTheHill@EdWorkforce@RepDrewFerguson@SenDougJones@KeithRothfus@HASCDemocrats@RepEdRoyce@SenPatRoberts@MarioDB@MacTXPress@RepEliotEngel@SenateBudget@RepAbraham@RepAndyBiggsAZ@RepGraceMeng@SenatorHassan@RepCardenas@RepAndyHarrisMD@RepHartzler@ToddRokita@RepChrisCollins@RepDennyHeck@RepMcGovern@repdinatitus@RepDonBacon@RepGarretGraves@RepMcKinley@RepAlGreen@RepGaramendi@RepJohnKatko@RepRatcliffe@RepAndyBarr@RepJoeBarton@RepKevinBrady@RepSwalwell@RepBeatty@RepJohnDuncanJr@RepLBR@RepThomasMassie@RepBetoORourke@RepKHandel@RepLouBarletta@SenWhitehouse@RepBonamici@RepKihuen@RepMGriffith@SenatorCarper@RepEspaillat@RepLaHood@RepRickAllen@SenatorFischer@RepGrothman@RepMarkWalker@RepRooney@jahimes@RepJenniffer@CongressmanRaja@RepChuck@RepKayGranger@RepWalberg@EleanorNorton@RepFilemonVela@RepLloydDoggett@RepWilson@GKButterfield@RepGallagher@RepLouCorrea@SenBobCasey@GOPLeader@RepGregoryMeeks@RepMarcyKaptur@SenGillibrand@GovRonDeSantis@RepJeffDenham@RepPittenger@SenThomTillis@GreggHarper@RepJimCosta@RepRichardNeal@SpeakerRyan@HouseJudDems@RepJoeKennedy@RepSarbanes@TomColeOK04@JudgeTedPoe@RepKClark@RepTomReed@boblatta@LindseyGrahamSC@RepKevinCramer@RonWyden@nikiinthehouse@MikeKellyPA@RepMikeBishop@SciCmteDems@timkaine@NydiaVelazquez@RepSanfordSC@SenatorTimScott@AustinScottGA08@OversightDems@RepShimkus@SteveKingIA@RepAnnieKuster@PeteSessions@RepStephenLynch@USRepKCastor@RepArrington@Raul_Labrador@SenAlexander@janschakowsky@RepBillShuster@RepDelBene@SenJohnHoeven@repblumenauer@RepBrianMast@RepDennisRoss@SenSherrodBrown@BillPascrell@RepCheri@RepDonaldPayne@SenatorMenendez@ConawayTX11@RepDavidRouzer@RepJerryNadler@SenatorRounds@HFACDemocrats@RepDougCollins@RepJimmyGomez@SteveKnight25@HouseCommerce@RepGosar@RepRaskin@TGowdySC@JeffFlake@RepGusBilirakis@RepRichHudson@maziehirono@NancyPelosi@RepGuthrie@RepScottPerry@repcleaver@PeterWelch@RepGwenMoore@RepZoeLofgren@repdavidscott@RepBlaine@RepHanabusa@SenFeinstein@repjohnlewis@RepChrisStewart@RepJohnFaso@SenJohnKennedy@CongressmanGT@RepDaveBrat@RepJohnLarson@SenJohnMcCain@DesJarlaisTN04@RepDeSaulnier@RepJoshG@SenJoniErnst@EnergyGOP@RepDebDingell@RepMikeCoffman@Sen_JoeManchin@HouseHomeland@RepDevinNunes@RepRaulGrijalva@SenatorBurr@JerryMoran@RepJuanVargas@RepRickCrawford@WarrenDavidson@MartinHeinrich@RepLawrence@Rep_Hunter@AppropsDems@RepAdams@RepLeeZeldin@SamsPressShop@ChrisMurphyCT@RepAmata@RepLipinski@SenCortezMasto@CongressmanRuiz@RepBobGibbs@RepMimiWalters@SenJeffMerkley@NormaJTorres@RepJudyChu@RepSheaPorter@stabenow@CongressmanHice@RepAdrianSmith@RepLoBiondo@SenKamalaHarris@JimInhofe@RepCuellar@RepMarciaFudge@SenatorIsakson@MarkWarner@RepDLamborn@RepPaulCook@SteveDaines@RepAdamSchiff@RepDWStweets@RepThompson@USRepMikeDoyle@RepBobbyRush@RepJayapal@RepTomGarrett@USRepRodney@RepCharlieDent@RepJohnDelaney@RepTomGraves@jaredpolis@RepDavidKustoff@RepKinzinger@RosLehtinen@Clyburn@RepGonzalez@RepLanceNJ7@SanfordBishop@JimLangevin@RepHuffman@RepMoBrooks@SenBillNelson@RepAdamSmith@RepJackyRosen@RepSinema@SenBobCorker@RepAlexMooney@RepJaredPolis@RepTimRyan@SenCoonsOffice@RepAndreCarson@RepJeffries@RepTomEmmer@SenRonJohnson@RepAnnWagner@RepLaMalfa@RepTomMarino@StaceyPlaskett@RepComstock@RepMcEachin@RepWalterJones@brianschatz@RepCummings@RepPaulMitchell@Robert_Aderholt@repmarkpocan@RepDarrenSoto@RepRichmond@SenMikeLee@DWStweets@RepDavidValadao@RepRonKind@SenatorDurbin@IndianCommittee@RepDianaDeGette@RepStefanik@TomRooney@MarkAmodeiNV2@RepDwightEvans@RepSusanDavis@Transport@RepChrisSmith@RepHultgren@RepTomRice@keithellison@RepDaveTrott@RepLoisFrankel@RoKhanna@rosadelauro@RepDerekKilmer@RepLoudermilk@RobWittman@senatemajldr@RepKathleenRice@RepLynnJenkins@SenDonnelly@FinancialCmte@RepLujanGrisham@RepMarthaRoby@SenWarren@JimPressOffice@RepMarkTakano@RepPaulTonko@SenatorHeitkamp@JudgeCarter@RepRodBlum@RepPeteKing@SenatorTester@LamarSmithTX21@RepRonEstes@RepPeteOlson@SenatorTomUdall@PeterRoskam@RepTedYoho@RepTenney@TXRandy14@RepBillJohnson@RepTimWalz@RepTomMacArthur@USRepKeating@RepByrne@RepWesterman@SenDanSullivan@daveloebsack@RepDanKildee@RepYvetteClarke@SenToomey@replouiegohmert@RepGoodlatte@SenAngusKing@lisamurkowski@DorisMatsui@RepHuizenga@SenJackReed@repgregwalden@JohnBoozman@CoryBooker@SenateBanking@marcorubio@RepSires@FrankPallone@SenatorLeahy@DrNealDunnFL2@RepTrey@HouseAgNews@SenatorRisch@DrPhilRoe@RulesReps@JuliaBrownley26@WaysMeansCmte@RepBarbaraLee@SenShelby@RepAnnaEshoo@congbillposey@RepJimBanks@SenatorCantwell@RepDaveJoyce@davidcicilline@RepMcNerney@SenatorCollins@RepDavidEPrice@repdonyoung@RepMikeQuigley@USRepGaryPalmer@RepDavidYoung@RepBrianBabin@RepOHalleran@RepBera@RepPoliquin@RepCurbelo@RepRoybalAllard@RepCohen@RepSchrader@RepHankJohnson@SenGaryPeters@RepJohnConyers@RepTomSuozzi@RepHastingsFL@aguilarpete@RepSteveStivers@RepTrentKelly@RepHolding@repbenraylujan@RepTedDeutch@SenCapito@RepKristiNoem@HouseScience@RepVisclosky@SenDuckworth@RepLarryBucshon@PatTiberi@USRepRickNolan@SenJohnBarrasso@RepMcClintock@RepDianeBlack@VernBuchanan@SenMarkey@RepMiaLove@RepFrenchHill@louiseslaughter@SenSchumer@RepPeterDeFazio@RepJimmyPanetta@repsandylevin@SenToddYoung@SteveScalise@RepScottTaylor@virginiafoxx@SenTomCotton@cathymcmorris@CongMikeSimpson@RepRobWoodall@RepMikeRogersAL@HerreraBeutler@GrahamBlog@RepTedLieu@RepWebster@RepKarenBass@HouseForeign@RepVeasey@SenTinaSmith@RepSchneider@JohnCornyn@SenateAgDems@WhipHoyer@senrobportman@PattyMurray@SenatorShaheen@BobbyScott@JeffFortenberry@RepErikPaulsen@RepDonBeyer@ChrisCoons@RepJasonLewis@RepFrankLucas@RepJBridenstine@RepKenBuck@SenDeanHeller@RepRoKhanna@RepLizCheney@michaelcburgess@RepCharlieCrist@RepRussell@RepKenMarchant@MarshaBlackburn@RepPerlmutter@RepTerriSewell@SenateGOP@RepRobinKelly@RepRubenGallego@sethmoulton@SenatorEnzi@SenatorBaldwin@SenSasse@RepBillFoster@davereichert@TulsiPress@RepPeteAguilar@RepJoeCourtney@DarrellIssa@amyklobuchar@RepSpeier@johnculberson@RepLowenthal@ChuckGrassley@SenTedCruz@RepJasonSmith@RoyBluntACTS MORE LIKE A HUMANACTS MORE LIKE A BOT • www.info.graphics • CC BY
Percent account is likely a botINFOGRAPHICS GROUP • www.info.graphics • CC BYSource: Botometer®, April 2018Botometer, with a claim of a 95% success rate,classified about half of US Congress members on Twitter as Bots.@RoyBlunt wasrated mostbot-like@SenatorWicker and@SusanWBrookswere rated least bot-likeACTS MORE LIKE A BOTACTS MORELIKE A HUMANUS Congress on Twitter: Bots or not?@SenatorWicker@SusanWBrooks@LacyClayMO1@RepRWilliams@HouseVetAffairs@JaredHuffman@HouseBudgetDems@senorrinhatch@JECRepublicans@NitaLowey@repjimcooper@SenCoryGardner@RepScottPeters@JECDems@SenSanders@RepMullin@RepMarkMeadows@justinamash@RepRutherfordFL@RepJimRenacci@RepKevinYoder@USRepLong@RepJohnYarmuth@RepBradWenstrup@RepHalRogers@SenBennetCO@RepHensarling@Jim_Jordan@RepFredUpton@RepSteveChabot@RepEBJ@dinatitus@RepBonnie@RepSmucker@RepBuddyCarter@RepValDemings@RepAlLawsonJr@RepNewhouse@RepBrianFitz@RepTedBudd@PatrickMcHenry@RepMcSally@RepBrendanBoyle@RepJeffDuncan@EnergyDems@RepJoseSerrano@RepBillFlores@RepDannyDavis@BettyMcCollum04@RepGeneGreen@RepBarragan@MikeCrapo@BennieGThompson@JoaquinCastrotx@RandPaul@InhofePress@RepJenniffer@RepMarkWalker@RepRooney@jahimes@RepGrothman@RepLaHood@RepRickAllen@SenatorFischer@RepEspaillat@RepKihuen@RepMGriffith@SenatorCarper@RepBonamici@RepKHandel@RepLouBarletta@SenWhitehouse@RepBetoORourke@RepJohnDuncanJr@RepLBR@RepThomasMassie@RepBeatty@RepJoeBarton@RepKevinBrady@RepSwalwell@RepAndyBarr@RepGaramendi@RepJohnKatko@RepRatcliffe@RepAlGreen@RepDonBacon@RepGarretGraves@RepMcKinley@repdinatitus@RepChrisCollins@RepDennyHeck@RepMcGovern@ToddRokita@RepCardenas@RepAndyHarrisMD@RepHartzler@SenatorHassan@RepAbraham@RepAndyBiggsAZ@RepGraceMeng@SenateBudget@MarioDB@MacTXPress@RepEliotEngel@SenPatRoberts@KeithRothfus@HASCDemocrats@RepEdRoyce@SenDougJones@HurdOnTheHill@EdWorkforce@RepDrewFerguson@RepLukeMesser@CongPalazzo@DonaldNorcross@RepClayHiggins@RepLindaSanchez@BillCassidy@Call_Me_Dutch@RepCartwright@RepJoeWilson@gracenapolitano@BradSherman@HouseAdmnGOP@RepEsty@SenatorLankford@sendavidperdue@CongBoyle@RepDavid@RobinLynneKelly@clairecmc@ChrisVanHollen@RepBrianHiggins@RepWalorski@SenJohnThune@rep_stevewomack@RepBrady@RepMikeTurner@SenBlumenthal@WaysandMeansGOP@McConnellPress@RepMikeJohnson@RepTipton@SenatorCardin@KenCalvert@RepMcCaul@RepSeanDuffy@SASCMajority@KYComer@RepMattGaetz@RepRalphNorman@RodneyDavis@GerryConnolly@RepMaloney@RepMaxineWaters@RepStephMurphy@EnergyCommerce@NormaJTorres@RepJudyChu@RepSheaPorter@stabenow@JacksonLeeTX18@RepGutierrez@RepSeanMaloney@repjoecrowley@HASCRepublicans@RepCarbajal@RepRickLarsen@chelliepingree@DanaRohrabacher@RepBost@RepMoolenaar@SenStabenow@CongressmanRuiz@RepBobGibbs@RepMimiWalters@SenJeffMerkley@ChrisMurphyCT@RepAmata@RepLipinski@SenCortezMasto@AppropsDems@RepAdams@RepLeeZeldin@SamsPressShop@WarrenDavidson@MartinHeinrich@RepLawrence@Rep_Hunter@SenatorBurr@JerryMoran@RepJuanVargas@RepRickCrawford@Sen_JoeManchin@HouseHomeland@RepDevinNunes@RepRaulGrijalva@SenJoniErnst@EnergyGOP@RepDebDingell@RepMikeCoffman@SenJohnMcCain@DesJarlaisTN04@RepDeSaulnier@RepJoshG@SenJohnKennedy@CongressmanGT@RepDaveBrat@RepJohnLarson@SenFeinstein@repjohnlewis@RepChrisStewart@RepJohnFaso@RepZoeLofgren@repdavidscott@RepBlaine@RepHanabusa@RepScottPerry@repcleaver@PeterWelch@RepGwenMoore@RepRichHudson@maziehirono@NancyPelosi@RepGuthrie@RepRaskin@TGowdySC@JeffFlake@RepGusBilirakis@RepJimmyGomez@SteveKnight25@HouseCommerce@RepGosar@RepJerryNadler@SenatorRounds@HFACDemocrats@RepDougCollins@RepDonaldPayne@SenatorMenendez@ConawayTX11@RepDavidRouzer@RepDennisRoss@SenSherrodBrown@BillPascrell@RepCheri@RepDelBene@SenJohnHoeven@repblumenauer@RepBrianMast@Raul_Labrador@SenAlexander@janschakowsky@RepBillShuster@PeteSessions@RepStephenLynch@USRepKCastor@RepArrington@OversightDems@RepShimkus@SteveKingIA@RepAnnieKuster@NydiaVelazquez@RepSanfordSC@SenatorTimScott@AustinScottGA08@MikeKellyPA@RepMikeBishop@SciCmteDems@timkaine@LindseyGrahamSC@RepKevinCramer@RonWyden@nikiinthehouse@JudgeTedPoe@RepKClark@RepTomReed@boblatta@HouseJudDems@RepJoeKennedy@RepSarbanes@TomColeOK04@GreggHarper@RepJimCosta@RepRichardNeal@SpeakerRyan@GovRonDeSantis@RepJeffDenham@RepPittenger@SenThomTillis@GOPLeader@RepGregoryMeeks@RepMarcyKaptur@SenGillibrand@GKButterfield@RepGallagher@RepLouCorrea@SenBobCasey@EleanorNorton@RepFilemonVela@RepLloydDoggett@RepWilson@CongressmanRaja@RepChuck@RepKayGranger@RepWalberg@JohnBoozman@RepHuizenga@SenJackReed@repgregwalden@DorisMatsui@RepGoodlatte@SenAngusKing@lisamurkowski@replouiegohmert@RepDanKildee@RepYvetteClarke@SenToomey@daveloebsack@RepByrne@RepWesterman@SenDanSullivan@USRepKeating@RepBillJohnson@RepTimWalz@RepTomMacArthur@TXRandy14@PeterRoskam@RepTedYoho@RepTenney@SenatorTomUdall@LamarSmithTX21@RepRonEstes@RepPeteOlson@SenatorTester@JudgeCarter@RepRodBlum@RepPeteKing@SenatorHeitkamp@JimPressOffice@RepMarkTakano@RepPaulTonko@SenWarren@FinancialCmte@RepLujanGrisham@RepMarthaRoby@SenDonnelly@senatemajldr@RepKathleenRice@RepLynnJenkins@RobWittman@rosadelauro@RepDerekKilmer@RepLoudermilk@RoKhanna@keithellison@RepDaveTrott@RepLoisFrankel@RepTomRice@Transport@RepChrisSmith@RepHultgren@RepSusanDavis@TomRooney@MarkAmodeiNV2@RepDwightEvans@RepStefanik@SenatorDurbin@IndianCommittee@RepDianaDeGette@RepRonKind@SenMikeLee@DWStweets@RepDavidValadao@RepRichmond@Robert_Aderholt@repmarkpocan@RepDarrenSoto@RepPaulMitchell@RepWalterJones@brianschatz@RepCummings@RepMcEachin@RepTomMarino@StaceyPlaskett@RepComstock@RepLaMalfa@RepTomEmmer@SenRonJohnson@RepAnnWagner@RepJeffries@RepTimRyan@SenCoonsOffice@RepAndreCarson@RepJaredPolis@RepSinema@SenBobCorker@RepAlexMooney@RepJackyRosen@RepMoBrooks@SenBillNelson@RepAdamSmith@RepHuffman@RepLanceNJ7@SanfordBishop@JimLangevin@RepGonzalez@RepKinzinger@RosLehtinen@Clyburn@RepDavidKustoff@RepJohnDelaney@RepTomGraves@jaredpolis@RepCharlieDent@RepJayapal@RepTomGarrett@USRepRodney@RepBobbyRush@RepDWStweets@RepThompson@USRepMikeDoyle@RepAdamSchiff@RepDLamborn@RepPaulCook@SteveDaines@MarkWarner@RepCuellar@RepMarciaFudge@SenatorIsakson@JimInhofe@RepAdrianSmith@RepLoBiondo@SenKamalaHarris@CongressmanHice@SenTomCotton@cathymcmorris@RepScottTaylor@virginiafoxx@SenToddYoung@SteveScalise@RepJimmyPanetta@repsandylevin@SenSchumer@RepPeterDeFazio@RepFrenchHill@louiseslaughter@SenMarkey@RepMiaLove@RepDianeBlack@VernBuchanan@SenJohnBarrasso@RepMcClintock@PatTiberi@USRepRickNolan@SenDuckworth@RepLarryBucshon@HouseScience@RepVisclosky@SenCapito@RepKristiNoem@repbenraylujan@RepTedDeutch@RepTrentKelly@RepHolding@aguilarpete@RepSteveStivers@RepTomSuozzi@RepHastingsFL@SenGaryPeters@RepJohnConyers@RepSchrader@RepHankJohnson@RepRoybalAllard@RepCohen@RepPoliquin@RepCurbelo@RepOHalleran@RepBera@RepDavidYoung@RepBrianBabin@RepMikeQuigley@USRepGaryPalmer@RepDavidEPrice@repdonyoung@RepMcNerney@SenatorCollins@RepDaveJoyce@davidcicilline@RepJimBanks@SenatorCantwell@RepAnnaEshoo@congbillposey@RepBarbaraLee@SenShelby@JuliaBrownley26@WaysMeansCmte@DrPhilRoe@RulesReps@HouseAgNews@SenatorRisch@DrNealDunnFL2@RepTrey@FrankPallone@SenatorLeahy@marcorubio@RepSires@CoryBooker@SenateBanking@RepRoKhanna@RepLizCheney@michaelcburgess@SenDeanHeller@RepFrankLucas@RepJBridenstine@RepKenBuck@RepJasonLewis@RepErikPaulsen@RepDonBeyer@ChrisCoons@JeffFortenberry@PattyMurray@SenatorShaheen@BobbyScott@senrobportman@JohnCornyn@SenateAgDems@WhipHoyer@RepSchneider@HouseForeign@RepVeasey@SenTinaSmith@RepKarenBass@GrahamBlog@RepTedLieu@RepWebster@HerreraBeutler@CongMikeSimpson@RepRobWoodall@RepMikeRogersAL@RepPeteAguilar@RepJoeCourtney@DarrellIssa@amyklobuchar@SenSasse@RepBillFoster@davereichert@TulsiPress@RepRubenGallego@sethmoulton@SenatorEnzi@SenatorBaldwin@RepPerlmutter@RepTerriSewell@SenateGOP@RepRobinKelly@RepCharlieCrist@RepRussell@RepKenMarchant@MarshaBlackburn@SenTedCruz@RepJasonSmith@RepLowenthal@ChuckGrassley@RepSpeier@johnculberson@RoyBlunt1009080706050403020100

Here is the raw data and the Excel sheet.

Surprisingly, according to the »Botometer«, almost half the US Congress members are bots. It is little to say that we are not impressed with the accuracy of this tool.

Berkeley/Swansea

The third study was focussing on the influence of social bots on the referendum to determine whether the UK should leave the European Union. The criteria are long, complicated and hard to verify, but the researchers checked their validity with:

To check the validity of our procedure to identify bots, we compare our bot definition with bot detection based on an online classification tool called Botometer (formerly BotorNot).

The fact that the flawed »Botometer« is supposedly validating the method doesn’t allow us to be very confident about the rest of this research.

You can't calibrate a broken scale with another broken scale.

Other qualms

We would very much like to look deeper into the three methods we mentioned above. But as we said earlier, none of these research groups has published their data.

There is no doubt that spam bots are being deployed on Twitter to sell bitcoins and cheap sunglasses. But as no one follows these accounts, they use hashtags to generate traffic to their sites, piggybacking on trending topics to reach people’s feeds. In the nine days before the US election in 2016 – when the Oxford team gathered its data – these hashtags were consistently trending. We can assume that some sites peddling fake Ray-Bans were probably counted as social bots in these studies. While they are obviously operated by bots, can we credibly assume that they influenced anyone’s vote?

Please remember: There's bots, and there's social bots. We should understand the difference.