Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


05:45 PM
Connect Directly

Twitter, Facebook, NSA Discuss Fight Against Misinformation

RSA panelists address the delicate technical challenges of combating information warfare online without causing First Amendment freedoms to take collateral damage.

RSA CONFERENCE 2019 – San Francisco – Information warfare is often waged on social media, where legitimate consumer communication tools are weaponized by bad actors. As Facebook, Twitter, and National Security Agency (NSA) representatives discussed here today, the battleground is civilian territory, and if the defenders aren't careful, First Amendment freedoms will suffer severe collateral damage.

"So far, America has emerged as one of the clearest losers in this kind of warfare," said panel moderator Ted Schlein, general partner at Kleiner Perkins Caufield & Byers, during the session "The Weaponization of the Internet."

(In another keynote session Wednesday, General Paul Nakasone, commander of US Cyber Command, told CBS News' Olivia Gazis that while Americans saw the Internet as a way for democracy to spread throughout the world, adversaries saw that same possibility as a threat.)

Schlein posed the question of why US intelligence agencies hadn't gotten ahead of threats sooner – threats like disinformation campaigns, voter manipulation, hate speech crimes, and recruitment by terror organizations.

"I think there were efforts, but ... we're trying to shape and react in a place where we're in the middle of speech," said panelist Rob Joyce, senior cybersecurity adviser to the NSA. "We're in a place where, as Americans, we value that First Amendment and the ability to say what I feel, I believe. And getting in the middle and breaking that disruptive speech that can be amplified on these platforms – that's a hard place for America to go."

However, panelist P.W. Singer, senior fellow at New America and author of "LikeWar: The Weaponization of Social Media," suggested that intelligence services, platforms, and politicians were all "looking in the wrong place" for bad actors.

"We were looking, for example, for people hacking Facebook client accounts, not buying ads at scale that over half the American population saw unwittingly," Singer said. "We were looking in the wrong place. They were looking for attackers who exploited Facebook accounts, not ones who bought Facebook ads."

Indeed, attackers are building off some techniques first perfected by marketers. As Twitter VP of trust and safety Del Harvey explained, the first type of manipulation that Twitter discovered was a campaign to convince Justin Bieber to do a tour in Brazil; it was the first example of a strategic effort to create and sustain a trending topic. (Bieber did end up touring in Brazil, she noted.)

"ISIS's top recruiter is mirroring off of [pop star] Taylor Swift and what works for her to win her online battles," Singer said. "Or, in turn, Russian information operations are using the tools created by [Facebook and Twitter] not to market how they were intended but to misuse them to go after American democracy.”

So can the platforms tackle the malicious use problem by simply scanning tweets for ISIS recruitment videos and Russian propaganda (and ignoring Taylor Swift)? Not necessarily.

"Content is actually one of the weaker signals' of a bad actor," Twitter's Harvey explained.

Content might not be shared for many reasons: Terrorist recruitment propaganda might be shared as part of a news report on that terrorist organization, for example. Conversely, Harvey said, "There are certain behaviors that you can identify as being attempts at manipulation." 

For example, a user may be part of a network of accounts pushing the same messaging. These accounts are also related by IP address and carrier. They may be targeting certain networks or trying to social engineer their way into a trusted group.

This behavior of a manipulator is actually quite dissimilar to that of the community-native true believer who shares the same content, Harvey said.  

NSA's Joyce says behavior is connected in some way to three main categories of an account: "The content itself, which we all agree is the most troublesome and the hardest to deal with. And then there's an identity; it may be real, it may be assumed. And then there's amplification.”

Panelist Nathaniel Gleicher, Facebook's head of cybersecurity policy, added that whenever there is a public discussion up for debate, bad actors will target that debate. The challenge is stopping the bad actors without stopping the debate.

"The way you make progress in the security world is you identify ways to impose more friction on the bad actors and the behaviors that they’re using, without simultaneously imposing friction on a meaningful public discussion," he said. "That's an incredibly hard balance." 

Facebook approaches this challenge, Gleicher said, with a combination of automated tools and human investigators, who look for the most sophisticated bad actors, identify their core behaviors, and develop ways to automatically make those behaviors more difficult to commit at scale.

Because regulating content is problematic, they may tackle the issues of identity and amplification instead – such as changing the way ads are purchased on Facebook and making it more difficult to create fake accounts or bots.

"None of this means that we shouldn't be taking action on content that clearly violates our policies," Gleischer noted. "The challenge is, the majority of the content we see in information operations doesn't violate our policies. It's not clearly hate speech, and it's potentially framed to not fit into that bucket. And it's not provably false. There's a lot that fits into that gray space."

Twitter's Harvey noted that the conversation of "bots" has become so pervasive that it has begun to have a cultural impact on regular human discourse.  

"It is amazing the number of times you will see two people who get in an argument and one of them decides to end it by just saying, 'Well, you're just a bot [when] it is demonstrably not a bot," she said.

Pasting the label of "bot" on anyone with a differing opinion is being used as "an exit path from conflict, from disagreement," Harvey added. "In fact, you're a Russian bot. And you are here to try to sway my mind on the topic of local football teams."

Related Content:



Join Dark Reading LIVE for two cybersecurity summits at Interop 2019. Learn from the industry's most knowledgeable IT security experts. Check out the Interop agenda here.

Sara Peters is Senior Editor at Dark Reading and formerly the editor-in-chief of Enterprise Efficiency. Prior that she was senior editor for the Computer Security Institute, writing and speaking about virtualization, identity management, cybersecurity law, and a myriad ... View Full Bio

Recommended Reading:

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
COVID-19: Latest Security News & Commentary
Dark Reading Staff 7/9/2020
Omdia Research Launches Page on Dark Reading
Tim Wilson, Editor in Chief, Dark Reading 7/9/2020
Mobile App Fraud Jumped in Q1 as Attackers Pivot from Browsers
Jai Vijayan, Contributing Writer,  7/10/2020
Register for Dark Reading Newsletters
White Papers
Current Issue
Special Report: Computing's New Normal, a Dark Reading Perspective
This special report examines how IT security organizations have adapted to the "new normal" of computing and what the long-term effects will be. Read it and get a unique set of perspectives on issues ranging from new threats & vulnerabilities as a result of remote working to how enterprise security strategy will be affected long term.
Flash Poll
The Threat from the Internetand What Your Organization Can Do About It
The Threat from the Internetand What Your Organization Can Do About It
This report describes some of the latest attacks and threats emanating from the Internet, as well as advice and tips on how your organization can mitigate those threats before they affect your business. Download it today!
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2020-07-10
Django Two-Factor Authentication before 1.12, stores the user's password in clear text in the user session (base64-encoded). The password is stored in the session when the user submits their username and password, and is removed once they complete authentication by entering a two-factor authenticati...
PUBLISHED: 2020-07-10
In Bareos Director less than or equal to 16.2.10, 17.2.9, 18.2.8, and 19.2.7, a heap overflow allows a malicious client to corrupt the director's memory via oversized digest strings sent during initialization of a verify job. Disabling verify jobs mitigates the problem. This issue is also patched in...
PUBLISHED: 2020-07-10
Bareos before version 19.2.8 and earlier allows a malicious client to communicate with the director without knowledge of the shared secret if the director allows client initiated connection and connects to the client itself. The malicious client can replay the Bareos director's cram-md5 challenge to...
PUBLISHED: 2020-07-10
osquery before version 4.4.0 enables a priviledge escalation vulnerability. If a Window system is configured with a PATH that contains a user-writable directory then a local user may write a zlib1.dll DLL, which osquery will attempt to load. Since osquery runs with elevated privileges this enables l...
PUBLISHED: 2020-07-10
An exploitable SQL injection vulnerability exists in the Admin Reports functionality of Glacies IceHRM v26.6.0.OS (Commit bb274de1751ffb9d09482fd2538f9950a94c510a) . A specially crafted HTTP request can cause SQL injection. An attacker can make an authenticated HTTP request to trigger this vulnerabi...