Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


05:00 PM
Connect Directly

Lessons Learned In San Francisco Hack Job

An IT administrator with the city of San Francisco is alleged to have created a private administrative account on systems within the city's FiberWAN project.

At one time or another, it happens in almost every IT shop: A handful of people, or even one person, has the sole responsibility for and knowledge of critical systems. So much so that if any one of them were hit by a bus, your IT department would be severely hampered. Virtually every IT manager recognizes this problem, but far fewer do anything about it. San Francisco is now learning its lesson the hard way.

Terry Childs, an IT administrator with the city of San Francisco, whom the city had tried to fire in recent months, is alleged to have created a private administrative account on systems within the city's FiberWAN project. According to prosecutors, he's keeping the password a secret. In so doing, he's locked everyone else out of the system's networking gear and landed himself in jail, with bail set at $5 million. The city is trying to regain access to its network and has brought in experts from Cisco to help break into the compromised systems. The details are murky but also not vital to learning from this painful incident. Simply put, a trusted employee is accused of using that trust to hose his employer.

Organizations must trust employees to act honestly and ethically while at the same time acknowledge that employees pose one of the greatest threats. Employees must have access to systems, and that means they can do damage. InformationWeek's 2008 Strategic Security study found that 53% of respondents considered authorized users/employees one of the greatest threats, and 48% of respondents considered all employees one of the greatest threats. While most employees are ethical, all it takes is one disgruntled person to cause you headache and misery.

No technology or process can fully stop an insider, especially a knowledgeable insider like Childs, from causing harm--but what can be managed, and what's shocking about this incident, is its apparent magnitude. Technology can help. Strong authentication systems could have made creating unique credentials more difficult, and change management products might have alerted someone to Childs' activities. But technology alone isn't the solution; good management plays a part, too.

Jonathan Feldman, director of IT for Asheville, N.C., says problems like these can crop up when the IT department relies on heroes versus using a team approach. He describes the IT hero as a lone wolf, a go-to person who is solely responsible for critical systems. IT heroes often don't like to give up control over their systems, Feldman points out. These employees pose a real risk to the organization because if they leave the job, are unable to work, or simply decide to strike back, no one else has the knowledge to work around them.

What's At Stake
Childs changed router and switch administrator passwords

The network serves 60% of the city's needs, including some police systems

Childs is suspended with pay while he sits in jail
The solution is to cross-train IT staff so that no one person alone understands and controls key systems; spread responsibility for systems over several people; and adopt a practical change management process. By doing those three things, the harm that any one person can inflict on your IT department is lessened because there are redundant skills and knowledge in play.

Building an IT team rather than relying on lone wizards requires management discipline and may seem a more costly approach; IT heroes are an easy shortcut for even the best-intentioned managers. However, the payoff in building a team is a more resilient IT department. "You know you are successful when you can let IT personnel go on vacation without having to ask for their cell phone and hotel numbers in case something comes up," says Feldman.

Rogue employees can still wreak havoc if they want. The best you can do is to take steps to minimize the damage.

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
I 'Hacked' My Accounts Using My Mobile Number: Here's What I Learned
Nicole Sette, Director in the Cyber Risk practice of Kroll, a division of Duff & Phelps,  11/19/2019
DevSecOps: The Answer to the Cloud Security Skills Gap
Lamont Orange, Chief Information Security Officer at Netskope,  11/15/2019
Attackers' Costs Increasing as Businesses Focus on Security
Robert Lemos, Contributing Writer,  11/15/2019
Register for Dark Reading Newsletters
White Papers
Cartoon Contest
Current Issue
Navigating the Deluge of Security Data
In this Tech Digest, Dark Reading shares the experiences of some top security practitioners as they navigate volumes of security data. We examine some examples of how enterprises can cull this data to find the clues they need.
Flash Poll
Rethinking Enterprise Data Defense
Rethinking Enterprise Data Defense
Frustrated with recurring intrusions and breaches, cybersecurity professionals are questioning some of the industrys conventional wisdom. Heres a look at what theyre thinking about.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
PUBLISHED: 2019-11-22
A cross-site request forgery (CSRF) vulnerability in the Activity module 6.x-1.x for Drupal.
PUBLISHED: 2019-11-21
An issue was discovered in Symfony before 4.2.12 and 4.3.x before 4.3.8. The VarExport component incorrectly escapes strings, allowing some specially crafted ones to escalate to execution of arbitrary PHP code. This is related to symfony/var-exporter.
PUBLISHED: 2019-11-21
An issue was discovered in Symfony 2.8.0 through 2.8.50, 3.4.0 through 3.4.34, 4.2.0 through 4.2.11, and 4.3.0 through 4.3.7. The UriSigner was subject to timing attacks. This is related to symfony/http-kernel.
PUBLISHED: 2019-11-21
An issue was discovered in Symfony 2.8.0 through 2.8.50, 3.4.0 through 3.4.34, 4.2.0 through 4.2.11, and 4.3.0 through 4.3.7. If an application passes unvalidated user input as the file for which MIME type validation should occur, then arbitrary arguments are passed to the underlying file command. T...
PUBLISHED: 2019-11-21
An issue was discovered in Symfony 3.4.0 through 3.4.34, 4.2.0 through 4.2.11, and 4.3.0 through 4.3.7. Serializing certain cache adapter interfaces could result in remote code injection. This is related to symfony/cache.