Shining light on dark data, shadow IT and shadow IoT

Are employees keeping you in the dark?

The answer: Yes, they are.

If you’re an IT pro, security specialist, c-level executive, manager or leader in your organization (and if you’re reading this, you almost certainly are), people in your organization are hiding information from you, and also installing hardware and software and using cloud services without company authorization.

These hidden resources represent a massive and growing security risk. Gartner says that by the end of next year, one-third of all successful attacks on enterprises will take place on shadow IT resources.

And these resources can create other, less obvious problems as well.

Here’s what’s new and urgent about dark data and shadow IT, with an extensive look at shadow IoT — and what to do about it.

Dark data

Dark data is unstructured, untagged, unclassified and unknown to IT. We’re talking about dark documents, shadow spreadsheets, secret censor logs and other content.

Much of this data resides on public cloud services or on mobile devices or both.

A Veritas Technologies survey of IT decision makers and data managers in the U.K. found that almost half of the data at organizations in that country is “dark data” — unclassified or untagged. That means this data, which includes assets valuable to the company, can’t be managed, protected or accessed the way it should be.

Cloud data is a special problem. Many falsely believe that data protection and regulatory compliance for data stored on cloud service providers is the responsibility of the provider, according to a Veritas report.

Dark data is not a black-and-white issue. It’s very common, for example, for employees to email a document to themselves to work from home. The copy on the network is authorized, but the copy on Gmail and the copy on their home MacBook are dark data. They still contain the same sensitive information; they’re just beyond the protection of the organization’s security systems. Worse, they may modify the document at home, email it back to work and replace the authorized copy with one that could now be infected with malware.

Dark data creates four problems. The first is that it represents an enormous opportunity for cyberattackers. Because it’s unknown, it’s unprotected. Hackers may use dark information to gain insights into the organization, its employees, the location of assets and who knows what else.

The second is that dark data may contain organizational insights that never reach leadership. In aggregate, the company is operating with 100 percent of the data. But if half the data is dark, then leaders are basing decisions on only half the data that should be available to them.

The third is that dark data is wasteful and inefficient. Without knowledge about or access to a necessary data set, employees may re-create information that already exists, duplicating effort.

And finally, dark data is illegal. Or, at best, dark data complicates regulatory compliance with Sarbanes-Oxley, GDPR and other regulations.

Shadow IT

Shadow IT is any unauthorized computer system, device, application or cloud service that is operating beyond IT’s knowledge or control.

Familiar trends are driving it. The consumerization of IT. BYOD. The low cost and wide availability of storage and cloud services. The proliferation of “smart” devices and IoT, generally.

The first major effort to combat shadow IT was bring your own device (BYOD) policies. Before BYOD, people just brought PCs and other devices to work, connected them to the network and proceeded like everything was OK.

The umbrella category of shadow IT even includes Oauth-connected applications. That opens the approved application to unknown levels of access by the non-approved application. That’s right. Facebook’s servers can operate as shadow IT devices if employees are using Facebook to authenticate cloud services or apps.

At least desktops, laptops, tablets and smartphones are designed with security in mind. Toasters…. not so much.

Shadow IoT is a subset of Shadow IT but deserves special attention because of the special risks this category represents.

Shadow IoT

An IoT device is any non-computer object with both processing power and internet connectivity.

The number of IoT devices and sensors in enterprises and other large organizations is exploding, and much of it is happening without the knowledge, permission or management of IT.

An Infloblox report published last year found that around one-third of companies surveyed in the US, UK and Germany estimated they had more than a thousand shadow IoT devices connected to their networks.

I’m talking about thumb drives, webcams, smart speakers, smart displays, motion detectors, smart coffee makers, WiFi-connected vending machines, smart watches, routers, fitness trackers, microwaves, smart TVs, game consoles, smart toasters, smart lights, air conditioners, door locks, data-collection terminals, printers, medical devices and so much more.

Gartner expects 14.2 billion IoT devices online by the end of the year, and 25 billion connected by the end of 2025.

The 5G revolution isn’t going to help. Ericsson predicts 3.5 billion IoT connections via cellular networks by 2023, driven mainly by the growth of 5G networks.

The first clue that Shadow IoT was going to involve weird exploits came in 2016 when cybercriminals accessed the network of a casino through an internet-connected fish tank thermometer in the lobby. Once they got their foot in the door — I mean got their foot in the aquarium — they were able to extract a confidential database of “high roller” clients right through the thermometer.

More weirdness lies ahead. It’s almost as if IoT was designed for insecurity. Shadow IoT devices threaten network security, data privacy, physical security, operations and more.

IoT devices often lack enterprise-level security features and are installed and configured by users who may fail to implement even the most basic security, such as changing the default password.

More than 90 percent of IoT device transactions in enterprises happen over unencrypted, plain-text channels, according to Zscaler.

The same study found that some IoT devices are using plain-text HTTP for authentication and updates.

Some 80 percent of IoT is wireless, offering hackers a menu of wireless access options that include WiFi, Bluetooth, LTE, Zigbee and others.

IoT devices tend to use off-the-shelf, commonly used internal components. That means even an obscure device is likely to contain a well-known Wi-Fi chipset or other components that cybercriminals are familiar with. Many IoT devices may be weird, no-name gadgets, but security through obscurity often doesn’t apply.

Worse, not only are shadow IoT devices difficult to discover by IT, but they’re easy to discover by hackers, who can often just search the internet for such devices.

Shadow IoT devices may or may not be remotely updatable.

In the old days (2016 and before) most IoT attacks involved routers being hijacked to execute DDoS attacks. Now specialists predict that IoT attacks will become far more diverse over the next few years and cause unpredictable mayhem.

IoT devices need good password management, encrypted channel communications, regularly updated firmware and isolation where possible from most internal and external networks, with unneeded ports blocked. But none of that is going to happen until IT knows what and where these devices are. 

Some shadow IoT devices are employee owned — some actually connected to on the employee themselves (fitness trackers, for example), and come and go as the employee does. But others are purchased through authorized channels and are owned by the company. Office equipment like printers, break-room devices like smart coffee makers — you name it. Just because the company bought it doesn’t mean IT knows about it.

Why dark and shadowy resources exist

Dark data and shadow IT are everybody’s fault, and nobody’s fault. Dark stuff exists for three reasons:

1. Creativity and ambition. Employees do dark data and use shadow IT devices to get around the inconvenience of company security policies. People want to get their work done. They see barriers. So they think up ways to get around those barriers in order to do their jobs better. Many shadow IT devices are installed after being explicitly rejected by IT.

2. Ignorance. People are unaware of the problems dark data and shadow IT present to an organization, either at the micro or macro level. They’ll bring in stuff from home — say, an electronic picture frame that cycles through photos of their family — and place it on their desk without ever considering security.

3. Policy failure. Many dark and shadowy resources are fully authorized — by somebody else who didn’t think to inform IT. Some team somewhere decides to use Slack or Dropbox for a specific project as a way to interact with outside, overseas contractors. An old vending machine is replaced by the vendor with a newer model that’s “smart” and connects via Wi-Fi. The marketing department starts experimenting with experiential marketing ideas, which means they’re bringing in various gadgets and testing them on the internal network.

Inside these reason for dark data and shadow IT lie the solutions to the wider problem.

How to shine the light on dark data and shadow IT

I recommend the following actions to bring IT resources out of the shadows:

• Streamlining approvals. Maximize the speed and ease by which IT devices get approved.

• Over-communicate. Make sure everyone gets a copy of the approval process, and also train aggressively about the dangers of dark data and shadow IT.

• Isolate IoT devices on their own dedicated WiFi network, and block most of these devices by default from incoming calls.

• Monitor outgoing traffic for weird behavior.

• Use an automated tool for finding and creating inventories of all devices on the network.

• Maintain a clear list of authorized resources and deny network access to everything else.

It’s unlikely to eliminate dark data and shadow IT and IoT completely. But every bit helps. The more IT resources you can bring out of the darkness and into the light, the better your organization will function.