When scientists began working with artist Francesca Samsel in order to visualize things like how deep ocean currents ran, or how diseases travel, or how fast flooding from rising oceans would affect specific shores, many were accustomed to working with the Rainbow Color Map. This simple visualization tool, of colors so bright they look like a poster for psychedelics, might show a field of bright red next to a field of bright green. The dramatic color change implies that something dramatic happened in the data. But often, that’s not the case; it’s just that the color palette is so limited. But if the artist tries to indicate that values are similar by using similar hues, key data can get lost in the pool of saturated color. The idea of data visualization is to understand what the data is saying. So Samsel designed custom maps that use color theory to tell a story of subtle and dramatic changes, and draw the eye to the important information.

Not only that, but she makes clay sculptures of various shapes and textures and incorporates images of them to represent various bits of information in her visualizations.

“The rule of thumb is that the eye can only detect five things at a time,” she said. But she discovered when she greatly varies the size, shape, color, and texture of the objects in a visualization, the eye can take them all in. In one visualization, for example, some of the data is represented by objects with the shape and texture of croissant, and others by sharp, metallic-looking pyramids. One doesn’t have to work to distinguish between them. The result isn’t chaos; it’s beautiful. And it reveals insights that scientists would never have seen in more simplistic maps.

As quantities of data pile up (we’re now measuring data in quintillion bytes—which sounds like a made up number), the ability to put that data into a form that reveals patterns and anomalies within complex systems is crucial. Whether the data scientist is working with scientific data visualization—known as Sciviz—or with business information data visualization—infovis—the visualization is useless unless it can give the viewer the insights they need to make decisions.

Taking a step back in time

Data visualization isn’t new. Hieroglyphs can be a form of data visualization. The simplest chart showing profits up or down is a form of data visualization. But with the growth of big data came the need to be able to wrangle huge and complex data sets into insights lest it become a burdensome cacophony. Data visualization has become the tool that made that possible—though not everyone wields it effectively.

“Having the tools to visualize data is one thing, but being able to communicate insight is another,” said Andy Cotgreave, senior technical evangelist for Tableau Software, the leading data visualization platform. “In the same way that having access to Microsoft Word doesn’t make you a writer, a data visualization tool doesn’t make you a good communicator or data analyst. Vendors and educators must now concentrate on data literacy and ensuring that people with access to these tools have the skills to use them properly.”

“Data visualization has completely dropped off the Hype Cycle, suggesting that Gartner believes it is a commodity and a standard amongst the tools available to organizations,” he said. “Now the hype has died down, the real work begins.”

In customer service, for example, every single interaction is packed with data about the customer’s demographics, geography, personality, disposition, and service preferences. There may also be information available about a product, device, customer website usage, marketing touches, sales or subscription terms, and prior agent interactions. This data may be layered to reflect fluctuations across times of day, week, month or year. Sometimes it’s difficult to know which of these pieces matters until you can dive into them at a granular level, where patterns emerge that explain phenomena you never knew was happening. Making the data visual allows you to see, literally, what needs to change.

Visualization reveals secrets

If you look at the preponderance of data visualization that is produced, a lot of it may be pretty, like the bright rainbow color maps, but reveals little. The point of data visualization is to present you with the information you need in a fraction of the time it would take to read the raw data. Or as Cotgreave put it: “exploring data at the speed of thought.” Then there are the data visualizations that reveal great information, but aren’t pretty—and so you don’t want to look at them. They are jarring to the eye. Paul Navratil, director of visualization for the Texas Advanced Computing Center’s Data Visualization Lab, notes that when something’s beautiful, you want to “hang out with it a little longer,” which allows you to see more in it.

The point of data visualization is to present you with the information you need in a fraction of the time it would take to read the raw data.

Data scientist Justin Fung has always loved maps. He used to spend hours looking at maps as a child. So after he got his degrees in applied math and computer science, he elected to do his graduate work designing emergency evacuation routes for the city of New York, using a map. Hurricane Sandy had just happened, taking nearly 300 lives. So Fung spent years collecting information about the populations of each neighborhood, about transportation and other data that would show where people were in New York at every hour of the day. Quite by accident, he also captured the city’s beating heart.

“A video is made up of a bunch of different frames and I did each one by hand,” he said. “Each one is an hour of a day. It wasn’t until the very last step when I turned it into a video that I saw that if I sped it up a little bit it became this organic thing. It starts breathing at you. It looks like an organ. Numbers are very objective. They don’t have artistic or organic value. But you put it all together and it starts to become artful. It starts doing all these things you didn’t expect it to. I guess it’s like you send a child into the world and they turn into something els

Samsel had a similar experience when she created a visualization of tides for scientists trying to understand the impact of climate change on the ocean. Until she applied her color map to the data, they didn’t know whether these irregular donut-shaped tides were only on the surface or went deeper. Her visualization revealed they go all the way to the ocean floor.

Scientists aren’t the only ones able to pull surprises from data visualizations. Tableau has case studies about companies like AmeriPride, a uniform rental and linen supply company, which created a product roadmap analyzing several data sources. One of these data sources was information from a fleet of trucks that drives more than 350 million miles a year. “The company discovered that their best margins came from customers within half a mile of each other. An insight like that can define an entire business strategy, increasing efficiency and cutting costs.”

The University of Michigan Medical Center used data visualization to save 5,000 hours of work per year because of the insights provided by its Fast Analytics team.

“The team created a charge estimate tool that better assists customer service staff in providing estimates to external customers about healthcare services. We took a four-and-a-half-hour process that involved a number of tools and high-level of expertise and turned it into four-and-a-half-second query,” said Jonathan Greenberg, the team’s director.

After that success, they combined over 50 data sources into more than 60 visualizations for the executive management team, who continues to use these dashboards to track success against organization-wide metrics and goals.

It’s that ability to reveal insights that makes data visualization so important to scientists, businesses, governments, and organizations. And though visualization has already come a long way from rainbow color maps, the possibilities for the future seem even more exciting.

The future of data visualization

The University of Texas has two data visualization labs in Austin. One at the main campus has one of the world’s highest resolution tiled displays. The smaller data visualization lab has two sets of nine giant televisions—literally, director Paul Navratil said, they just hit Best Buy and walked away with 18 huge TVs—that face each other and form a concave shape. They found that commoditized TVs worked better with most of the other hardware and software they had available. With these screens, they can spread a visualization across the entire surface, revealing small, but important bits of data that might otherwise be overlooked. Or they can put several related visualizations on different screens to reveal patterns across different data sets.

“We find value in having the real estate because of what it unlocks in terms of potential,” Navratil said. “If this is your sandbox…” he said, pointing to a typical single computer or monitor, much less can be seen and explored.

He stretched several images across the host of screens. A visualization of Arctic tides in blue and silver, when blown up, revealed golden eddies that had far more impact than their size implied. A blown up version of Vincent Van Gogh’s Starry Starry Night showed the brushstrokes and bits of exposed canvas so clearly it was like watching the artist paint.

The skills of this data lab—including Samsel’s artistic skills—are used by organizations like NASA, the Center for Disease Control, and international research facilities like Sweden’s Karolinska Institutet.

As one of the world’s leading data visualization labs, UT has to keep on the cutting edge. And there’s a lot of cutting edge to be had in data visualization. Samsel’s clay objects, for example, are pushing the boundaries of the “five objects” rule. With her color map, and by playing with organic-versus-digital representations of data, she’s expanding the language that’s available to express complex data sets.

Another important place on cutting edge, she said, is interactivity. Whether with virtual reality, or just creating layers of data on a touch screen, or through some other interactive means, it’s possible to bring the viewer to deeper and deeper insights.

Tableau’s Cotgreave said Tableau is working on many fronts and tools to push the envelope, through AI and machine learning, for example. Though data-driven decision making is “still a human endeavor,” Tableau is trying to move some of the more tedious parts of data visualization to technology, and also to help build a more data-literate world. Justin Fung said one challenge for data visualization going forward is visualizing the data from things like facial recognition software which makes it clear to people that computers are watching them when they’re not looking.

Fung said when he thinks about how to represent a data set, he thinks in terms of how to explain something statistically dense to his mom, or to someone else for whom the data might be foreign. For many people, he said, the information gleaned from AI, machine learning, and facial recognition software can be scary. You have to do the visualization just right.

How will he do it then?

“I don’t know,” he joked. “Add a lot of cute colors and rainbows. Maybe dogs.

Similarly Cotgreave said one of the biggest challenges he sees going forward is visualizing uncertainty. “In a world beset by major political challenges and accusations of fake news, the use of data stands accused of portraying values with too much certainty. When we show data without the levels of uncertainty, data visualization is at risk of being overly precise, describing a reality that is, at best, vague.”

Samsel believes it’s the art that helps put the data in context with the world.

“If you want people to change their behavior, you have to touch them in an emotional and intellectual way,” she said. “You have to let them follow the threads that interest them—visualization, animations, poetry. Artists provide the context. Why do we tell children stories? We’re telling them about the world via metaphor.”

Data visualization creates a context for individual stories, streams of activity or behavior, that show relationships and reveal the impact of relationships. It’s a high calling for data.