Top banner ad available.

chart_junk_effective.jpg
Yep, it has been scientifically proven: the accuracy of people in describing charts with 'chart junk' is no worse than for plain charts, and the recall after a 2-3 week gap was actually significantly better. In addition, people overwhelmingly preferred 'chart junk' diagrams for reading and remembering over plain charts. In all, the researchers conclude that if memorability is important, elaborate visual imagery has the potential to help fix a chart in a viewer's memory.

I am sure Tufte is not going to like this...

The findings have been described in the paper "Useful Junk? The Effects of Visual Embellishment on Comprehension and Memorability of Charts" [hci.usask.ca]. About 60 participants were asked to look to 14 different information graphs created by Nigel Holmes (see also his book Designer's Guide to Creating Charts and Diagrams) and their equivalent, custom-made 'plain' versions. The 'chart junk' charts were all designed to attract the eye, engage the reader, and sometimes provide a particular value message over and above the presentation of the data itself. In fact, the researchers deliberately chose the most extreme type of visual embellishment that they could: namely, the full cartoon imagery used by Holmes.

The participants then answered questions about each chart's topic and details, such as 'What is the chart is about?', 'What are the displayed categories and values?', 'What is the basic trend of the graph?' and 'Is the author trying to communicate some message through the chart?'. Half of the participants then answered the same questions again, after about 5 minutes of playing a game, and half after around 12 days. The experimenters then recorded any correctly recalled charts (e.g. 'I remember one about the price of diamonds').

"The illusion of objectivity (as used in minimalist charts) and the use of evocative imagery (as used in Holmes charts) are perhaps just different approaches that work at different ends of the rhetorical spectrum. Designers and readers should remember that a Holmes chart is not necessarily more biased than its plain counterpart - but it may be more effective at conveying the value message that is part of the overall argument."

Via Eager Eyes.

MORE

chart_junk_effective2.jpg

12 COMMENTS

Interesting work. Reminds me of Pourang Irani's Ph.D. work that compared the effectiveness of plain UML diagrams to "3D-ized" versions that showed the same information but with more visually salient use of shapes, shading, etc. http://hci.cs.umanitoba.ca/Projects/GeonDiagrams

Wed 28 Apr 2010 at 1:29 AM

Is recall the most important thing when looking at a chart? I'd say the ability to read the data and understand the relationship between the data points is the most important thing. I briefly looked through the study and it seems to miss the point of a Tufte style minimalist chart. Recall of a general idea is fine, but a chart is there for reference and comparison, which "chart junk" has the potential to disrupt.

Wed 28 Apr 2010 at 6:35 AM
Nate Demarest

Wait, the 2nd chart is still junk though--the Y axis doesn't start at zero, so the relative-value implications can be misleading.

I think that's a deeper sort of chartjunk than just stockings.

Wed 28 Apr 2010 at 10:54 AM
Chris Chen

Most people are a little dumb. Sorry, this is sad but true.

In popular media, the diamonds chart with the leg would win out over a Tufte-approved chart every time.

There is still value in what Tufte says, but if the extra ink provides some EMOTIONAL response, there's actually some value in that.

Wed 28 Apr 2010 at 11:52 AM

Well, what I got of this is that the addition of chart junk improves recall, i.e. advertising. Show someone ten of the most important graphs you can think of and a graph of your weight over the course of a week, but add a pretty girl (or Brad Pitt laughing at your weight) to the weight graph and ask them in a year which one they remember.

Wed 28 Apr 2010 at 5:01 PM
Ken Pierce

Do like the term 'chart junk', very nice examples to demo as well.

Will keep in mind for my charts in the future. Thank you

Wed 28 Apr 2010 at 5:07 PM

This study comes across as very biased. Beautifully crafted original art representaions of data(in color) are more memorable than plain(or more accurally described, ugly) standard graphs in greyscale. And the material is from the artistic versions, which the standard graphs are to emulate. What a find!
There are some interesting questions here, but this study misses most of them.

And whats with this quote:
"Charts and diagrams are certainly useful for offering general, relational explications of an issue but they necessarily shave away the ambiguous, nuanced, or obscure aspects of any idea. The information has been preprocessed, prechewed; it can only lead to one conclusion."

Idea, information, conclusion. This argument is just as valid for any form of information presentation, but I would say less so for graphs than text. A text is certainly preprocessed, prechewed and leaves typically very little room for making your own conclusions, whereas a good information visualization can present information on many levels, the overall idea, grouping or classes within the information and specific data(obsure aspects). All in the same context.

Wed 28 Apr 2010 at 11:44 PM
J├Ârgen Abrahamsson

@Chris Chen : either they have changed the "Tufte-approved" chart or you have misread it : the Y-axis does start at zero !

Even the 'junk' sample, though loaded with it, is not as basically inaccurate like some exmaples that Tufte condemns (especially those with 3D effects).

That being said, I'm not surprised at the findings themselves, because they already have been amply publicised in other cognitive work, such as those underlying the effectiveness of highly colorful and image-loaded mindmaps.

In the end, designing a graphical representation of your data does depend on who you want to communicate your data/conclusions to, what other info is provided with the chart, how it will be conveyed (paper, screen, video, with or without comments etc), so I do not believe there is a one-size-fits-all approach that can produce the perfect chart.

So there will always be a great deal of compromise in any chart, which makes very narrowly focused studies such as there seem a little... too narrow. It still sheds some additional light, however small in quantity, on a complex subject, and we certainly gain by enhancing our understanding of the pros and cons of each approach.

Thanks for sharing !


Sat 01 May 2010 at 3:34 PM
Robin

Who are we making charts and graphs for? Edward Tufte? Or everyone else? Just please don't use 'chart junk' at the expense of accuracy.

Tue 04 May 2010 at 12:10 PM

Quite frankly, this study validates Tufte's points exactly, with the leading graphic (diamond prices peaking in 1980, over a 5-year span) being a prime example. Why restrict the data range to 5 years? Perhaps because it wouldn't have resulted in a pretty chart? If you were investing in diamonds wouldn't you want long-term performance information? If you were trading in them, wouldn't you want a more granular view of the data, or perhaps this would put unattractive ladders in the lady's tights...

The less said about the "plain" charts in the study the better, suffice it to say that they seem to have been created with the intention of being very poor examples of their type.

I wouldn't be surprised to see Tufte quote this study in future, in order to show how the drive to create (and justify the use of) pretty charts can lead to misrepesentation, irrelevance and deception, however good the intentions.

Fri 07 May 2010 at 12:51 AM
Kevin

It was amusing to see your own bare graph indicating the ineffectiveness of bare graphs. I can only assume you do not want people to remember these results!

Wed 19 May 2010 at 4:10 AM
Jen

This study would be much more interesting if it had been applied to non-trivial data. What if one had presented viewers with 100-data point graphs, or multivariate graphs. The reason that people don't remember the plain charts, and only remember the decorated ones seem very straightforward: The graphs do not display interesting data. Data is often from 20-30 years ago. As it is now the study can only conclude that very data-impoverished graphs with what is arguably uninteresting information can be made more memorable by large amount of decoration.

Fri 06 Aug 2010 at 9:50 PM
Rafael
ADD A COMMENT
Commenting has been temporarily disabled.