Problems In Communicating Complex Analytics

So food for thought can simplify technical problems to simple bullet points and get anyone killed? Well, the answer might be yes...

Excel to the rescue?

Many businesses I have been in are addicted to excel. I also do not mean to pick on excel all spreadsheets carry the following risks and of course, these risks may vary with similar but different tools. But again like PowerPoint excel is an archetype of a certain style of tool. 

Unlike emails and PowerPoint, spreadsheets include data. While it cannot perfectly present relationships in the data it can present some relationships via graphs and statistical analysis can be performed by using formulas. 

Conclusions are not presented in any verbal form but illustrated using graphs. Graphs are powerful in that they make conclusions inescapable and demonstrate the size of the effects; it is hard to ignore simple conclusions when one bar is bigger than another.

This being said graphs do not convey directives nor set objectives. A manager cannot no matter how implicit the conclusion shows a staff member a graph and expect performance to improve. There is a need to still include writing in business and explain conclusions and sometimes this gets missed out in spreadsheet-like reporting. 

Therefore the communication that excel and graphs serve is more informal and possibly presents more information but without formality. If you disagree I challenge you to write down a paragraph explaining the information you see in a graph it will probably take a while to cover the nuance of exact proportions between the different bars in a meaningful way. 

Negatives For Spreadsheets

• It is not a database: 
• Spreadsheet Risk: Because of the above it is unlikely to have real-time live data and there may be a risk of somebody in the business keeping "personal copy" (read old and bad data). 
• It is not a programming language (despite having VBA in it).
• Danger of creating "shadow IT" where a series of tasks are being done by spreadsheets and VBA automation that would be better handled by IT implementing a proper solution; though I for one think this assumes IT can or will in a reasonable timeframe create a solution. 
• Does not facilitate two-way communication or brainstorming ideas.
• Cannot communicate a directive or order. Any communication it facilitates does not create an obligation to act. Presents ideas and facts, not orders. 
• Rarely set up to present during meetings (there are exceptions with some dashboard tools).

Positives

• Visual graphs explain insights simply
• Carries with itself data allowing criticism and dissent if conclusions disagreed with
• Refresh of data can be automated and made a dashboard (or similar tools like PowerBi, Qlik etc)
• Can create database connections from various tools
• Formulas can automate presentation
• Presentation tools are equal to or better than PowerPoint in many ways but do not have tools to allow presenting in meetings

Honourable Mentions

• Dashboards in power bi or otherwise created and published by a central function. I champion this as the solution to the issues raised here. They can be set up with security, created by departments, published centrally, and distributed to everyone in a company.
• Databases: Allows ad hoc querying of the data and can support communications with statistics and data to 
• Word Documents: Often overlooked but the old technical report with full executive summary and essay fully exploring the problem space. 

Keep It Simple Keep It Safe

Excel may present the ability to resolve many of these issues but in reality, in most cases and for the good reasons of GDPR and security data is going to be sectioned away from the average user. Some businesses will have data marts set up as cleaned and pre-processed data that is safe for users to work with and does not carry a security risk though.

Therefore all the positives listed above assume a user can have access to the relevant data securely and ethically. If they cannot then usually output is throttled by centralised control such as IT or another organisation. 

Though many businesses are not that good at the data engineering required to set up this DataMart; there is a reason that data engineers speak about "one version of the truth" because most businesses' data has been scattered across several departmental databases, fragmented, the column's renamed and arguments over whether a customer with X marker is Y category or Z category being different in different departments; means that many businesses lack any single one central DataMart.

So even if a single clean DataMart exists to connect your excel and dashboards to it is possible IT does not want to allow widespread access to it for either security. Even worse some IT departments may want to limit access reasoning that without access the fragmentation (sometimes people politely call it federalising) from a single database into multiple departmental databases will be stopped if access is limited and controlled.

Therefore while excel does have many tools that allow communicating with data in powerful ways and while we can choose dashboarding tools that limit or remove spreadsheet risk and might be a better fit there might be issues around getting the data needed to communicate in the first place. Therefore while many businesses will think from the above that excel is enough I wish to over the rest of the article examine why this is not enough to just buy these tools and hand them to staff but also requires a reorganisation around their use. 

Bad Faith Data

Giving a tool like excel to everyone in the business sounds like it ought to at a stroke resolve the problems present by communicating technical ideas across the business. Though by supplying across the business we are supplying these tools to people of all skill levels and even potentially malicious intentions. Let us consider what are the downsides where data becomes misused.

• We have Chart Junk a term coined by Edward Tufte. The term arises from the quote at the beginning of the article. I also love the definition which is Chart Junk is defined as when the information in the chart is low compared to the ink on the page. 
• Then We have lying with statistics; a phrase from the book by Darrell Huff. Of course, it brings to mind the quote attributed to Mark Twain; "There is lies damn, lies and statistics" we will be learning about the common ways to lie with statistics in a shorthand way.

In the below sections I will hopefully give you an idea of how to spot both.  I will try to convince you that if the problem with PowerPoint and Email is they simplify information In the case study on the use of Graphs I will try to show they have been historically used to misdirect investors according to accountancy research.

This should hopefully show you why excel and spreadsheets are also not an easy way for businesses to solve the problems with communicating complex analytics is not solved only with spreadsheets.

Chart Junk

Graphs are a simplification of verbal communication. The below represents the relationship between profits along the X-axis and the genetic code for a neural evolution algorithm for stock trading I wrote where the genes represent behaviours turned on and off in the AI and the Hexes lightness or darkness represents the density of results in that section.

If a gene is particularly dark and far along the X-axis you would conclude that the gene probably has something to do with making money. You might ask if there was a big enough population but if I said yes it's a 10,000 sample size you probably have certain views. The negative numbers for genes on the Y-axis represent the absence of a given gene i.e. when it was turned off. All in all this graph shows 100 genes and the performance of the AI.

I am certain you can guess what the bar graphs above and to the side mean. They don't even need a description. Knowing that X is profit and Y is genes take a guess.

Now imagine writing a report on the below, imagine describing the relationship of every hex to every other hex and doing so in 10,000 words? You can see why Graphs act as a massive time saver.

So you get the point graphs to communicate more in one picture than 1000s of words. Graphs when true, concise and well designed let us switch off the verbal centre of the brain which has clunky linear qualities of being one word after another and deal with visual-spatial relationships which show relationships between any part and the whole. 

It is, for this reason, I suggest to you that graphs rule the business world. If you wanted to know the relationships between all the departments in the business and I give you a choice between hearing a lecture or I can show you this one graph; which one will you choose

Chart Junk Illustrated

I love Chart Junk as an Analyst nothing makes me feel more like a genius than seeing someone else screw up a simple bar chart. See below an example of unnecessary dimensions added to and the muddling of a simple time series. Just look at it you know you have no idea what it is about and you love it anyway.

Even if we cut down the number of rows 3D should not be used where it provides no additional information. Compare the below to my article on multi-dimension graphs here Multidimensional Graphs | A Logic Called Joe (webador.co.uk). You can see no additional information is supplied. This breaks a rule that we will discuss below that any extra ink without more information added is a waste.

In short, the below contains things that provide no information and therefore ought to be removed.

Further Notes

• Why the .5s? Why 1.5 or 2.5?
• where are the legends?
• what is this graph saying?

I also found the below when researching the Iraq war PowerPoint presentations. The original is slide 11 found here at everett.ppt (live.com). What does the added 3rd dimension give this chart? Why are the Award Actions labelled and Award Dollars not? Are the grid lines behind made worse because the 3 dimensions distort around the edges?

This being said I think we do have to take our hats off to the intelligence agencies they do declassify their PowerPoint presentations for public consumption. How bad would it be if your organisation was forced to do the same and how many errors would we find?

Another candidate is the pie chart below it appears to show the top 10 emojis thereafter you cannot gain any further information. The majority of that pie chart is just a kaleidoscope of colours. 

This is because the pie chart has two ways to show information the first is a coloured segment and the second is the angle between these segments. When there are too many segments it suffers f4rom lateral masking which is where the human eye cannot distinguish between lines that are too close together. Likewise, the colours become indistinguishable. 

This graph would be better represented as a table just a list of emojis in descending order. You would at least know more than the top 10 emojis and if presented even as a list of top 10 you would have greater clarity. Even better some tag information might let you filter by emojis.

Ornament Is A Crime

"The evolution of culture marches with the elimination of ornament from useful objects," Adolf Loos

There is a whiff of the ideas of Adolf Loos in the conventions of chart junk. Edward Tufte has arrived at the idea that chart junk is best defined as having a higher ratio of ink to information in the graph. It is obvious when we encounter these extremes of the chart junk showed above that we rightly identify them as "ornamental" and consider them "criminal" against our convention of tastes. 

With Chart Junk Information and insight are limited by the need for the graphs to be garishly ornamental. Excessive colours constrain the communication of information, 3d graphs distract from the facts intended to be conveyed. Form follows function and charts junk sins against its function.

Graphs act as a summary of statistical information in a spatial visual format that because of our visual nature can communicate very rapidly statistical information. 

Though having considered that the function of graphs is to share information about statistics in a simple format and chart junk is a bad, or inept attempt at creating a graph what happens when graphs go bad?

Misleading Graphs 

"It [a misleading graph] is vastly more effective, however, because it contains no adjectives or adverbs to spoil the illusion of objectivity, there's nothing anyone can pin on you." — How to Lie with Statistics (1954)^[2] 

If good graphs explain information simply and chart junk complicates simple information then surely a malicious graph actively lies?

Ok so there is chart Junk but we do not produce junk maybe being more scientific would be the fairer approach. Let us refine the definition away from chart Junk to define chart design that while not malicious is not helpful. Though about the above quote that would make it the perfect crime. 

It has already been noted that graphs convey more information with less text than a report, despite being shown as an elegant graph I would suggest the first graph in this blog suffers from this. It requires a paragraph of text to explain it. would it have suffered to have been done as a bar graph representing the average profit related to that gene? 

So how do we lie with statistics? What should we teach our data analysts and managers to look out for?

Excessive Usage

One way to lie with statistics would be to insert graphs where the use of such would distract from the message is that the illustration provided by the graph juxtaposes with the text and interjects a different message. It has been shown and is quite intuitive that the brain is bad at code-switching between different circuits and the visual-spatial circuits used to assess a graph and the verbal centre's areas already discussed in different parts of the brain. So a clever trick when discussing a difficult or problematic issue is to insert a graph tangentially related but showing the data in the best manner.

So beware of graphs used where no graphs were asked for! Especially if the graph is generic and the words accompanying it are verbose. The obvious rule of thumb here is the more a graph needs explaining the less valuable the graph is and surely there is a tipping point where the graph should just be removed and replaced with a flat table of data and explanatory text.

There are several studies on accounting practices that notes that there is a statistical correlation between graphs inserted into accounting works and whether we should consider them to be misleading. 

Biased Wording

The simplest trick here is to insert additional words. Graphs titles should be something like "income of ethnic communities", and "amount of stock per business sector". I.e. the text should read measure (i.e. whether a count, average, median etc) and what the object is (i.e. businesses, people, endangered species).

Wording such as a graph to show substantial growth of our company.

A good example I found is in politics. Nothing about either graph is dishonest and nothing in the title is a lie. Though knowing the title will be read first I can put the written message in the title then many will not look at the graph. Being critical you might note the graph on the left includes only 4/52 states while the graph on the right misses there is only a 6% swing between the candidates and usually changes as candidates drop out.

Nonetheless, the message you will remember is in the title... Though also still look for any captions or labels as secondary places where subjective terms are being injected to colour your interpretation of objective data. 

Pie Charts

There are lots of ways to lie with pie charts. I am not averse to a pie chart I think they take up less space than a table. Though ideally they should never be used. I'll leave the below comment from Edward Tufte. 

"Tables are preferable to graphics for many small data sets. A table is nearly always better than a dumb pie chart; the only thing worse than a pie chart is several of them, for then the viewer is asked to compare quantities located in spatial disarray both within and between pies – Given their low data-density and failure to order numbers along a visual dimension, pie charts should never be used." Edward Tufte, a prominent American statistician, noted why tables may be preferred to pie charts in The Visual Display of Quantitative Information:^[5]

The point here is best illustrated with the emoji pie chart angles getting lost you run out of subdued colours. 

Fabricated Trends

I take a graph of data it's real data. I then draw a line on it. I shift the line up a bit or down and that can massively affect that line. It is not necessarily even a lie if I play games with outliers either adding or removing them. 

Another method to lie with trends is easy to see if you look up a stock price and flip between a 1-week view and a 5-year view. Many of the trends can look quite bad when compared to the previous week but might not be that bad compared to that market collapse 5 years ago. 

There is a related error here in the extrapolation of trends where because a trend exists it is assumed that it will continue. This is why bodies of knowledge like ITIL often focus on SLA and KPIs as a definition of acceptable thresholds often helps to communicate when action is required better than trends which can be extrapolated.

Therefore if we combine these two effects by manipulating the trend and selecting the right timescale to show it over we can have a lot of impacts.

The 5-day trend looks good...but...

Does not look so great overall history... The above trading peak is nothing in the grand scheme of things.