Tag Archives: Eric Abrahamson

About 2396 days ago Computer Security

The Dashboard Dilemma

When the Boston Red Sox reversed their curse in 2004 by vanquishing the Yankees and going on to take the World Series, many fans and pundits were quick to give much of the credit to management’s decision to enlist sophisticated computerized analyses of player performance data to make staffing decisions. This year, the team’s all-too-familiar collapse left these same observers wondering how the numbers could have led the Red Sox astray. Einstein kept a sign in his office that read, “Not everything that counts can be counted, and not everything that can be counted counts.” Baseball fans are not the only ones being forced to consider that the best decisions aren’t necessarily the ones based on analyzing reams of data. Companies of all sorts are setting themselves up for the same hard lesson, thanks to the growing excitement over technology’s ability to place all manner of salient data at the fingertips of managers. Armed with so-called dashboard displays on their PCs, CEOs can effortlessly summon up a cornucopia of performance indicators. Call up this week’s sales by product line, throw in profit by region, cost per widget produced, and change in inventory levels. Compare it all with last week, with the same week last year, and with forecasts and goals. And there you have it: a comprehensive guide to the organization’s performance and to what you need to do to improve it. If only it were that easy. The fact is, this sort of data worship can provide a distorted, misleading view of what’s going on, and it can lead to flat-out bad decisions. Part of the problem is that some of the most important inputs aren’t easily quantifiable. (Einstein used to keep a sign in his office that read, “Not everything that counts can be counted, and not everything that can be counted counts.”) Adam Galinsky, an associate professor at Northwestern University’s Kellogg School of Management who studies decision making, notes that a bias toward hard data causes organizations real harm. “Things that are hard to quantify tend to get left out of the decision-making process,” he says. For business owners, there are all sorts of intangible or hard-to-quantify factors that can mean the difference between life or death: employee morale, the emergence of new technologies, changes in the competitive landscape, evolution in customer tastes. Think of executives at Ford (NYSE:F) and GM (NYSE:GM) endlessly twiddling with the easy-to-track impact of rebates, advertising, and health care costs, while remaining oblivious to the cultural and political forces–forces that helped trigger higher gas prices–that spelled doom for their obsession with SUVs. This isn’t a new problem. Managers have always had a misplaced confidence in numbers. But the ability to do more and more with the data in an increasingly high-tech fashion is making things worse. Indeed, the availability of slick new data-crunching tools, and the hype they’re receiving, leads executives to rely on them more. Good managers know the nonnumerical stuff matters, of course, but it sure must feel as if you’re being highly effective when you can dash off a few e-mail notes and then in a few days or even hours watch those real-time graphs leap skyward on your screen. And, of course, companies can’t help but treasure these data tools, given that they’ve spent a fortune investing in them, egged on by consultants and other experts. Thus, the CEO risks becoming Nero, fiddling with his keyboard, as it were, while the company burns. The problem goes beyond neglecting the intangibles. Sometimes the data that goes into a dashboard is incomplete or biased in subtle but significant ways that can lead a manager astray. Jorge Grau, the CIO at cell-phone-tower operator SBA Communications (NASDAQ:SBAC) in Boca Raton, Florida, helped build a system to place virtually any sort of real-time report in the hands of managers. But there have been occasions when managers, delighted by the sheer number of facts at their fingertips, were oblivious to problems with the data. “We’ve been bitten,” Grau says. In several cases, managers were making decisions based on summary performance for the entire company, not realizing that those numbers did not include data from a few key regions. When data is misread this way, it can lead to bad calls on anything from budgeting to promotions. “Reports can do more harm than good,” Grau says, which is why he now cautions managers about accepting a slick table of figures at face value. Even when the data is complete, there are plenty of ways to misinterpret it. One temptation for managers is to assume that when two sets of numbers go up and down together, changing one will lead to a change in the other–inspiring a CEO, for example, to order up a series of costly facility upgrades simply because the last two facility investments happened to be followed by sales spikes. “You can find correlations between the most improbable things, if you look for them,” says Luca Rigotti, a decision sciences researcher at Duke University’s Fuqua School of Business. “You’ll just end up doing silly stuff.” In fact, there’s a range of potentially costly psychological tricks that managers can inflict on themselves in the face of perfectly solid data, says Northwestern’s Galinsky. For example, the same set of numbers can provoke wildly different decisions depending on the order in which the numbers are presented. That’s because people tend to jump to conclusions based on the first numbers they hear, and they don’t allow the later figures to change their minds. The same numbers can also have different impacts depending on how they are framed–for example, stating that 10 percent of the customer base will defect if a product is changed, as opposed to stating that 90 percent will stay. “Managers need to protect themselves against falling for those sorts of biases,” says Galinsky. Even if you avoid false correlations and biases, you’re still in danger of getting smacked by the law of unintended consequences when you try to manage by dashboard, says Christopher Zappe, who studies decision sciences at Bucknell University. “People tend to respond to falling numbers with a knee-jerk reaction to try to halt the fall,” Zappe says. “But it often ends up aggravating the situation because they often don’t really understand the ways in which the different things going on at an organization affect each other.” Thus, the manager who sees the on-screen sales graph start to droop might order sales reps to increase the number of prospects they contact, not realizing that sales are falling because the reps are already stretched too thin. Or a jump in inventory might lead to a decision to slow production, even though much of the inventory is set to go out the door. None of this is to say that dashboards are the enemy here. Indeed, CEOs probably get into more trouble when they fail to bring enough data into the decision-making process. And companies that sell dashboard-related systems are themselves quick to point out some of the pitfalls of data-driven decision making. “The concept of data quality is a very unsexy part of our business,” says Kendall Collins, vice president of product marketing for Salesforce.com (NYSE:CRM). “But without it you’re not going to have the ability to make decisions or react in an intelligent way.” Dashboards also can be modified to deal with the problems. Howard Diamond, CEO of ePartners, an Irving, Texas, consultancy that installs information systems that often include executive dashboards, notes that even intangible aspects of a business can be brought into a dashboard if companies are clever. For example, part of the Cool Cuts 4 Kids chain of hairstyling shops, an ePartners client, wanted to track trends in customer satisfaction. The company’s system will chart the number of customers who fail to return within 10 weeks. But part of the key to success also has to be to retain a certain amount of healthy skepticism about the value of the information we get from our computers, no matter how slick or costly the system. That, and not trading Babe Ruth to the Yankees. Contributing editor David H. Freedman (whatsnext@inc.com) is a Boston-based author of several books about business and technology. His latest, A Perfect Mess, co-authored with Eric Abrahamson, will be published by Little, Brown in January.

About 2427 days ago E-Commerce

Mistakes Were Made

I’m a licensed pilot, and I once managed to land a small plane on the wrong strip at a small airport. I could have gotten in pretty big trouble for it, so you might have thought my best move would have been to keep my mouth shut and hope the FAA wouldn’t find out. But I couldn’t wait to report my screwup to the government. I wish I could say it was because I’m such a conscientious fellow. But the truth is, I fessed up fast because the U.S. government rewards pilots for quickly owning up to their mistakes, agreeing to waive punitive action if they report themselves. In fact, most pilots carry the self-reporting form with them in their flight kit, just in case. Why the free ride for confessors? Because the government wants pilots to learn from one another’s mistakes in an effort to keep accident rates low. Indeed, the flood of self-reports–nearly 3,000 a month on average, including confessions from air traffic controllers, aircraft mechanics, and flight attendants–are gathered by NASA and selectively published in a monthly newsletter, “Callback,” that’s a must-read for 150,000 pilots and other aviation personnel each month. In the August issue, for example, a commercial airline pilot admitted that he failed to position the flaps at the back of the wings correctly at takeoff, nearly causing the plane to plunge. The pilot of a business jet described how he didn’t hit the right buttons for the cabin pressurization system, which could have left crew and passengers starved for oxygen at 22,000 feet. Neither pilot was fined or punished. You can’t stamp out mistakes at your company, no matter how good a manager you are or how brilliantly you hone your processes. Essentially, you can do one of two things: create an environment in which mistakes are seen as shameful and counterproductive; or create one in which each mistake becomes a collective learning experience. To me, the choice seems obvious. Do employees at your company point out their own screwups–even the ones that might not otherwise have been discovered by management? Probably not. But let me suggest an easy way to fix the situation: Blog it. We’ve heard a lot about blogs over the past few years, and it’s clear that corporate America is in on them. Many corporate blogs are sanitized, public-relations-oriented affairs intended to create bonds with existing and potential customers. Others serve as internal message boards to keep employees up to date. But I’m proposing something else: a blog that encourages employees and managers to tell their peers what they themselves have done wrong. It’s an easy step that could quickly effect a large, positive change in your corporate culture. Such blogs are rare. But they do exist. The Mayo Clinic in Rochester, Minnesota, for example, has set up a system in which medical residents electronically log their mistakes or any other problems they see so the hospital can analyze the errors and look for fixes. It’s more of a database than a blog, but any resident can post to it without fear of recrimination. (The residents identify themselves on their reports, but are only publicly named if they’re getting credit for coming up with a solution to a problem.) “The residents are in the trenches seeing what’s going on, and they pick up on things we may not be aware of,” says Furman McDonald, a faculty member in internal medicine. “We’re trying to get across that people can present their mistakes in a forum that won’t leave them feeling chastised.” Because residents are promised that what they confess to the Mayo stays at the Mayo, the clinic wouldn’t share an example of a confessed mistake. (We may be better off not knowing.) But feedback from residents has led to some significant changes at the clinic. It’s improved the gathering of information about different drugs prescribed by different doctors to the same patient to avoid problematic drug interactions. It has also enabled doctors to do a better job of picking up on obesity-related complications and created a safer way to deliver some medicines to diabetic patients. The U.S. Department of Energy, meanwhile, is pushing its contractors to gather and share goof-ups via so-called “lessons learned” databases accessible to employees at all of its contractors. Fluor Hanford, an engineering firm that is tackling one of the world’s toughest environmental cleanup sites in Hanford, Washington, has been particularly enthusiastic about getting its employees to fess up. Tony Umek, Fluor Hanford’s vice president for safety and health, says that the database led to the discovery that a number of employees were going around standard company purchase procedures to buy their own electrical equipment–and that some of the equipment was flawed. Everyone involved was thanked for their honesty. “The best people make mistakes, and that’s a fact of life,” says Umek. “We want our people to see the mistakes coming and report them, so we don’t have to lament a debilitating injury after the fact.” Of course, the culture of hiding error to avoid blame and punishment is ingrained in most of us from an early age and perpetuated in companies because managers tend to whack employees who mess up. Managers themselves don’t go around blabbing about their own errors because they want to be seen as role models for the highly effective. Some organizations, including the U.S. Marines and aerospace maverick Burt Rutan’s Scaled Composites, have created cultures that celebrate honest mistakes and confession, but they are exceptions and are not easily replicated. Deep down, people want to confess their mistakes. And that’s more true than ever in the Internet age. And this is where the specific appeal of piggybacking on new blogging technology comes in. It’s fine to include an error-confession session in meetings, on databases, and on suggestion slips. But blogs are tailor-made for sharing screwups because they’re conversational, intimate, low key, and usually easy to read or post to. It’s also a perfect format for small companies. A blog with, say, three or four posts a day is a lot easier to maintain than one with 60 posts a day. Blogs are cheap–many blogging tools are free–and can be set up in a few hours. A confessional blog can provide real benefits in a matter of weeks, if not days. As great as we are at learning from our own mistakes, you can learn almost as much from the mistakes of others. Don’t take my word for it. Research recently published in Nature Neuroscience by a team of Dutch scientists indicates that “similar neural mechanisms are involved in monitoring one’s own actions and the actions of others.” In other words, watching your colleague blow a sale has much the same effect on your brain as hearing yourself do it. Afraid someone will leak the contents of your confessional blog to the public? As the saying goes, just hope the blogger spells your company’s name right. Any customer of yours that stops doing business with you because it is shocked–shocked!–to learn your employees make mistakes on occasion probably was pretty precarious to begin with. More likely, your customers will admire you for your candor. (I’m assuming here that your employees do not regularly make mistakes of the criminal variety.) I also believe that deep down, people want to confess their mistakes. And that’s more true than ever in the Internet age. Sure, much of what you see in online forums is whining and blaming, but increasingly, personal bloggers are coming forward to talk about how they screwed up–and often in business situations. These what-I-did-wrong tales so far are disproportionately focused on software projects, Web businesses, and, fittingly, blog development because that’s where you find people who are most comfortable sharing information online. But why wouldn’t it work just as well for an insurance company or building contractor? My only specific recommendation: Encourage everyone to keep it breezy, even fun. Believe it or not, the NASA aviation reports are almost always framed in a humorous light, and they’re a blast to read. The hardest part, of course, will be sticking to your promise to not bring down the hammer on those who confess to good-faith mistakes. That’s not to say you need to let everything slide. Make it clear that confessions of malfeasance, real damage, and repeated offenses don’t merit get-out-of-jail-free cards. (The FAA gives pilots one freebie every five years, and there’s no amnesty if a crime is committed or something gets smashed up.) Not everyone will be happy about where you draw the line, but it’s easy enough to draw it and to make clear that anyone who doesn’t cross it will truly be appreciated for sharing. By the way, my flying mistake apparently wasn’t deemed instructive enough to make the newsletter. However, a warning was added a bit later to the aviation map for that airport to alert pilots to the very mistake I had made. I like to think of it as my contribution to aviation. Contributing editor David H. Freedman (whatsnext@inc.com) is a Boston-based author of several books about business and technology. His latest, A Perfect Mess, co-authored with Eric Abrahamson, will be published by Little, Brown in January.