San Jose, California, is home to one of the most peculiar structures ever built: the Winchester Mystery House, a 160-room Victorian mansion that includes 40 bedrooms, two ballrooms, 47 fireplaces, gold and silver chandeliers, parquet floors, and other high-end appointments. It features a number of architectural details that serve no purpose: doorways that open onto walls, labyrinthine hallways that lead nowhere, and stairways that rise only to a ceiling.
According to legend, Sarah Winchester, the extremely wealthy widow of the founder of the Winchester rifle company, was told by a spiritual medium that she would be haunted by the ghosts of the people killed by her husband’s rifles unless she built a house to appease the spirits. Construction began in 1884 and by some accounts continued around the clock until her death in 1922. There was no blueprint or master plan and no consideration of what it would mean to reach completion. The point was simply to keep building, hence the sprawling and incoherent result.
Is the Winchester Mystery House a good house? It’s certainly beautiful in its own way. Any given room might be well proportioned and full of appealing features. A stairway might be made of fine wood, nicely joined and varnished, and covered in a colorful carpet. Yet it ends in a ceiling and serves no useful purpose other than keeping its builders busy. In assessing whether a house is good, we have to ask, “Good for what? Good for whom?” — the questions we would ask about other kinds of constructions. For the Winchester Mystery House, the act of building was an end in itself. It is a paradigmatic folly, “a costly ornamental building with no practical purpose.”
Is management research a folly? If not, whose interests does it serve? And whose interests should it serve?
The questions of good for what and good for whom are worth revisiting. There is reason to worry that the reward system in our field, particularly in the publication process, is misaligned with the goals of good science.
There can be little doubt that a lot of activity goes into management research: according to the Web of Knowledge, over 8,000 articles are published every year in the 170+ journals in the field of “Management,” adding more and more new rooms. But how do we evaluate this research? How do we know what a contribution is or how individual articles add up? In some sciences, progress can be measured by finding answers to questions, not merely reporting significant effects. In many social sciences, however, including organization studies, progress is harder to judge, and the kinds of questions we ask may not yield firm answers (e.g., do nice guys finish last?). Instead we seek to measure the contribution of research by its impact.
There are many ways to assess the impact of scientific work, from book sales (where Who Moved My Cheese? is the best-selling management book in history) to prizes to Google hits and article downloads. By far the dominant measure of impact is citations: how often a piece is cited in subsequent works. An advantage of this measure is that it is easily accessible: Google Scholar and Web of Knowledge are just a click away. Citation metrics are widely used in faculty evaluations and routinely come up in tenure reviews. By this accounting, good science is widely cited science.
But at a fundamental level, impact in this sense may not measure what we want. Consider what happens when police are evaluated according to their numbers of citations and arrests. We might imagine that as a society we want “safety” or “justice,” but if what we count when we evaluate police officers is “number of arrests and citations issued,” we get something rather different—in the worst case, entire populations weighed down with criminal records for trivial offenses. Similarly, it is unclear why “impact” is an apt measure if the goal of research is to answer questions. If anything, raising questions that do not get answered, or being surprising and counterintuitive, may be better strategies for being widely cited than actually answering questions accurately. Being provocative may be more impactful than being right.
What will the advent of big data mean for management research, given the incentives in our publication process I’ve described above? Ironically, it is possible it will make it even more difficult to evaluate researchers’ contributions. Data used to be a constraining factor for organizational research. Gaining access to sufficient data to yield statistically significant results was often difficult, which encouraged researchers to use widely available sources such as Compustat or archives such as ICPSR. Some of the most influential papers from the 1970s featured simple correlations and occasional regressions on cross-sectional data for modest samples, any of which might be rejected out of hand today. Yet the availability of endless data may paradoxically make things worse. Rewarding statistical significance and surprise in a world of big data could easily lead to a grid search for intriguing and counter-intuitive regularities that do not add up to a coherent understanding.
At the same time, it is long past time for the field to have a serious conversation where their data comes from, and the ethics of using big data. Social life and the operation of organizations increasingly leave more or less permanent data traces, like the contrails left by jets. Sometimes these become available to researchers; even more often, they are available to corporations and government agencies such as the National Security Agency. A preview of this is the Enron email archive, containing about one-half million e-mails from 150 senior managers of Enron, which was made public by the Federal Energy Regulatory Commission during its investigations and subsequently cleaned and shared by researchers. Researchers have mapped the networks created by “From” and “To” headers in messages, analyzed the prevalence of sentiments in the text, coded classification schemes, and more. The data provide a fascinating look into a corrupt organization in action. The 2014 hack of Sony Pictures Entertainment’s computers, allegedly by North Korean operatives, yielded scores of embarrassing e-mails and memos, as well as sensitive data on thousands of employees, including “a Microsoft Excel file that includes the name, location, employee ID, network username, base salary and date of birth for more than 6,800 individuals,” according to Time. For some researchers, that spreadsheet could have been an ethically questionable window into contemporary corporate practices. (For instance, journalists have already mined it to explore gender and race pay gap issues.)
Data on the whereabouts, productivity, compensation, demographics, social networks, emotional expression, and perhaps medical records and Fitbit streams of employees can yield horrifyingly intrusive information within individual organizations, and modest versions of these are beginning to appear in the literature. A few companies such as Google and Oracle have some of these data for hundreds of client organizations, potentially allowing unprecedented comparative data on organizational structures and processes. It is inevitable that variants of such data will end up in the hands of researchers, one way or another. Moreover, A/B testing of experimental and control conditions is now standard practice in technology firms such as Google and Facebook, from the mundane (testing which headline yields the most click-throughs) to the less mundane (examining how exposure to positive or negative updates influences one’s own expressed emotions or how knowledge of friends’ voting influences the propensity to show up at the polls). Experiments are nearly costless in this environment, in which informed consent is evidently optional.
This brings us back to the question of whose interests are served by business research. Traditionally, the ultimate constituency for organizational research was managers. Scholars were encouraged to conduct research with “managerial relevance” or possibly “policy relevance.” And as business corporations kept growing bigger during the 1960s and 1970s, the need for managers to staff their internal hierarchies led to a massive expansion in management education. The demand for managerially relevant research was evident. Yet beginning in the 1980s, changes in the economy were reflected in the kinds of jobs taken by MBA students. Instead of seeking management jobs at GM or Eastman Kodak or Westinghouse, MBAs from elite schools went into finance and consulting. Traditional corporations, particularly manufacturers, shrank or even disappeared through multiple rounds of outsourcing and downsizing, while the largest employers came to be in retail, where hierarchies within stores are relatively short. Meanwhile, information technologies increasingly turn the tasks of management (measuring and rewarding performance, scheduling) over to algorithms. There are nearly 7 million Americans classified as “managers,” but the content of their tasks may not involve the actual supervision of other people.
More recently, alternative business models have arisen that dispense with “employees” and “managers” entirely. Uber reported that it had 162,000 “driver-partners” in the U.S. at the end of 2014. These are not employees of Uber — which itself employed perhaps 2,000 people — but independent contractors without need for management. Amazon expands and shrinks by tens of thousands of workers at a time through the use of temporary staffing companies for its warehouses — it added 80,000 temporary workers for the 2014 holiday season. The tasks are straightforward and largely supervised by computer. Retail, fast food, and the “sharing economy” are increasingly moving to a world in which algorithms and platforms replace human management. Meanwhile, GM’s North American workforce has shrunk to under 120,000, Eastman Kodak has 3,200 U.S. employees, and Westinghouse has effectively evaporated.
Management of humans by other humans may be increasingly anachronistic. If managers are not our primary constituency, then who is? Perhaps it is each other. But this might lead us back into the Winchester Mystery House, where novelty rules. Alternatively, if our ultimate constituency is the broader public that is meant to benefit from the activities of business, then this suggests a different set of standards for evaluation.
Businesses and governments are making decisions now that will shape the life chances of workers, consumers, and citizens for decades to come. If we want to shape those decisions for public benefit, on the basis of rigorous research, we need to make sure we know the constituency that research is serving.
This has been adapted from an editorial essay titled “What Is Organizational Research For?” that originally appeared in the Administrative Science Quarterly, June 2015, Vol. 60.