In a research study at four Fortune 500 companies, when managers were asked about their teams’ work, on average they either did not know or could not remember 60% of the work their teams do. This is a major problem because it can lead to unrealistic digital transformation targets and the poor allocation of resources. But in the same study, machine learning tools were able to bridge the gap between manager intuition and reality. The study showed that employing ML algorithms reduced the average work-recall gap from ~60% to 24%. Managers should roll out such ML tools but take steps to ensure employees don’t feel surveilled — they can anonymize and aggregate data, and communicate openly with employees about what they are measuring and what they hope to achieve.
How much do managers know about how their teams work? We recently ran a research study involving 14 teams comprising 283 employees in four Fortune 500 companies. When managers were asked about their teams’ work, on average they either did not know or could not remember 60% of the work their teams do. In one extreme instance, a manager in our study could describe only 4% of their team’s work.
The cost of managers not knowing this gap exists can be high, even in teams as small as five members, and is therefore applicable to any company, big or small. Managers and key decision makers at all levels set digital transformation targets without sufficiently understanding how their teams get work done or where the pain points lie. Commonly, they resort to relying on guesswork to decide what investments will help their teams. Consequently, they systematically underestimate employees’ productivity or poorly allocate resources and investments in technology such as automation. Covid-19 and the transition to remote digital work have only made it harder for managers to understand how their teams are working.
But our study also showed that the problem is fixable using Machine Learning (ML) algorithms to learn from how teams use technology to do their work — as long as safeguards are put in place to protect employee privacy.
What We Found
In the study, we had managers teach a software system the processes they thought were taking up most of their teams’ time. Using an interface similar to the one people use when they tag photos of themselves on Facebook, managers executed samples of each process on their machines in the manner in which they expected their teams to do the work. They then tagged these processes under categories such as “order management”, “accounting processes,” and “supply-chain operations”. There was no limit on the number of processes a manager could teach the system. Managers relied on their intuition, judgment, and experience to shortlist and teach those processes that they believed occupied most of their teams’ efforts. This data was collected into a “work graph”, a map of how these teams get work done.
Using the manager-taught processes, our machine learning algorithms attempted to find similar patterns of work being done by the members of the team. We then measured the fraction of each team’s day in which the team members demonstrated similar patterns of the taught processes. This is, in essence, a measure for the extent to which a manager’s intuition accounts for a team’s daily work.
A key aspect of these studies was maintaining user privacy: we ensured that all tools and data collection anonymized the end-user, aggregated data to a team, and gave teams the tools to define and filter sensitive personal identifiable information. All analysis was performed only at the aggregated team level, without identifying any individual.
We assumed that in an ideal scenario, the manager ought to be able to account for at least 80% of their teams’ daily work — basing that nominal threshold on a survey we performed among managers, where we asked them to rate how much of their teams’ daily work they expected to understand. We define the work recall gap as the fraction of the team’s daily work that a manager could not account for, assuming a ceiling of 80%. This is also a measure for lack of completeness in a manager’s understanding of the work his/her team does daily.
We found a sizable work recall gap in all 14 teams — to the surprise of their managers — in functions ranging from supply chain operations, project management, customer interactions, master data management, finance/accounting, and HR.
An example helps illustrate the specific problems our study uncovered: In one company, the supply-chain team constantly faced complaints from employees about a poor enterprise-resource planning (ERP) implementation. Though technically correct and sufficient, the implementation lacked several features for processing data. As a result, for common transactions the employees were forced to spend time copying data from the ERP system into Excel, creating pivot tables, and iterating on the data. When they finally had answers, they copied the data back into the ERP system.
When this extra effort was added up across several transactions, it accounted for a large chunk of the team’s monthly work. Everybody on the team knew this was a problem to some degree; they felt the friction every day; but nobody understood how bad the situation was until we helped them measure the gap of their manager’s underestimation.
What Can Leaders Do?
The good news is that our study demonstrated that the work-recall gap can be closed with the use of Machine Learning (ML). In the study, we employed a class of ML algorithms that did not require any managerial input to detect patterns of the teams’ work. We excluded patterns that overlapped with the patterns described by the manager, and then measured the incremental fraction of a team’s day that could be accounted for using the patterns detected entirely by ML algorithms and no human input. Briefly, these ML algorithms find short bursts of repeated activities in a team’s work patterns. They then combine the most commonly occurring repeated activities to form a longer chain of activities. And repeat this process until they cannot combine activities any further.
We found that employing ML algorithms reduced the average work-recall gap in our study from ~60% to 24%. In the team where manager-described processes accounted for only 4% of the team’s daily work, ML algorithms were able to account for an additional 48% of the team’s daily work in productive activities (reduced the gap from 76% to 28%).*
Broadly, the algorithms did better than managers in our study for two reasons. First, managers had an outdated and/or incomplete view of their team’s work patterns. ML algorithms, by contrast, could find patterns without relying on pre-existing intuitions about what work is being done. Second, ML algorithms can scalably account for a multitude of ways in which the same work is done. We saw cases where the manager typically taught a few examples of how they thought the work ought to happen, but the team executed the same work in different ways from what the manager expected. For example, when performing a trade reconciliation, several experienced members of a team had found shorter paths for achieving the reconciliation and hence deviated from the prescribed standard operating procedures.
***
Without the use of machine learning tools to compensate, managers’ gaps in recalling their teams’ baseline work will likely only grow in the future given the trend toward remote work. And without intervention, managers are likely to remain in the dark about what they do not know — in our study, managers were routinely shocked when we revealed our results to them.
The future of any work environment — not just remote work environments — thus depends on equipping managers with new tools and techniques to understand and manage their teams more effectively. The use of such tools will require consistent and open privacy standards such as anonymity of users, aggregation of data, and consistent communication from leaders so that employees understand their intentions. Our entire study focused only on teams and did not permit identifying any individuals.
Our advice to change leaders and managers is to treat your team’s experience at work as data. Such data will likely reveal what ails your teams and what is realistically possible with investments in digital transformation and other new initiatives. Then all changes with the best-intentioned managers will be measurable. Conversely, in the absence of such data, top-down goals are set without the facts being known, and teams have little choice but to sign up for plans without understanding their implications, leading to immense pressure on teams. Our hope is that if managers understand more about the specifics of their teams’ work, they will set more realistic targets and help their teams become more productive.
* Editor’s Note: We have updated the text to correct the figures stating how much the average work-recall gap was reduced with the ML algorithms. The gap was reduced from 76% to 28%.
