Workers in many industries have begun using large language models (LLMs) like ChatGPT since its launch in 2022, enabling researchers to study the effects of AI on labor productivity. The results provide empirical evidence of the productivity-boosting effects of AI for many professional tasks including writing, coding, administrative tasks, text summarization, and research.
Some studies have looked at the impact of LLMs on writing productivity. Noy and Zhang (2023) conducted a natural experiment, recruiting college-educated professionals to complete writing tasks. showed that LLMs help writers. They found that participants who used ChatGPT produced faster and higher quality writing and enjoyed their tasks more than those not using it. Participants with weaker skills benefitted the most from using ChatGPT. Doshi and Hauser (2023), meanwhile, explored the impact of generative AI on writing creativity, finding that access to AI-generated ideas enhanced the creativity of stories and quality of writing—as judged by a group of readers—particularly for weaker writers. And finally, Chen and Chan (2023) found that LLMs could boost advertisement copywriting effectiveness when used to provide feedback on human-generated content.
LLMs also boost coding productivity. Yilmaz and Yilmaz (2023) investigate the effect of programming education using ChatGPT on students’ computational thinking skills, programming self-efficacy, and motivation. Splitting students into randomly assigned control groups, the authors found that students who used ChatGPT were significantly more effective coders and better computational thinkers. These findings are mirrored by another experiment by Peng et al. (2023), which measured the effect of GitHub Copilot, an AI tool that automatically suggests code. The study found that Copilot users completed the programming task 56 percent faster than those who did not. Ziegler et al. (2022) found similar effects in an experiment with 2,631 developers.
Scholars have also identified broader AI-induced productivity benefits across different white-collar occupations. For example, Brynjolfsson et al. (2023) analyzed data on 5,179 customer support agents using a generative AI-based conversational assistant and found that it increased productivity by 14 percent on average, with the most significant improvements occurring for novice and low-skilled workers. Meanwhile, Dell’Acqua et al. (2023) examined the impact of LLMs on the productivity and quality of work performed by 758 consultants in a randomized controlled trial at Boston Consulting Group (BCG). The authors found that consultants with access to GPT-4 were able to complete tasks significantly faster and with higher quality than those without access, particularly after workers were trained in prompt engineering. Workers using GPT-4 completed, on average, 12 percent more tasks, at 25 percent faster work speeds, with 40 percent higher work quality. Finally, Choi et al. (2023) conducted the first randomized controlled trial to study the effect of AI assistance on legal tasks. The paper found that access to GPT-4 did not consistently improve the quality of law students’ work but significantly increased their speed. These benefits again were most pronounced for lower-skilled participants.
Beyond this, a series of recent papers suggest that LLMs may improve information worker productivity across a diversity of office tasks. A randomized controlled trial by Microsoft economists found that workers who were given access to Copilot were able to complete tasks—such as email retrieval and meeting summarization—significantly faster than those without access, while maintaining a similar level of accuracy. In addition, Edelman et al. (2023) conducted two randomized controlled trials to evaluate the impact of Copilot on information worker productivity, finding similar statistically significant improvements in efficiency.
Policymakers should read preliminary evidence of LLM productivity-boosting effects as a promising sign and a reason to continue to support AI advancement and adoption. Collectively, the emerging field that studies LLM productivity-boosting effects offers increasingly auspicious evidence that AI is boosting productivity, particularly for less productive workers. Labor productivity growth is the best vehicle to boost standards of living, and AI’s potential to trigger such growth would be a welcomed development. However, additional studies will be necessary to understand AI’s broader impact on firm productivity and the economy.
* Julian Jacobs is a Google Public Policy Fellow with the Information Technology and Innovation Foundation. Outside of his work for ITIF, Julian is a PhD student specializing in comparative political economy. His research areas of focus include artificial intelligence, the political implications of technological shocks, inequality, debt, and polarization. He is a recipient of the Fulbright Scholarship, and he received his MSc from The London School of Economics and a BA from Brown University. Outside of academia, he has worked at the Office of Barack Obama, The Brookings Institution, Google DeepMind, the Center for AI Safety, OMFIF, and University College London.
Source: Center for Data Innovation
