DBT's Microsoft 365 Copilot Pilot — a UK Government AI Trial That Found No Productivity Gain

The UK Department for Business and Trade (DBT) piloted Microsoft 365 Copilot from October to December 2024, issuing 1,000 licences to UK-based staff, about 70% volunteers and 30% randomly selected. The pilot aimed to test the tool's risks and benefits before any wider rollout across the department. DBT published its evaluation findings in full on GOV.UK in August 2025.

The pilot set out to test Microsoft 365 Copilot's risks and benefits among DBT staff ahead of any decision on wider rollout. It combined quantitative usage tracking with qualitative feedback to assess both productivity impact and user experience.

The evaluation combined a diary study (312 diaries submitted, 300 consented to analysis, a 32% response rate), 19 qualitative interviews (13 pilot participants and 6 from a comparison group), observed-task sessions, and Microsoft's own usage-dashboard data. Word, Teams and Outlook saw the most use, while Loop and Edge usage stayed below 1% of licence holders.

Usage was thin, averaging 72 Copilot actions per user over the pilot, or 1.14 actions per user per working day. Satisfaction was fairly high, with 72% of users satisfied or very satisfied and a Net Promoter Score of 31, and written tasks like summarising and drafting saw the largest time savings, though scheduling and image generation sometimes took longer with Copilot than without it. DBT's report states it 'did not find evidence that time savings have led to improved productivity,' and the comparison group did not observe productivity gains among Copilot-using colleagues either. The evaluation also documented user-reported hallucinations and inconsistent quality-assurance practice across participants and task types.

DBT concluded that despite fairly high satisfaction and some time savings on specific written tasks, there was no evidence that these translated into measurably higher productivity. The department published this null result in full, including the finding that self-led training outperformed formal departmental training at building user confidence.