evidoria

← Back to browse

Good practice

GovTech Singapore's Engineering Productivity Programme — A Controlled GitHub Copilot Pilot for Government Developers

Singapore · Singapore · See the Singapore profile

Over four months, GovTech Singapore's own research team measured GitHub Copilot's effect on 70 government software engineers using IDE telemetry plus SPACE-framework surveys: coding time fell 21–28%, junior developers gained most, and the team published every caveat openly.

70
Developers in the pilot (Oct 2023 - Jan 2024)
21-28 %
Reduction in coding time (range) (Oct 2023 - Jan 2024)
22 %
Reduction in coding time (average) (Oct 2023 - Jan 2024)
21 %
Rise in task-completion speed (Oct 2023 - Jan 2024)
24 %
Estimated overall productivity gain (Oct 2023 - Jan 2024)
33 %
Reduction in coding time for junior developers (Oct 2023 - Jan 2024)
13-15 %
Reduction in coding time for senior/lead developers (Oct 2023 - Jan 2024)
95 %
Respondents who said the tool let them focus on more satisfying work
70+ %
Respondents who observed fewer coding mistakes
90 %
Respondents who wanted their department to keep providing the tool
~12 %
Overall productivity impact once non-IDE time is accounted for
670+
Government systems underpinned by the SHIP-HATS platform
85,000+
Monthly deployments on the SHIP-HATS platform

Details

Maturity
Pilot
Promoter
Government Technology Agency of Singapore (GovTech) — Engineering Productivity Programme, Government Digital Products division
Period
October 2023 – January 2024 (pilot); published September 2024
Keywords
AI coding assistant, GitHub Copilot, developer productivity, DevSecOps

Context

From October 2023 to January 2024, Singapore's Government Technology Agency (GovTech) ran a controlled pilot of GitHub Copilot for Business with 70 developers under its Engineering Productivity Programme (EPP) and SHIP-HATS DevSecOps platform, which today underpins over 670 government systems and 85,000+ monthly deployments. Participants were mostly junior and mid-level software engineers, reflecting roughly three-quarters of GovTech's engineering population.

Objectives

Measure GitHub Copilot's effect on government developer productivity using a mixed-methods approach combining objective IDE telemetry with a survey built on the SPACE productivity framework.

Activities

GovTech's own researchers combined IDE telemetry (prompt-acceptance rates, accepted lines of code) with a SPACE-framework survey of the same 70 developers over the four-month pilot, comparing coding activity before and after Copilot access rather than against a separate control group of non-users.

Results

The study measured a 21-28% reduction in coding time (averaging 22%), a 21% rise in task-completion speed and an estimated 24% overall productivity gain — with junior developers seeing the largest effect (33% less coding time) and senior/lead developers the smallest (13-15%). 95% of respondents said the tool let them focus on more satisfying work, over 70% observed fewer coding mistakes, and 90% wanted their department to keep providing it.

Conclusions

The published study is notably candid about its limits: overall productivity impact drops to about 12% once time spent outside the IDE is accounted for; the 21-28% gain trails industry claims of 30-100%, which the authors attribute to limited usage under Singapore's in-country data-residency rules; over half of respondents saw no reduction in code-review time; and senior engineers worried Copilot's occasional inaccuracies could entrench poor habits in junior colleagues. The pilot's small size (70 developers, 40 survey respondents) and single-agency scope limit how far the results generalise.

Implementation
Indicative cost
Medium (€50k–€500k) — Not itemised in the source; involves GitHub Copilot for Business licensing for the participating developer cohort plus GovTech's internal research staff time; no separate control-group cost.
Time to results
Short (< 1 year) — Pilot ran four months, October 2023 to January 2024; findings published September 2024.
Staffing & skills
70 GovTech developers, mostly junior and mid-level, drawn from SHIP-HATS users, GovTech's own Engineering Productivity Programme research team

Conditions for success

  • Pre-existing SHIP-HATS DevSecOps platform and IDE telemetry instrumentation
  • SPACE productivity-framework survey methodology
  • Voluntary, informed participant sign-up

Common failure modes

  • Measured gains (21-28%) trail industry claims of 30-100%, attributed to limited usage under Singapore's data-residency rules
  • Over half of respondents saw no reduction in code-review time
  • Senior engineers worried Copilot inaccuracies could entrench poor habits in juniors
  • Small sample (70 developers, 40 survey respondents) and single-agency scope limit generalisability

Where it fits

Governance type
national government technology agency
Scale
single-agency pilot (70 developers)
Income level
high-income

Data sources

Where this practice's information was retrieved from, and when.

Similar practices you may find useful