AI coding tools like Copilot from the stable of GitHub were advertised as game-changers. It was promised to boost productivity, reduce errors and make coding easier for developers. A recent study by Uplevel reveals a different story. The tools might not be living up to the hype yet. It is questioned whether AI coding assistants are more harmful than being helpful.
The Uplevel study followed 800 developers over three months period and revealed that something is surprising. Developers using AI tools like Copilot saw no significant improvement in productivity. Metrics like pull request cycle time and throughput remained unchanged.
The rise in coding errors has also been witnessed. Developers using AI assistants introduced 41% more bugs into their code compared to those working manually. AI should help in reducing defects by providing smarter code suggestions and reviews. Uplevel data analyst Matt Hoffman pointed out that the findings contradict expectation. The tools are not as ready for mainstream use as once thought.
Another big selling point was the potential to ease burnout. The developing sector is known for long hours and high stress. Many hoped that AI automation could reduce the mental load on developers. Uplevel found no evidence to support this. Developers reported feeling mentally drained as before and suggested that the tools are not lightening the load in meaningful ways.
However, some developers have found success with AI coding assistants. Travis Rehl, CTO of Innovative Solutions, claims Copilot tripled the productivity of his team. Ivan Gekht, CEO of Gehtsoft USA, pointed out that AI-generated code was often so complex that his team found it faster to rewrite from scratch.
AI coding is a revolutionary force or just overhyped novelties? The answer seems to lie somewhere in between. They may show promise, but they are clearly not a one-size-fits-all solution. While they might eventually transform coding workflows, for now, they appear to introduce as many challenges as they solve.
