Generative AI has changed the way we do almost everything, from finding sources for academic research to drafting that awkward email you’ve been putting off. (You know the one.)

Programming is no exception; AI-driven code assistants have transformed software development. AI coding tools automate repetitive tasks, accelerate development cycles, and boost developer productivity. They promise to make coding less boring for engineers while generating better, more secure code and finishing projects faster.

A win-win-win. 

The catch? It can be difficult for leadership to know whether code assistants are actually delivering those benefits — the effect of code assistants is tough to measure, leaving CTOs with very little baseline data about how much of an impact AI tools are having. 

The rise of the AI code assistant

The very first GenAI code completion tool was launched in 2018. Since then, many tools have been brought to market. Evidence suggests that those tools have been welcomed by engineers and organizations alike. One survey from 2023 found that 60% of CTOs and engineering leaders were actively rolling out AI coding assistants to their software teams, and that number continues to climb today. 

Here’s a quick look at some of the most popular code assistants:

• Amazon Q: Q is an extended version of Amazon’s CodeWhisperer. While CodeWhisperer provides code completion, Q also manages enterprise workflows and cloud operations. Q is best used by those working with AWS services and tools. 
• Codeium: Codeium is a fast alternative for coders who don’t have access to GitHub. It also works on-prem, so it’s a strong choice for those who are interested in privacy. 
• Cody AI: Cody AI’s claim to fame is that it’s deeply integrated with codebases, which makes it particularly useful for large-scale projects that need deep code searches. Cody’s ability to work with large codebases means that it’s able to generate code suggestions based on context gleaned from entire repositories, including proprietary and legacy codebases.
• Cursor: Cursor was built specifically for AI-assisted coding; it offers an AI chat inside VS Code, acting as a pair programmer. This also makes it easy for Cursor to explain and refactor sections of code. 
• GitHub Copilot: Launched by Microsoft in 2021, Copilot is one of the most widely used AI code assistants. It helps engineers by suggesting and autocompleting code, taking care of some of the more repetitive parts of programming.
• Google Gemini: Gemini is Google’s multimodal AI system that can generate, complete, debug, and explain code—even to beginners. It supports inputs like diagrams, screenshots, and text descriptions and is especially useful for building software within the Google and Android ecosystems.
• Poolside AI: Poolside AI is designed for modernization. It specializes in old programming languages, automating legacy code migrations, and converting old languages like COBOL, Fortran, and Java into Python, JavaScript, and modern frameworks. If you’re migrating legacy code or converting old code, Poolside AI is the assistant you’re looking for. 

Why measurement matters for AI adoption

Back when Bain & Co. conducted its 2023 survey about code assistant adoption, CTOs sang the praises of AI coding tools; 75% said their code assistants met or exceeded their expectations, praising AI for increasing speed to market, code quality, and decreasing costs. 

But in the last year or so, that tune has changed a little; two-thirds of leaders say they’re not satisfied with the insights they get about the impact of AI code assistants. Others say they’re not getting much data at all, or to borrow a phrase from Bain & Co.’s most recent research: “Many executives see software engineering as a black box. They don’t know where the money’s going.”

Being able to track metrics associated with AI adoption in software development is invaluable for business leaders. Without that data, leadership doesn’t know how much time is being saved, how much stronger the code is, and how much productivity has been boosted. 

What’s the solution? Use a reliable benchmark to measure AI code assist data.

How does Opsera track AI in development? 

Opsera simplifies DevOps measurement, delivering insights into AI’s impact on DevEx, productivity, ROI trends, and downstream value. By connecting all your DevOps tools across the entire software delivery process, Opsera gives your team a complete view of your software delivery process, allowing you to analyze the impact AI has on your development cycle and business goals. 

The platform allows your team to make baseline comparisons with data from up to a year in the past, letting you easily chart your organization’s adoption journey and make continuous improvements along the way. 

Leaders need to be able to see their most important DevOps metrics in a single pane of glass; let Opsera be that window. 

Learn more about the Opsera Leadership Dashboard and the organizational benefits of a unified view of DevOps.