Case study: BT rolls out Amazon’s generative AI developer tool to more coders
BT is rolling out Amazon’s generative artificial intelligence (GenAI) coding tool to 1,200 developers and said it has now helped write 200,000 lines of code.
The telecoms giant is making the Amazon Q Developer AI-powered assistant available to all of its coders, after starting to use the tool with smaller groups of its software developers late last year.
The tool provides real-time AI-powered code suggestions – from snippets up to full functions – in a number of integrated development environments (IDE), based on natural-language comments and existing code.
BT Group’s digital unit, responsible for leading digital transformation across the organisation, started using the GenAI tool last year, when it was called Amazon CodeWhisperer.
BT said that 100,000 lines of code had been created using the tool in its first four months at BT Group, and that it had automated around 12% of the tedious, repetitive and time-consuming work done by an initial group of volunteers.
When BT first said it was using the tool in back in February, it was providing 15-20 suggestions of code per active user per day, with an acceptance rate of 37% by its software engineers who are using the platform.
For comparison, at the launch of Amazon Q in April, National Australia Bank said its developers were accepting 50% of the code suggestions made by Amazon Q Developer, while a healthcare IT company said its team had seen a code suggestion acceptance rate of 35%. BT Group has now made the offering more widely available to all 1,200 of its engineers across the business, and said 200,000 lines of code have been written with the help of the tool.
Core reinvention
Deepika Adusumilli, chief data and AI officer for digital at BT Group, told Computer Weekly that BT’s philosophy in terms of AI was to let the experts do the job, so that might mean picking tools like Amazon Q rather than trying to invent it for themselves. “Where we invest in the organisation, how we use GenAI, how we build our models, what data we use, is going to be focused towards core reinvention,” she said.
“How do we invest towards that and what are the most meaningful things we do? All of our strategy so far has been underpinned by where we are going as a company.”
Adusumilli said the Amazon Q tool has seen “active engagement” from the developers.
“We absolutely could not move forward if the developers did not accept the tool,” she said. “We absolutely could not get here if we did not see positive outcomes. It cannot get to that number without positive word of mouth.”
Adusumilli said the quality of the lines of code has gone up, with fewer lines needed to get to the same outcome. “Everything we are talking about is giving wings to our team members,” she said.
BT said the tool was helping to remove time-consuming tasks and give developers more time for innovation. “We are absolutely getting that feedback,” said Adusumilli. “We’ve not had one developer say, ‘This is taking my job’. The feedback has been, ‘We are getting more productivity and we appreciate it – how can we do this more?’”
Full roll-out
Amazon Q is now being rolled out to all of BT’s developers, who are encouraged to use it wherever makes sense to unlock their productivity, although BT said it was not being prescriptive about where coders have to use it.
Amazon Q now supports 20 languages, and BT said its usage was most popular across languages including Java, Javascript, Typescript and Python. BT said where the tool really shines is if a project builds on other Amazon technologies, as it’s able to give very specific and relevant advice in those situations. BT is currently reviewing other features such as Amazon Q Developer Agents.
Amazon argues that developers spend 30% or less of their time on actual coding: much of it is taken up by repetitive tasks such as researching best practices from or learning how things work through documentation, as well as managing infrastructure and resources or troubleshooting errors. It said tools such as Amazon Q are able to remove some of that ‘coding muck’ and deliver code and features faster.
BT’s developer trainees don’t get access to the Amazon Q tool until they have learned the basic principles, because they need to learn their coding foundations to understand whether the recommendations coming from the tool are right or not. The most experienced developers have less need for the tool, so it’s the developers between those two extremes that seem to be getting the most benefit from having a coding assistant.
Preventing hallucinations
BT is also protected from the risk of GenAI hallucination because the coding is not automated – instead, the developer has to accept the code recommendations made by the tool, which should stop bad suggestions getting very far.
Beyond this, the code is going into the existing CI/CD process so there is already a well-embedded quality assurance and testing process, which means even if someone does accept the wrong recommendation, it gets spotted pretty quickly.
Tools such as Amazon Q for developers have been one of the early success stories for GenAI. These tools can help developers by providing code recommendations, but also by helping to complete some of the more boring jobs that can take them out of the coding flow. Tech analyst Gartner predicts that the three-quarters of developers will be using such tools by 2029. Probably the most widely known developer assistant is Github Copilot, though the market for these tools is getting increasingly crowded.
Developers are expensive and in short supply, so anything that can help make them more efficient and effective is likely to be attractive.
Amazon Q Developer has a free tier that offers code suggestions in the IDE and command line interface, while there is also an Amazon Q Developer Pro Tier that starts at $19 per month and includes additional features such as the ability to customise the tool to a particular code base.
