Introduced a year ago, Github’s programming assistance software is now open to everyone. There are still many unanswered questions about its capabilities.
“ Your programming partner is here “, announced Github in a press release on June 21, 2022. It is not just any partner, but Github Copilot, a programming assistance software powered by artificial intelligence. Its promise is as simple as it is revolutionary: Copilot could help developers to code by offering them lines of code on its own.
A year after its presentation to the public, Copilot, which was developed by the platform specializing in hosting Github and OpenAI code projects, is available for everyone. Copilot was until now reserved for testers: more than 1.2 million developers had registered and, for nearly a year, evaluated the prowess of the software.
Copilot can generate complex algorithms
Copilot works in a simple way: “ As you type lines of code or comments, Copilot suggests the next line says Github. ” It’s not just a single word or line: Copilot can suggest complete methods, standard code formulas, entire paragraphs of tests and even complex algorithms », Promises the press release.
The promises are impressive, and on the site, the examples are just as impressive: you can see the software complete alone and very quickly several lines of code to automate the extraction of tweets, and this in several languages. Copilot would also share code recommendations based on the context, and even the programming style of the project, says Github.
But the arrival of Copilot would not necessarily be a good thing. Last year, when it was presented, the developers were divided on the software’s potential. Some felt that it was a small revolution, which would make it possible to code better and faster, while others raised many criticisms. Lack of transparency on the provenance of the code, low quality content, and possible copyright issues… the list of complaints collected by Numerama was long.
“Copilot does not write perfect code”
Has Copilot improved on these points? Hard to say. The software overview page does not offer much detail. To find out more about the source of the code, you have to go to the bottom of the project’s presentation page, where it is simply indicated that the AI was “etrained on public source codes, or from repositories Public Githubs “. The question of code in Copyleft, raised by Numerama last year, therefore still has no answer. It is nevertheless fundamental: the code under the Copyleft license can be reused for free, but only on condition that the project for which it is used is, in the end, also accessible for free.
Github assures that this would not be the case: “ Most of the code Copilot suggests has never been seen before “, is it explained. ” Our internal research shows that a suggestion may contain unchanged code snippets in only 1% of cases. However, if Copilot offers unmodified copyleft lines of code, this would raise serious questions.
As for the quality of the code proposed by the AI, the answer is not clear either. Github recognizes that ” Copilot does not write perfect code », and that the software « doesn’t test the code it suggests, so it won’t always work, and won’t always make sense “. It is even explained on the site that the software can offer passages “ old and obsolete under certain conditions, and that it does not work equally well on all programming languages.
But it would seem however that, despite everything, Copilot works. According to the site’s data (to be taken with a little hindsight, therefore), on average, users would accept 26% of the lines of code offered by Copilot. And in files where the software is activated, Copilot code represents 27% of the content — a figure that rises to 40% in Python projects.