GitHub Copilot at Salsita: An In-depth Analysis

Right when GitHub Copilot was introduced, two camps of people emerged: those who predicted developers would be extinct as a profession within a year, and those who believed it's useless, produces gibberish, and costs more time than it saves. Which group do I fall into? Neither of them. I actually think that both camps are mistaken, and not just because they represent extremes.

GitHub Copilot is a tool, and like any other tool, it requires learning. You need to understand its strengths and limitations to utilize it to its full potential. Sadly, the internet is often dominated by the loudest voices, leading to a proliferation of articles from both extremes, most of which are highly speculative:

• Proponents often set the bar quite low, focusing on academic use-cases rather than real-world scenarios.
• Critics simply bash the tool for producing subpar results in areas where it's not particularly strong.

In my presentation, I aim for a more balanced view of the tool and its capabilities, drawing from my personal experience over the past few months. Instead of blind praise or undue criticism, I want to guide people on how to best use the tool. They can then decide its value for themselves.

<beware of spoilers below>

I begin by introducing the tool and explaining its mechanics, which should help set the right expectations regarding its limitations.

The next section shares my perspective on its strengths and weaknesses, or more specifically, what it's adept at and what it struggles with. I highlight the distinction between using Copilot as a solution for complex coding challenges and simply as a typing aid. GitHub Copilot excels in the latter but often falters in the former. I also touch on some notable shortcomings, such as outdated information and the limited size and accuracy of the context it derives from your codebase.

Perhaps the most entertaining part of my presentation is the live demo, where, unfortunately, some rehearsed scenarios don't go as planned. This demonstrates the unpredictability of the tool, as results can vary based on numerous factors. I showcase how Copilot uses context to suggest tailored snippets and emphasize the importance of clean, readable code for the tool, suggesting it's more effective than relying on comments alone. I also discuss two of my favorite applications: auto-generating documentation blocks and crafting meaningful error messages—two tedious tasks that are now streamlined. I had planned to illustrate how one can effortlessly produce significant sample data, but the tool didn't cooperate during the demonstration. My attempt at playing with unit tests also veered off course, though it did exhibit some reduction in boilerplate code.

After the less-than-stellar demonstration, I wrap up the session, giving Copilot perhaps more praise than it might seem to deserve after its recent performance. I conclude with a glimpse into the upcoming features of Copilot X and other related initiatives.

Truth be told, my live demos have never been this challenging, so be sure to watch and enjoy the spectacle at my expense.