During a recent demo session with a group of backend developers, I had the opportunity to showcase how GitHub Copilot can jumpstart your testing workflow. In this post, I’ll walk you through the key takeaways from the session, focusing on how Copilot can help generate unit tests.
Setting the Stage: A Real-World Context
The demo was framed around a real-world scenario at Voyantic, a company specializing in quality control systems for RFID tag manufacturers. Their setup typically involves:
- Multiple Tag Readers & Marker Devices: Hardware components that perform the testing as well as mark any faulty tags.
- Central Control Unit: Server unit responsible for configuring and coordinating the different components.
With this context, we can explore test generation in a realistic setting, making the insights more applicable to everyday development challenges.
Copilot for Unit Testing: A First Look
For the initial demonstration, I focused on a simple timer implementation. Instead of relying on JavaScript’s setInterval, this timer class uses setTimeout and takes into account any potential drift—ensuring that exactly four triggers occur each second.
If you’re looking to speed up your testing process, give Copilot a try.
To generate the tests, I used Copilot in edits mode, providing the relevant files in the working set. I started with the following prompt: Write unit tests for the TriggerTimer class using jest.
Key Observations:
- Quick Start: Copilot generated several test cases rapidly, providing a basic structure that covered various aspects of the timer’s behavior.
- Verification is Crucial: Although the tests looked reasonable, they weren’t all passing out of the box. Automated test generation can give you a head start, but you must always review and adjust the results to ensure they correctly reflect your intentions.
Enhancing Test Generation with Detailed Prompts
The demo then moved to a more complex scenario. I introduced a class responsible for monitoring tag results and counting how many tags are produced each second. Here, the goal was to ensure that incoming timestamps were correctly processed, and that the system dropped outdated data as time progressed.
This time I approached the test generation a bit differently. I had already prewritten the titles for a couple of test cases I wanted to cover, and asked Copilot to implement these test cases. I also used a prompt that was more detailed than last time.
This approach produced more accurate test scaffolding:
- Incremental Improvement: Focusing on one test case at a time allowed for inspecting each test case once at a time.
- Controlled Test Generation: By prewriting part of the test case, we ensured more predictable outcomes. This approach not only offered greater control but also forced us to think through the edge cases more carefully.
- Tailored Prompts: The extra context in the prompt resulted in tests that aligned more closely with the intended behavior of the system. However, validating the output is still critical.
The takeaway here is that while more detailed prompts require additional effort, they can significantly improve the quality of the generated code.
Copilot for e2e Testing: A Mixed Bag
The final example was a simple end-to-end (e2e) test for a POST endpoint used to start a tag production run. Given the complexity of API specifications and the additional challenge of integrating with physical devices in a test environment, Copilot’s performance here was somewhat underwhelming:
- Structural Competence: The generated test provided a basic structure that would be expected from a human-written counterpart.
- Limited Depth: However, it was generic and lacked the specificity required for handling the intricacies of real-world e2e scenarios.
This example reinforced the idea that Copilot is best suited for generating small, focused snippets of code rather than comprehensive test suites for complex endpoints.
Key Takeaways
Unit Tests Shine with Copilot
For quick scaffolding and simple test cases, Copilot proves quite useful. It can reduce the “blank screen” syndrome and kickstart the test-writing process.
Prompting is Critical
The quality of your initial prompt can make a significant difference. Investing time in detailing what you expect can lead to more reliable outputs.
Iterate and Validate
Always review and iterate on the generated tests. While Copilot gives you a robust starting point, it’s essential to tailor tests to cover all necessary edge cases.
Final Thoughts
GitHub Copilot is a powerful assistant for backend developers, especially when it comes to generating unit tests. It offers a great way to overcome the inertia of starting from scratch, but like any tool, it works best when combined with a developer’s insight and iterative refinement.
If you’re looking to speed up your testing process, give Copilot a try—but always ensure you verify and enhance its output for a comprehensive test suite.