Speak, Code, Deploy: What if voice was your primary tool for coding?

In an era where natural language interfaces quickly become integral to user interaction, the developer community is on the brink of a transformation that could reshape coding. At the forefront of this movement is Sami Ekblad, Developer Experience Lead at Vaadin, whose recent talk at We Are Developers explored the potential of voice-driven development.

Sami’s session, “Speak, Code, Deploy: Transforming Developer Experience with Voice Commands,” provided a glimpse into a future where your voice, rather than just your hands, might be the primary tool for coding.

Sami started pondering why many helpful accessibility features, such as voice commands and swipe gestures, are often underutilized by developers, even though they could significantly enhance the coding experience:

Don’t think of accessibility features as just for accessibility—they are just features. They are helpful for everyone.

From buttons to voice commands

Sami started experimenting with voice commands as part of his efforts to improve the developer experience. Developer surveys showed that their biggest frustrations were waiting times during compilation and deployment, interruptions, and distractions. Ekblad suggested a faster, more engaging development cycle that uses voice commands to tackle these issues.

Sami demonstrated how voice commands could streamline the “develop, deploy, test” process, focusing on UI development. He used tools like PicoVoice for wake word detection, the Web Speech API for converting speech to text, and large language models (LLMs) for code generation.

Sami basically showed how developers could speak their code and instantly see the results in real time. To be more precise, Sami showed how a simple voice command could create a button with dynamic functionality, such as changing its color on click while maintaining contrast.

It’s so cheap to test. You can experiment with things that you wouldn’t try otherwise.

Photo: Marin Pavelić

Challenges and limitations

Sami was upfront about the difficulties associated with voice-driven development. The effectiveness of voice commands depends on a quiet setting, precise speech recognition, and the speed of the underlying language and linguistic models (LLMs). Although LLMs are getting better, their speed, particularly in local environments, continues to be a bottleneck that requires more work:

You need a quiet place for this to work. The limitations of the voice command technology are quite apparent in noisy environments

Additionally, Sami emphasized the importance of managing context within the development cycle, as the AI might persist in a previous context, leading to unexpected results if not properly supervised. For instance, if a developer commands the AI to create a button but continues to issue commands in the same context, the AI might inadvertently modify or duplicate elements from earlier tasks.

So, is the future of coding in voice commands?

Sami wants to encourage developers to reconsider the importance of accessibility features, proposing that they should be viewed as indispensable tools rather than specialized options. He also advised the audience to streamline their development processes for faster and more enjoyable experiences, creating a more engaging overall process.

The key point from Samis’s presentation is that voice-driven development, although still in its early stages, can significantly boost productivity and creativity in coding. By incorporating voice commands into the development process, developers can explore more freely, minimize interruptions, and, most importantly, make coding feel more natural and intuitive:

Voice-driven development makes coding a more interactive and dynamic experience, bringing new dimensions to how we approach coding taskss

Sami recommended delving deeper into the possibilities of AI and voice in development by exploring additional talks and demos, such as those by Red Hat, for those interested in exploring some new ways of coding.