The Eleventh HOPE (2016): Coding by Voice with Open Source Speech Recognition

While you were yucking it up, someone already did it. I'd say it takes about a minute: this video Of course, there's plenty of room for improvement. Why not have the assistant understand keywords and perform high-level tasks? "import" "create function named main" "return" "Add two int arguments"

It's nice and all, but this is a nightmare for accessibility. Not everyone can read paper or even move around. With this design, coding around those limitations is not easy. In the current model, everything could be done 100% accessibly. Why it isn't is a completely different problem. When everything is inherently physical, though, accessibility is not just about the code, it involves much more parts, such as automated devices to move things around. Considering how expensive devices made for accessibility are (a braille display is usually well over $1000), that would be a huge setback. I think the beautiful thing about computers and technology is that you're not tied to any physical medium and that information can flow freely between devices that can represent it in different ways. Now, there can be ten programmers working on a codebase, where one person uses a normal Windows box with a lot of GUI editors, another one uses Linux in text mode with vim, a third one has a Mac Mini with a screen reader and no monitor attached (my also blind friend programs like that), and the fourth uses switches, eye movement sensors or voice-recognition technology because they're physically disabled (see this video ). In other words, when everything is purely digital, you can consume and manipulate information in any form you like and in any form the computer can work with. It can be on a smartphone screen and with your finger, on a TV with a remote, through a voice-controlled speaker or a laptop

I have been lucky to not be impacted by hand or wrist pain. I did experience chronic back pain for years due to a pinched sciatic nerve, it can be debilitating and depressing. Chronic pain is a large precursor to suicide, if you have it, you need to get it fixed. My interest in this is from a FutureOfProgramming perspective, how can we interact with computers in different, higher order ways. What does the Mother of All Demos [0] look like in 2020? Travis Rudd, "Coding by Voice" (2013) [1] this uses Dragon in a Windows VM to handle the speech to text. This was the original, "termie slash slap", a kind of Pootie Tang crossed structural editing. Coding by Voice with Open Source Speech Recognition, from HOPE XI (2016) [2] Another writeup [3] that outlines Dragon and a project I hadn't heard of called Silvius [4] It looks like most of these systems rely on Dragon, and Dragon on Windows at that due to not having the extension APIs on Mac/Linux. Are there any efforts to use the Mac's built in STT or the Cloud APIs [5,6]? [0] Mother of All Demos link [1a] link [1b] link [2] this video [3] link [4] link [5] link [6] link