Jump to content
Sign in to follow this  
=VG= SemlerPDX

Elite: Dangerous - Perfect Excuse for Voice Control in a game...

Recommended Posts

So, I've been playing this Elite: Dangerous Space Sim that models our real galaxy pretty well, has good physics, textures, and the default controls are so concise and intuitive that mouse and keyboard are easy, but the X52 is the actual flight stick in the cockpit of these ships. With it, and perhaps a head tracker to negate the need to slave one of the X52 hat switches to POV, it is a near perfect melding of man and machine (or game, in this case). So much so, that one doesn't feel the need to use multiple modes (green/orange/red) on the stick to cram all the keybindings onto the stick - it's pretty well HOTAS to the core if you merely select the default joystick option under "controls", but it gets much deeper...


After watching many YouTube vids for training and tips on this game, I noticed some of them "talking" to their ship. Being a Star Trek geek at heart, I had to investigate, and I talked with the most watch pilot using that on Twitch. His ship "a.i." is Amy, and using a variety of pre-programmed commands and responses, "she" gives the illusion of an artificial intelligence operating autonomously. It is a great program, one of the best I've seen on Twitch, but I looked into the software behind it, and became even more intrigued.


Voice Attack is a simple GUI (free to try for 21 days, or $8) to work with Windows Speech Recognition Module and give simple function editing to the less than literate computer users that want to use speech to activate button presses on their computer.
First of all, it's way awesome to just say out loud, "Start Fraps" or "Close Fraps", or "Start Recording" or "Take a Screenshot" without having to bind any keys to my flight stick, or go press a button. Simply cool!


On the surface, it is easy to program the phrase, "Lights On" to make Voice Attack press the "L" key on your computer, thereby doing so in your game, turning the lights on, for example. The fun bit comes in when you also make it respond to either "Lights On" or "Lights Off" to press the "L" key. It can even respond with "roger", or any number of random responses if you write them in separated by a semi-colon.
Problem is that "Malcolm" will turn the lights off (if they are on) when you say, "Lights On" - no difference.


But what is really useful is the use of variables, which gets into the advanced realm but still able to be implemented by a novice thanks to the Voice Attack GUI. I've been working with my software for a few days now, and for Elite: Dangerous, I've programmed several useful functions to be activated by voice, including docking request that is much faster than I could possibly input the keystrokes/joystick buttons.

So, now my ship's voice activated computer "Malcolm" knows whether the lights are on or off, and can even tell me if they are already on when I request them on (and they are already in that state).
Yes, this is time consuming, but the commands are easy to export, the name "Malcolm" in the entire code set is easy to swap out for a different name, and the final product I create can be shared with anyone who'd like to work with this kind of interface.

Here's a snippet showing some of the code I did today for some of the standard on/off or opened/closed functions of a ship in Elite:Dangerous to be controlled by many variations of the same command to add flexibility (kill the lights, lights off, shut off the lights, turn off lights, turn off the lights, etc.).

next thing I'm gonna do is give him full "knowledge" and control of the state of modules and functions under the ship's right hand console - I'll be able to tell him to activate self destruct, he'll ask for authorization, I can program it to respond to a verbal code (Riker Omega 3), and that will change a variable thereby authorizing the first command to execute. lol



  • Upvote 1

Share this post


Link to post
Share on other sites
Wow this is really awesome! So many useful things can come of this if done properly. You can go outside of the gaming community and have it coded to do macro's for programs as well

Share this post


Link to post
Share on other sites
well, bout to write a block of instructions (over 256 actually) to give the voice control knowledge of the cursor position in the modules menu - it would be a simple 10 line code with C# or VB but I don't know either of those languages (if anyone here does, I'd love the help).

The voice control can access external plugins written in VB or C#, but I'm gonna go the long route for now with a >256 part block of conditions and global variables, and see if we can make it more concise and modifiable later using actual programming, for a spaceship other than mine (with it's 18 modules) as the cursor move routine wouldn't work with a different ship or loadout.

This may sound like gibberish to most people, so I'll get a demonstration video up this week. It's one thing to have minor voice control, but I'm not satisfied with that when this thing can do so much more, like the computer in the Star Trek series.

I want that, and I want it to control my game when I have a hand off the controller for some random reason...whatever that may be
!puffpass

Share this post


Link to post
Share on other sites
Probably a ton more work, but could it also be setup to have a 'personality' and provide comments/feedback in specific situations, like your shields drop to 10% it responds with "Shields at 10%, I would advise dodging more'.

Share this post


Link to post
Share on other sites
Unfortunately, if I make it hack into any data in game, it violates the EULA - at this point, it's more like a voice command macro controller on steroids. *meaning there's no way for him to know the shields current status, or any current status, unless I tell him. Gave him variables to track cursor movement position, and module status on/off, but I could turn it off and he wouldn't know I did it...

Got the menu navigation done, about 4000 lines for each VPOS cursor, but as there is no real game data going to "Malcolm", my voice controlled slave, he could lose place if I move the cursor manually, so I have "Reset Menu Cursor" commands, too.

Also, just added command code authorization - can make any command get a query for a voice code before the keypress - will be handy for such in-game functions as "Self Destruct" or "Reboot".
For now, he's responding to "Riker-Omega-3" (yea, I'm a star trek geek, hence my passion for voice control PC)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×