GNOME.org
 

Re: [orca-list] ot programming games with an accessible interface

Sorry it took a while to send this. Life's been stressful lately and I dropped the ball. 


Here is the speech.lua:
-- The Windows portions of this code were adapted from the Pokecrystal Access scripts by Tyler Spivey. 
-- Original scripts: https://github.com/tspivey/pokecrystal-access

local ffi = require("ffi")

ffi.cdef[[
typedef enum {
  SPD_MODE_SINGLE = 0,
  SPD_MODE_THREADED = 1
} SPDConnectionMode;

typedef enum {
  SPD_IMPORTANT = 1,
  SPD_MESSAGE = 2,
  SPD_TEXT = 3,
  SPD_NOTIFICATION = 4,
  SPD_PROGRESS = 5
} SPDPriority;
void *spd_open(const char *client_name, const char *connection_name, const char *user_name, SPDConnectionMode mode); 

int spd_say(void *connection, SPDPriority priority, const char *text);

int spd_cancel(void *connection);

int spd_set_voice_rate(void *connection, signed int rate);

void spd_close(void *connection);

typedef unsigned int UINT;
typedef unsigned int DWORD;
typedef const char * LPCSTR;
typedef wchar_t * LPWSTR;
int MultiByteToWideChar(UINT CodePage, DWORD dwFlags, LPCSTR lpMultiByteStr, int cbMultiByte, LPWSTR lpWideCharStr, int cchWideChar); 

void Tolk_TrySAPI(bool trySAPI);
void Tolk_Load();
bool Tolk_Output(const char *s, bool interrupt);
void Tolk_Silence();

]]

local lib

local kernel32

local speech

if ffi.os == "Linux" then
  lib = ffi.load("libspeechd.so.2")
  speech = lib.spd_open("love2d", "love2d", "love2d", lib.SPD_MODE_SINGLE)
  lib.spd_set_voice_rate(speech, 50)
elseif ffi.os == "Windows" then
  kernel32 = ffi.load("kernel32")
  lib = ffi.load("Tolk")
  lib.Tolk_TrySAPI(true)
  lib.Tolk_Load()
end

local CP_UTF8 = 65001

function to_utf16(s)
  local needed = kernel32.MultiByteToWideChar(CP_UTF8, 0, s, -1, NULL, 0)
  local buf = ffi.new("wchar_t[?]", needed)
  local written = kernel32.MultiByteToWideChar(CP_UTF8, 0, s, -1, buf, needed) 
  return ffi.string(buf, written*2)
end

local M = {}

function M.say(text, interrupt)
  if interrupt == nil then
    interrupt = true
  end
  if ffi.os == "Linux" then
    if interrupt then
      lib.spd_cancel(speech)
    end
    lib.spd_say(speech, lib.SPD_IMPORTANT, text)
  elseif ffi.os == "Windows" then
    lib.Tolk_Output(to_utf16(text), interrupt)
  end
end

function M.setRate(rate)
  if ffi.os == "Linux" then
    lib.spd_set_voice_rate(speech, rate * 100)
  elseif ffi.os == "Windows" then
    -- TODO: Can we support this with Tolk/SSML?
  end
end

function M.stop()
  if ffi.os == "Linux" then
    lib.spd_cancel(speech)
  elseif ffi.os == "Windows" then
    lib.Tolk_Silence()
  end
end

return M
And here is a "hello, world" game that launches with a blank screen and just speaks some text. Put this in a file called main.lua in the same directory as the above speech.lua and run "love .", assuming you've either installed the package or renamed the AppImage appropriately and put it in your path: 


local speech = require("speech")

function love.load()
  speech.say("Hello, world.", false)
  speech.setRate(1)
  speech.say("This is faster.", false)
  speech.setRate(0)
  speech.say("This is very slow.", false)
  speech.setRate(0.5)
  speech.say("Goodbye.", false)
end
The website has some great tutorials. If you want to do audio games, grab their platformer tutorial code and make it accessible. Add footstep sounds on motion, collision sounds. Use the built-in audio APIs to play sounds when objects appear on screen, panned to their position. Use speech.lua to speak your coordinates and other debugging information. You've got a great audio game platform and tutorial resource if you're willing to do a bit of experimentation and modification of tutorial source code. I wish I'd had this 12 years ago when I tried audio games in Java. :) Hell, please let me know if you do anything with my advice. I'd love (har har) to see what you've done, and maybe even collaborate. Linux needs more audio games, and I'm doing what I am here to help make that happen. 


On 8/16/19 7:00 PM, Nolan Darilek wrote:

Er, rather, I'll put the speech code up on Dropbox. Sorry, long day.


On 8/16/19 6:58 PM, Nolan Darilek wrote:
Love2D is a full 2-D game engine, so absolutely it has key down/up-tracking, joystick controls, touchscreen event-handlers, etc.
I'll put the key-handler code up on Dropbox tomorrow and paste in a link. I haven't figured out how or if I'll open source my accessibility code for the engine, but other than the speech stuff, there's nothing special about much of it. 




On 8/16/19 6:32 PM, Storm Dragon wrote:

Howdy Nolan,

I'm 100% interested in this. Please send me the speech stuff.
Also, one thing that I have found to be essential for any kind of game is, does it have the ability to track key_down and key_release events and turn off key_repeat? 

Thanks,
Storm

On Fri, Aug 16, 2019 at 01:22:16PM -0500, Nolan darilek wrote:
I'm doing a bit of this myself. If you want something like an audio game without a UI, I'd suggest something like Love2D: 


https://www.love2d.org
It's essentially a Lua interpreter with sound, graphics, physics engine, etc. linked in, and simple, well-integrated APIs to easily access all of these. The AppImage includes everything you need to start in a single file, and games you build can run under Linux, Windows, MacOS, and with a bit of extra effort, iOS and Android.
When launched, you'll get an inaccessible window. This is an empty game window. All work is done via the console, editing the main.lua entrypoint, then launching the game with something like: 


love .


from the directory containing main.lua.
I've actually been doing Linux audio game development for a while with this platform, but $dayjob has had me busy enough that I haven't polished what I have for release yet. I should probably get back on that wagon. If you decide to go this route, I can share with you a speech.lua that binds to Speech-Dispatcher under Linux and Tolk under Windows, so you can speak coordinates and such. I have other more advanced code, like a simple accessibility interface over the Luigi GUI module, that I might eventually open source at some point. Currently that makes buttons, sliders, and maybe a few other widgets accessible, and eventually I'd like to do some sort of explore-by-touch interface for touchscreens, games like roguelikes, etc.
It's a nice platform, and easy to get started with for 2-D, which is essentially what audio games are anyway. We should probably take further discussions off-list as to not clog things up further. Let me know if you make any progress and I'll shoot you my speech.lua. It's kind of hard to do that sort of development if you can't introspect the game environment, and speech is probably the best/easiest way to do that. 


On 8/16/19 1:10 PM, Pavel Vlček via orca-list wrote:

Hi,
I want to create a little game, probably in Python. I want to have sound and graphical interface. Which interface is best to use? I don't want to use WX and I don't know, if pygame for sound is good idea. Distro Fedora. Can you give me some recommendations? I will use geany as code editor. 

Thanks,

Pavel


_______________________________________________
orca-list mailing list
orca-list gnome org
https://mail.gnome.org/mailman/listinfo/orca-list
Orca wiki: https://wiki.gnome.org/Projects/Orca
Orca documentation: https://help.gnome.org/users/orca/stable/
GNOME Universal Access guide: https://help.gnome.org/users/gnome-help/stable/a11y.html 
_______________________________________________
orca-list mailing list
orca-list gnome org
https://mail.gnome.org/mailman/listinfo/orca-list
Orca wiki: https://wiki.gnome.org/Projects/Orca
Orca documentation: https://help.gnome.org/users/orca/stable/
GNOME Universal Access guide: https://help.gnome.org/users/gnome-help/stable/a11y.html 


_______________________________________________
orca-list mailing list
orca-list gnome org
https://mail.gnome.org/mailman/listinfo/orca-list
Orca wiki: https://wiki.gnome.org/Projects/Orca
Orca documentation: https://help.gnome.org/users/orca/stable/
GNOME Universal Access guide: https://help.gnome.org/users/gnome-help/stable/a11y.html
[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]