2021-12-13 23:38:30

O wow, ok, that's really cool.  Wx doesn't do that.  Will have to investigate more.  In principle I'm okay with just using Flutter like SDL until their accessibility is stable enough to, well, be stable.  If nothing else it puts me in  a great position to get onto the phones, though I have no idea what Apple will do with an audio-only audiogame without graphics.  Still, one problem at a time, and with the popularity of Android these days it'd definitely be worth it even without iOS.

My Blog
Twitter: @ajhicks1992

2021-12-13 23:45:23

Not to double post, but it looks like you can make custom widgets focusable via FocusNode.  You willing to do an experiment w.r.t. if we can use that to, e.g., focus text instructions?

That and support for live regions would be amazing.

My Blog
Twitter: @ajhicks1992

2021-12-14 00:30:58 (edited by Ethin 2021-12-14 00:32:04)

@21, trust me, AT libraries have to appear overengineered. But it definitely isn't overengineered at all.
When you create a widget that can't have a label directly attached to it that the screen reader can detect like a check box, a combo box, a text field, a rich text area, a spin button, a slider, etc., you do the following depending on platform:

  1. On .NET you set the AccessibleName property (you can set AccessibleDescription for extra info, but you usually never have to touch accessibleRole because that's already known, and tampering with that can have weird and unexpected consequences because the screen reader uses that to determine how interaction with the control is handled).

  2. On win32, you set the labeled by property (I think) or the name property.

  3. On Linux, you call atk_object_set_name () and atk_object_set_description () (and if you absolutely have to, atk_object_set_role ()) for that information. Retrieval occurs through atk_object_get_attributes (): you call that function on the ATK object you want to examine, then walk the list of attributes until you find the "accessible-name", "accessible-description", and "accessible-role" attributes in the property_name field.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2021-12-14 04:29:09 (edited by Sage_Lancaster 2021-12-14 04:30:13)

this sort of relates to the entire thing. I opened up an issue about Linux accessibility. I don’t know if I explained everything very well because I am not that good of a writer but I hope it helps in someway.
https://github.com/flutter/flutter/issues/95230

2021-12-14 04:55:00

@29
They'll probably link you to the other one they already have for that purpose.  It's not forgotten I'm pretty sure.

My Blog
Twitter: @ajhicks1992

2021-12-14 05:54:51

@Camlorn
that one was windows spicific. Nowhere was orca mentioned, it has a label saying "windows desktop” and that issue may close because of the work that was done to fix it, which, by the way, understandably, did not cover linux. That is why I made a new one for this purpose.

2021-12-14 06:59:16

@31
I know, they have like 10 or 15 accessibility issues open right now and at least 3 are split out by platform.  Limiting the scope of issues is normal.  I am 99% sure the other one exists, but I don't have them all bookmarked so you'll either need to look for it or let them triage your issue as a duplicate if they choose to do so.  I know that they have already landed at least some of the Mac work as well.

That said, tbh, if you do go Linux get used to anything that doesn't use gtk not working though.

My Blog
Twitter: @ajhicks1992

2021-12-14 07:01:54

@camlorn
I probably do not understand you're issue correctly, But did you ever look into winit? It's a cross platform, Native rust windowing thingy
https://docs.rs/winit/latest/winit/

2021-12-14 07:19:54

@32
I already whent linux, why do you think I opened this issue?
I realized the part about non GTK apps having a hard time. QT apps seem to be the best example I have found, electron works with the "--force-renderer-accessibility" and "enable-caret-browsing" flags, that is unless someone doesn't label something. But what's the harm in getting something else to work?
And about those 15 issues, what I saw in terms of accessibility issues for desktop all have been closed besides Chris's and mine, most of the accessibility issues are opened for the mobile platform.

2021-12-14 08:09:15 (edited by Ethin 2021-12-14 08:09:54)

@34, I've found that many electron apps that I use today don't require any flags at all -- they just work out of the box. Perhaps its time to contribute to electron to force accessibility to be enabled on all platforms as well as  carrot browsing... There's no reason it should ever be disabled IMO. The problem is then actually getting companies to use that updated version of electron...

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2021-12-14 08:14:27

@Ethin
it isn’t only electron that does this. Flutter does the same thing with its web version. You have to click enable accessibility every time for some stupid reason. I opened an issue about that and they just gave me a line of code that will enable accessibility by default. The thing is, I feel like a lot of companies will not do this or just forget that it even exists

2021-12-14 08:48:08

@36
The reason that Flutter doesn't enable accessibility by default is because it slows down the loading of the widget tree. IIRC, it's not a permanent thing, but only while they improve the speed of loading semantics.

@27
Of course I will mate. Anything I can do to help. I'm working on an app right now that has some place holder texts. I can try and make one of those focusable.

-----
I have code on GitHub

2021-12-14 13:51:48

@36
Yeah so the way this works out is that for what they're doing the web is entirely fundamentally horribly, horribly broken.  What they actually have to do is spin up and manipulate a bunch of HTML, rather than just exposing an API.  This is something like a 2x to 10x slowdown on your app, depending.  There are efforts going on to try to fix that by providing something like what desktop apps can do, but they're in the experiment to see what we want stage last I checked and this means 5-10 years before they can be used in production.

If there's not a button or spoken hint they should add one, though.

As for Linux keep in mind that BGT Lover had to go as far as reading the source code to figure out how any of it works.  You have bigger problems than things not being accessible over there right now.  As far as I know there are no up-to-date docs, and the only big thing I know of was an aspirational spec that was never entirely implemented.  Not saying I'm necessarily up to date on that understanding, but still: if you're going to cry accessibility, start by making it possible without having to hunt for the information and read source and whatever; all other platforms don't have this problem.

My Blog
Twitter: @ajhicks1992

2021-12-14 14:28:28

@camlorn
I think BGTLover is working on something like that. That is at least what I have heard from him. There is also a guide to atspi on his screen reader project's site. Don't know how up to date that information actually is.
I want to help, but I am not that good of a programmer yet, so improving accessibility of flutter is out of the question. Is there really anything I am able to contribute here besides the issues I make in order to at least try to help?
And I wouldn't say I am crying about accessibility, If someone tells me that they need docs for something, I will either hunt down some docs or I could get in contact with someone that knows about it and ask them if they would throw something together as a kind of temporary documentation, And if there is no documentation able to be created or if they just simply can’t work on it, I won’t press the issue unless I have a justifiable reason to.

2021-12-14 14:43:02

@33
No, it doesn't; I can get all of that with libsdl2.  I want to use actual controls rather than being the nth person to reinvent screen readers from scratch for their game.

My Blog
Twitter: @ajhicks1992

2021-12-14 15:35:58

@38
There is a "Enable Accessibility" button, so it's pretty clear how you can proceed. 3 lines of code and you can get rid of that though, so it's fine.

Also, there are already live regions via the Semantics class.

-----
I have code on GitHub

2021-12-14 15:46:22

@27
OK, so I have experimented.

I've had good results wrapping widgets in Focus and FocusScode widgets.

I also tried using a FocusTraversalGroup widget, but that wasn't quite so useful, although that could be because I didn't try overly hard.

HTH.

-----
I have code on GitHub

2021-12-14 16:46:02

@42
Wow, awesome.  Once I settle on a design I will totally try this out. You'd think that I'd be able to throw together robust keyboard handling/ui architecture for a networked game in my sleep but this is sadly not the case, kind of punting it down the road a little bit since everything I'm writing has unit tests and stuff and in theory just needs wiring at the end.  Hopefully we'll have a soundmud-quality prototype sometime "soon" though.

Fwiw live regions may or may not work on desktop.  They used to not work on iOS I believe, though when I was looking into this I didn't check on the timelines.  They also don't offer assertive ones which is kind of meh so probably we end up using tolk or whatever, but that's fine.  Still an improvement over reinventing the UI wheel.

My Blog
Twitter: @ajhicks1992

2021-12-14 21:12:03

@43
Well, as Flutter is more my bag than yours, please feel free to ask anything you like. I've used it quite a lot up to this point.

Live regions definitely work on Android, because I've used them, but yes, you're right about the assertive thing: Talkback just keeps on going regardless haha.

-----
I have code on GitHub

2021-12-14 21:50:23

Yeah.  Tbh I'm glad they don't offer assertive live regions, because generally people use them wrong in the rare cases where they do get used.  It's inconvenient for my case, but it's one of the things you can use so badly that the page is literally unusable because you can't get the damned thing to shut up and it keeps interrupting the speech that you need to hear to do anything.  I have encountered sites that for some reason think that (1) timers need live regions around them and (2) this live region should interrupt the screen reader every second.  If you asked me to rate "bugs where people really drop the ball and it's unusable without opening the web debugger and live-tweaking the html", this one is number 2 on my list behind misuse/forgetting to toggle aria-hidden.

But it would be nice to not have to juggle packaging a bunch of dlls.  Alas.

My Blog
Twitter: @ajhicks1992

2021-12-14 21:52:30

@45, or those websites that have help bots that use live regions in them that are constantly interrupting the screen reader. Talk about infuriating...

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2021-12-16 22:51:15

because of a frustrating odilia bug that had me scrapping an entire branch pending redue from scratch tomorrow, I decided to do something not in rust at all, then I remembered the flutter accessibility things on windows since linux is not usable enough yet, so I decided to try it.
I see focus is working, even if I apply it to a label or some other controll like that, so that's not a problem.
however, can someone verify that live regions actually don't work on desktop, at least not with nvda?
if it's of some help, here's a fragment from the build method of the code I tryed it with, kinda the starter application, just a bit modified to illustrate my point.
if the code seemns too nested and weird, it's either because I just wanted to see if it'd work and I did no refactoring before posting, or because vscode is scruing itself and decides the norm today is two spaces in stead of four per indentation level, or actually both. Either way, here goes

return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: Center(
        child: Row(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            Semantics(
              child: Focus(
                child: Text(
                  'You have pushed the button $_counter times',
                  style: Theme.of(context).textTheme.headline4,
                ),
              ),
              liveRegion: true,
            ),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
          onPressed: _incrementCounter,
          tooltip: 'Increment',
          child: const Icon(Icons.add),
          style: ButtonStyle(
              shape: MaterialStateProperty.all<RoundedRectangleBorder>(
                  RoundedRectangleBorder(
                      borderRadius: BorderRadius.circular(18.0),
                      side: BorderSide(
                          color: Colors
                              .black))))), // This trailing comma makes auto-formatting nicer for build methods.
    );

So, focus works, however live regions don't work on windows at all. I mean, one could do so that the function that updates the live region would also speak it with tts, but anyway, am I doing something wrong here, or it is just how it is for now? if it's not normal, I'm ready to open an issue and we'll see what comes of it. Anyone has any insight? how is the live region stuff usually handled in the desktop realm?

2021-12-16 23:07:47

In other things besides Dart, basically like that.  However in general it is less supported in the sense that most UI frameworks with accessibility don't expose it.  They can though, and tbh I'm betting they probably will if they haven't.  You'll need Chris to speak to whether it's correct or not.

My Blog
Twitter: @ajhicks1992

2021-12-16 23:46:20

@47: That kind of threw out my chances of trying dart... Oh dear, 5 closing parenthesies on one line?

2021-12-17 00:17:37

@49
Everyone always, always, always, always, does this.  It's code.  Factor it out into variables and functions and stuff.  These frameworks are supposed to be used to write small composable components that you put together like code, and it annoys me so much that blind people look at the educational examples, forget that it's code and assume it's special, and then make this statement.

You will eventually discover the hard way that the WX-style thing where you imperatively build up your form by writing code that is like "add button x" is buggy and stuff because it becomes a very difficult proposition to keep it updated as data changes.  GO try to do some complex verification of a textbox with that or to keep a window updated off 5 data sources all of which change due to external events.  I'll wait.

But seriously it's like a lot of you new people put things into two categories.  On one side are your functions, classes, variables, etc. where you're fine treating them like code.  On the other side is things like UI components, Synthizer buffers, Pygame event loops etc. where the fact that they're code goes out the window all together and you forget that this is a programming language.  Like, for example with Pygame events, you can break it down and handle some events in different functions where you go "O it's keyboard, better send it off to handle_keyboard" and stuff and instead just don't even think of it because it's in this weird special category you construct that random things get thrown in where you don't really understand composability, and you assume they suck because you have this in this cannot-be-broken-down box and thus don't actually try to break it down at all.  As an example, the last time Dart came up there was this general omfg wtf amazement when I pointed out that actually you can put components in variables and use the variables, and it's just, like, you're looking at expressions and I don't understand at all why these expressions that happen to be building a UI end up in this box labeled special fairy magic, along with all of the other aforementioned things.  like, so often people who are otherwise fine with writing abstractions look at Synthizer, go this is complicated, and are just like wow wait what do you mean I can put it behind a class that does what I need? even though they know what classes are and how to put things in them.

My Blog
Twitter: @ajhicks1992