2021-12-06 18:28:51 (edited by chrisnorman7 2021-12-06 18:30:24)

Hi all,
Periodically I switch Flutter to the master branch and play with it to see what's improved. Last night when I did it I was pleasantly surprised to find that I can actually see the controls on the screen.

You have to use NVDA's object navigation to move through the app, and simulate mouse clicks (although keyboard shortcuts work), but it can be done.

It's not perfect, and I'm not entirely sure if that's Flutter's fault or NVDA's, although I suspect the former. Either way, if anyone else is waiting on desktop accessibility, you can at least begin prototyping, and of course it's only going to get better (hopefully).

Edit: To get to the master branch, type:

flutter channel master
flutter upgrade

I've changed my minimum Dart SDK constraint to

">=2.15.0 <3.0.0"

, but I'm not sure that's needed or not.

-----
I have code on GitHub

2021-12-06 19:24:55

Hopefully they're going to add support for tab. If not, you might want to open an issue.  Unless they do, this is useless.

My Blog
Twitter: @ajhicks1992

2021-12-06 20:49:32

Interesting, Although trying to find that desktop accessibility issue thingy, It seems to be closed on november 1st and windows stuff is not yet completed, Although that issue is maybe old and there's another, Not quite sure, But I really hope it's going to be accessible because that's going to be perfect.

2021-12-06 21:28:49

Now all they have to do is make it possible to compile Dart code to native code *without* the Dart VM also being compiled in -- that's what it sounds like "dart compile exe" does, at any rate.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2021-12-06 21:42:57

Shrug, why does that matter?  Lightweight vms are fine.  Being as native as C++ isn't what Dart is about.  Though I do think they have that, or are working on it.

My Blog
Twitter: @ajhicks1992

2021-12-06 23:58:59

@2
As @3 says, they've not finished working on it yet. When it lands on Beta, if it's not improved I shall open an issue.

@4
Yeah, sorry, that doesn't bother me all that much. If I wanted speed I'd be learning Rust.

-----
I have code on GitHub

2021-12-07 02:02:19

@6
Yeah but the thing is that Google going "well object nav exists, good enough!" would in fact be a very Google thing to do.

My Blog
Twitter: @ajhicks1992

2021-12-07 07:06:11

@4
I dout that will ever happen, And besides I'm quite sure the usidge dart is used for doesn't need that stuff, I mean python is around 4 times slower but people get around it fine without ever using cython.
Also, Without the vm? Wouldn't that make it basicly C++ but with a bit of different syntax or something? Since the vm is responsible for garbage collection?

2021-12-07 07:27:24 (edited by mazen 2021-12-07 07:28:19)

So, do flutter apps not support keyboard navigation or they don't send an event to the accessibility API when the focus changes?

2021-12-07 11:36:10 (edited by chrisnorman7 2021-12-07 11:54:24)

@7
Ah, OK, I didn't think about that. I'll open an issue when I've got time.

@9
So keys do do stuff: You can tab between elements, the focus changes just aren't recognised by NVDA.

I think that maybe there's more to do, because for example when you register shortcuts in your app, they don't get recognised until you've tabbed for the first time.

Edit:
I've created the issue here for anyone who wants to follow along.

-----
I have code on GitHub

2021-12-07 12:02:03

great! I can't wait to finish my university stuff and try it, this seemns so goddamn awesome. If that's the case and indeed apps like these are doable with flutter now, I don't think I'll ever think about xamarin or maui again, not for a good amount of time anyway. Furthermore, if I really want to write in dotnet as well as flutter, perhaps for the backend or hell knows what, .net class libraries can be compiled to native code now, so we can theoretically expose C interfaces to things inside dotnet assemblies, though I DK why would I need that. That said though, if the situation with focus changes reporting doesn't work, I would indeed open an issue, but more than that, I could try to open a pull request once I figure out my self what's wrong in there and how to fix it.

2021-12-07 13:54:05

good to hear. I hope Linux accessibility will be worked on because that would be very useful

2021-12-07 14:46:52

@8
Don't underestimate how much the language can matter.  Compiling to native code doesn't make it the same as all the other languages that compile to native code, save that you can use it for that domain.  For something really, really different that illustrated my point, look up nim (please don't use it: it's unpopular with good reason. But is still interesting nonetheless).

My Blog
Twitter: @ajhicks1992

2021-12-07 15:03:54

@13
Sorry if people got my post wrong, But I didn't mean native = same as all languages or exactly C++, I ment since the vm is responsible for garbage collection and those that make dart people never care about memory management except in cases where FFI is involved or something, I was just confused why @4 would want the vm removed. Unless if my understanding of what the vm does is wrong, That can be possible.

2021-12-07 15:04:37

good news for us Linux users, flutter desktop seems to work. I installed the flutter SDK on my machine, created the counter project and ran it. There are some issues, such as the thing where you can't read the text labels without using flat review commands and the fact that you have to tab out of the window a couple of times in order to find the increment button, but it works and I am tempted to open some issues about these things. Keep in mind that I haven't switched to the master channel and it works ok, I will try the master channel before opening anything but I'm excited about what I see at the moment.

2021-12-07 15:16:59

indeed, linux accessibility was working before windows, mac OS as well, windows was kinda the low hanging fruit in this. It's not perfect, but certainly more usable than the windows side for now. I DK why, maybe because of canonical, something else? we may never know. About text fields not being read upon tabbing, it's normal as far as I know, labels don't get focus. So, either text review or some kind of object nav should be used. This is not only linux, it happens in windows as well.

2021-12-07 15:18:24

so it’s impossible to fix then or what? How are you supposed to display information to users and all of that?

2021-12-07 15:19:53

@14
Yeah you can do gc without a vm no problem.  Gc is kind of separate, you just have to know where all the pointers are at and that's easy enough for a compiler to do.

@16
On at least Windows, there is 100% a way to support textbox labels through the accessibility API.

My Blog
Twitter: @ajhicks1992

2021-12-07 17:00:55 (edited by Ethin 2021-12-07 17:01:14)

@18, and there's a way to do it through this function on Linux, at least I think that's the one.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2021-12-13 15:55:53

I have just updated Flutter, and now things seem to be much better. You can't tab to bits of text, but then I wouldn't have expected that anyways. I shall investigate if that can be made to happen, but arrow keys tab to bits of the UI, the enter and space keys work, and everything seems to be generally pretty good.

-----
I have code on GitHub

2021-12-13 16:40:10

perhaps I wasn't clear enough in what I ment. By text fields, I ment the labels them selfs, the label controlls, of course you can't tab to those. Annotate another controll with them, perhaps, however focus can't arrive to labels unless they are marked with some tab order attribute, I forgot exact details.
@ethin, I DK if you linked the proper function or not, as I see many there, everything supported by an accessible. How it's done on linux, well, a bit more involved, that kind of overengineering in a box camlorn was talking about some time ago. So, a relationship object is created that has as the source the accessible object it's annotating, the type as labeled by, and the target yet another accessible object with the role of text. If you want to find the text an accessible is labeled by, you have to query for all the relationships of that object, a functionality all accessibles support. Then, you have to itterate over the collection, testing if type is labeled by. If so, it'll get a hold of the target, aka the label the accessible component is tied to. Finally, it will query for the text on that object, which is actually the text read to you by orca when that controll gains focus.
Yes, I get the implications of this, I hope people who read this do as well. Because the target of that relationship will be another accessible, nothing and no one would prevent someone for annotating an accessible component with, say, a button. Nothing much bad would happen anyway, the text will be read from the button in that case. In case it's not found, the controll will just be considered unlabeled and that's the end of that. However, such tiny flaws are many in atspi, enough a redesign would be required ages ago, perhaps we'll arrive to that soon enough.

2021-12-13 17:04:59

@20
Yeah, as long as there's a way to get a screen reader to read text.

Unfortunately now that I'm as far as working on it, it looks like all my plans for an accessible UI on my games may have fallen through: only SDL provides cross-platform access to scancodes.  This makes me very mad.  Yes yes I know Dart has one, but it's not standardized and you (probably) have to use different constants for everything.  Otherwise your keyboard layouts only work on English keyboards without the actual layout changing.  I haven't finished figuring that out yet because for example how do you do localized labels?  So we'll see.

@21
I think you're referring to the other thread.  Accessibility APIs on Windows aren't significantly better, but that level of complexity is present for a reason, it just happens that our AT doesn't use it.  The problem with the JAB is that Oracle went off and did their own thing because they can, applying the usual Java-esk thing that leads to issues like the Log4j remote code execution vulnerability from yesterday where it both does more and less than you need at the same time and by the way you have to install it perfectly and and and...

My Blog
Twitter: @ajhicks1992

2021-12-13 21:40:43

@22
Oh, that doesn't sound good. I'm guessing by Dart you mean Flutter's SingleActivator and CharacterActivator classes?

-----
I have code on GitHub

2021-12-13 22:07:27

@23
Basically.  The long and the short of it is that I'm still untangling it and ask me for details in a week or something.  I *think* it may be okay in the sense that most keyboards send ascii, but for example in Germany they use azerty, which moves a and z to q and w's location, so if you try to do the wasd movement thing then it doesn't work unless they or you remap.  Or you use scancodes, which are locale-independent, and don't change position even though the label on the key might be different--except then I sure hope you didn't say A is left because it might not be an A.

I'll make some sort of decision once I'm done looking into it.  If everything sends ascii or close enough, then I will probably solve it with dedicated locale-specific layouts and a lookup table, then go ahead with my plans in order to find out if there will be screen reader issues with them.  Otherwise, back to the drawing board and the land of haha you reinvented all these self-voicing controls, perhaps.  Mostly audiogames have just shrugged at this sort of thing, but I would obviously like to do much better when it comes to writing an online game that's trying not to just include the English speakers.

My Blog
Twitter: @ajhicks1992

2021-12-13 22:45:09

@24
It seems that Flutter already does this for you with the KeyboardListener widget, specifically when retrieving the phyusicalKey property of KeyEvent. AZERTY is specifically mentioned.

-----
I have code on GitHub