So, this morning, after years and years of using the GUI in Linux, I gave up. The state of the GUI does nothing but deteriorate over time for accessibility, and it's exhausting. It's only getting worse. We're far, far away from what it used to be, years ago. Certainly, the QT framework has improved since 5 and now 6 came out, but GTK? Oh dear, oh dear... So, let's dive into it. #linux#xorg#wayland#a11y#accessibility#blind
So, at the beginning, Linux started with console only for us blind folks. It is only in 2007 or so that the GUI really started taking off. Back in the days of gnome 2, the accessibility was extremely impressive. The UI was fluid, and every program in ubuntu (that's what I was using back then) was seemingly designed with accessibility in mind. Reality was probably different, but to me, it seemed this way. I loved it. Ubuntu 8.10 was the first Linux I ever used. Of course, QT was not even on the table yet when it came to accessibility, but, it was the glorious days of GTK.
But then, you may be wondering, why did it change? There are probably a lot of reasons why that is, but I can think of a couple.
One, technology keeps on evolving. When GTK 3 got introduced, it took quite some time for the single GUI screen reader to catch up with it. Regressions were also a thing. Not to mention that the orca screen reader has only one really active developer.
@xogium the AFB (American Foundation for the Blind) was forced to withdraw support and funding for various projects that worked on GTK/gnome 2 blind accessibility by an AFB corporate donor, and forced to fire the AFB CTO who ran these programs, Janina Sajka. The corporate donor that did this? Microsoft.
@xogium Many blind developers were sponsored by the AFB at the time (2000's) before then, and this is likely why there has been such loss of capabilities over the last decade.
@xogium Janina believed the blind could be empowered to enable and support themselves rather than begging corporations for crumbs. This was a threat to the idea of corporate patronage thru subservience.
@xogium at the time the AFB also sponsored GNU Alexandria; e-government services for the blind, and we had just produced a prototype server for the SSA. They also had made a portable daisy talking e-book reader using embedded Linux.
@xogium I believe a number of atk contributors were funded by them. They also had made a special GNU/Linux distro for the blind, I think using a serial connected talking device that was also presented at the LSM in Bordeaux in 2002.
@tychosoft Wow this is nice! Not what happened mind, but the technology everyone tried to bring. Shame it got ended before it began, but I'm not surprised in the slightest.
@tychosoft@xogium could this be restarted? I mean that sounds like something that could be sold to government, a lot of ngo with money and even some corporate sponsors...
@dgold@scruss@xogium I do, and had not appreciated it before. The AFB dropped driving development of gnome accessibility and many blind developers who worked on it lost funding just as gnome 3 was getting started.... it's long enough ago, and even I did not realize that timing.
@dgold@scruss@xogium but it made me realize something else. That could have been around the time Microsoft was pressuring Redhat to sell out and sign up for patent protection. Redhat and Microsoft negotiated for several years, and Redhat, to their credit, finally eventually did say no. Some 6 or so months later Novell eagerly embraced essentially the same deal.
@leifur@xogium I was working with the AFB at the time, I was impacted as one of my projects, e-gov services for the blind, which Janina was the principle contact, was shut down. We had just successfully demonstrated a prototype platform with the social security administration, too. What other people discussed this at the time I do not offhand know.
@xogium Don't forget that circa 2010-2011, Oracle acquired Sun Microsystems and disbanded the accessibility team. They were responsible for GNOME's accessibility infrastructure. We went from about 10 full-time people working on accessibility to none.
@federicomena@xogium Sun’s contribution to GNOME accessibility was not just on the development side either, but in things like applying for & managing EU grant money, testing against US Sec 508 standards, and serving on the various accessibility standards committees to ensure the standards gave free desktops a chance to meet US & EU purchasing requirements.
Another reason for this is sadly, that accessibility isn't a wildly known thing. Developers don't care about it for the most part, and if they do, it is never enough, due to the behavior of other developers.
How many times did the orca folks and various associations for the blind have to slap the gnome people on the wrist for taking out accessibility features or causing regression without caring? Too many, that's how much this happens. That's how regular it has become for us to have to fight over every version of gnome released to keep the accessibility more or less working.
Another good example of this is GTK. When GTK 3 came out, there was a bit of fighting over accessibility, but nothing alarming. The GTK folks were generally happy to make the toolkit in a way that it had built-in accessibility support. Default widgets came with accessibility from the get go, a bit like html 5 does nowadays for websites. And it was great this way.
When GTK 4 came out, on the other hand, things had shifted. No longer were the GTK people happy to provide accessibility for us. No longer did they care about it. Their grand plan was to remove accessibility from GTK altogether, claiming that it was up to the applications themselves to become accessible. It took several weeks, and even days during fosdem for them to recognize this wasn't the way forward, thanks to the orca developer and the Hypra folks, but they got the idea. Or did they?
It seems that they haven't gotten the idea, after all. I got interested in a nice GUI app for mastodon recently, called tuba. I discovered that it used GTK 4, however. I knew there were rumors about accessibility problems, of course, but what better way to learn than to experience it for yourself? So that's what I did.
Oh dear, was I disapointed! The first thing that struk me when launching the app, is the fact that the flat review of orca stopped working. Second, that it appears that GTK 4 and orca are fighting over the keyboard. I can't make orca stop talking, I can't know on which element on the UI I'm on until it finished talking the previous thing it was saying and moves onto this one. I can't use any screen reader shortcut, either.
It is exactly, if not worse, than QT 4 was, when accessibility was barely considered.
@xogium This is a common theme with the Gnome folks. If I want to have titlebars but don’t want to use GTK in Gnome… I have to draw them myself, and handle them myself. Honestly it’s made me drop the whole desktop environment into “unsupported, won’t work right under Gnome,” and I know I’m not the only one.
@xogium Hey there - I'm very interested in ensuring that our apps are accessible. May I ask a few questions to help diagnose the issues you encountered here?
@xogium I want to help get things to a point where you don't need to struggle :)
On GTK4, my understanding is that the main issue is how the screen reader keeps reading things that are no longer selected, right?
Tuba has some accessibility issues it inherits by using libadwaita. Up until recently, GTK did not expose the API required to make many external widgets accessible. That's been fixed in newer versions of GTK and libadwaita and Tuba will inherit those updates.
@devinprater@xogium Some have, yes - GTK 4.10 is in distros that follow releases closely and in the latest GNOME SDK. Not entirely sure what fixes in libadwaita have reached distros.
@brainblasted@devinprater I will have to check what archlinux has to offer. Biggest issues for me are gtk and orca fighting for control of the keyboard, orca not immediately speaking the element you just tabbed to, and flat review simply not working.T
But, enough about GTK. I'm now moving on to xorg and wayland.
Xorg was good. It was buggy, it was not especially secure for various things, but it worked. More importantly, it allowed the screen reader to perform relatively good.
Wayland, on the other hand, is the complete oposite. Oh sure it works for general use. But the minute you try to use mouse emulation to click on an element of the interface, be it on a website or in a program, orca crashes. Don't ask me why, I don't know. Orca is now no longer allowed to provide a clipboard, either. The excuse the wayland folks gave was security. Applications that lack a window, focused window at that, will not be allowed to use the clipboard. Well, guess what? To be able to copy the content of a window for example an error message to share it with people for assistance, one needs to focus that window. Not orca. Oh, and by the way, it's been years orca has had a window.
@xogium If applications without a window can't use the clipboard … that affects me as a developer. It means that applications that run in the terminal can't use a clipboard. The terminal has a window. The applications don't.
I use vim and tmux. Integration to the clipboard is not brilliant, but it's there, and it's very useful.
@fishidwardrobe@xogium I agree that accessibility is... rough to say the least, but the technical claims are also not really on point. It is totally possible for terminal applications to access the clipboard (e.g., wl-clipboard does just that). In general, I'm pretty confident that all the required bits for accessibility in Wayland are there... but there's not enough people updating tools such as Orca to work with the new protocols.
Some of the reasons why are mentioned elsewhere in the thread and are wholly non-technical. It really is more a social than a technical problem, Wayland is not the issue, but people not caring (or those who care don't have enough time/funding). That's more depressing however.
This all filled the glass of my patience pretty quickly, I do have to admit. I went through this for months. What put the figurative nail into the coffin for me was that I realized that somehow, GTK 4 programs are actually even worse on wayland than they were on xorg. The accessibility and usability degraded further! How's that possible? No clue. The very same program will behave a miniscule bit better on xorg than on wayland. Go figure that one out.
So then, you might ask me, why don't you just use QT programs if they work better? Well, the fact that QT 6 is out is great, but it also cause regressions. QT is nowhere near perfect. It is nowhere as usable as gtk 2 and 3 ever were, when things used to be done properly.
That aside it seems that QT accessibility features are opt-in rather than opt-out. I could be wrong about that, and as a matter of fact, I hope I am.
@fireborn@xogium Which is presently why I'm just not even going to try, I do not have the time to fiddle with complex hacked together setups and constantly un-break my system. I may be somewhat considered a power user but I'm not that good. I'll just use WSL for the occasional CLI thing and ignore all Linux GUI until the entire situation improves or until Windows falls past it.
And, even if it was that simple, there remains the fact that wayland complicates my life even more due to the so-called security and the problems it creates for assistive technology. And I'm tired.
I'm done dealing with this. I'm done trying and trying to summon the energy to make yet another bug report and feeling like deep down it won't matter in the end. I'm officially jaded.
As of this morning, I've gone back to windows for any program with a GUI. Granted so far it is only for web browsing, as the rest of my activities are doable in console, and I do enjoy the console.
@xogium this also touches upon the whole security fetish in Wayland vs X. I don't need or want to be secured "against" myself ;). I am not an island of silos that can't be permitted to talk to each other.
@xogium@tychosoft This is not how security works though unfortunately. It's not simply en encrypted channel where you can skip encryption, it's a different security architecture. And it's not protecting "you from yourself", but I bet you know that. The threat is a different one (and a real one).
The problem is not "security". The problem is that we have not enough people who care, and those who care don't have time/money for that.
Maybe some other people will pick up where I left. Maybe noone will. But at this point, I'm done. I can't keep on damaging my own mental health for the benefit of reporting bugs that might get fixed, but most likely won't. I can't keep dealing with this. As of today, I'm done with this entire thing.
To whoever resist out there, I can only wish one thing. Good luck, and don't sacrifice your mental health for it. Don't do the same thing I've done. Please be safe, and remember that going back to windows for some thing is nothing to be ashamed of. There comes a time where open source doesn't matter and accessibility gets the priority for everyone. Mine has been reached, and then some.
@xogium It's really depressing like that, and I'm not sure if we'll have another corporation 'let's try to get Linux usable on the desktop' moment again. Everything is moving to the web and mobile. Which means only hardcore Linux users get punished for being blind.
@xogium A lot of this makes me wonder if the current solution should be to just write applications that provide accessibility itself. Like bundling espeak in to an app, colorblind modes, etc.
It's not a pretty solution but it's becoming clear that we can't rely on infrastructure to stay accessible.
@devinprater@jookia@xogium Or the ones that would would all do so differently, requiring the users to learn every program as if it were an entirely different AT stack, like switching between JAWS and NVDA and VoiceOver and different browsers and... Perhaps the mainstream understandable Linux equivalent would be being required to know how to totally use both emacs and vim without any adaptation scripts to make one feel like the other.
@x0@devinprater@jookia Yes, exactly. I went through this w/ chromevox for years. I am never doing that again. I didn't mind orca vs speakup. But learning a screen reader for each and every app? Nop, thanks.
@xogium@devinprater@jookia From my outside perspective, I think along with a fair few developers not caring, there are just too many choices. The reason marginally more Windows apps are accessible is because unless you use something like tkinter which just draws the entire UI, you're eventually using win32, and win32 talks to the accessibility APIs by default. Unsure what QT is doing but I know the so-called accessible UI it exposes to screen readers is unnecessarily verbose, all buttons say enter for instance, and edit fields and tables just don't behave like the corresponding native controls. But while AtSpy or whatever it's called exists, the fact of FOSS means that all these UI toolkits and window managers and everything else are so fragmented as to require the NVDA app module/JAWS script style workarounds for every framework just as well as every app's idiosyncratic use of said framework. Of course centralization into one dominent framework for everything is not the answer, as the odds are said thing would be inaccessible by default and be staffed with people who don't care. I personally just don't think it possible without some fundamental change, beyond devs caring about accessibility. Something about the entire ecosystem has to change or it will always be a hopeless tangle of barely working this and that.
@xogium@devinprater@jookia Maybe accessibility becoming part of the kernel? That seems unlikely though and probably out of scope for very good reasons. TO my knowledge the kernel doesn't have anything to do with GUI, the only interaction it has is display rendering and you can't hook accessibility into that, that would be absurd.
@x0@xogium@devinprater@jookia Another direction you could go is to go via "the analog hole" - somehow build a system that takes in the video output of the primary system and OCRs everything back into a screen reader. Then you literally don't care what OS or video composter or graphics framework or anything is running.
@JessTheUnstill@xogium@devinprater@jookia Sounds like VO screen recognission, except then you have to deal with OCR errors everywhere and the fact that controls look different in different software and also even with such a model the refreshes would be slow af
@jookia@x0@xogium@devinprater Yeah, you're right. The next best option would be regulations from some of the main distros - "make your stuff accessible or else you're getting dropped from the debian/ubuntu/fedora etc. repos" would probably spur a lot of devs into action.
@JessTheUnstill@jookia@x0@xogium That's kinda what I want to push for but I know that'd be seen as way too radical for folks like Redhat or Debian or Arch.
@devinprater@JessTheUnstill@jookia@xogium Especially since accessibility, as a whole, is quite daunting. It would likely have a chilling effect on publishing in official repositories, which wonly further fragment the landscape. There would be fork distros created solely for the purpose of avoiding it.
I think having a unified accessibility commitment projects can adopt would be good. I tried to get this slipped in to the software conservancy's thing, but they didn't. Still put my names in the credits though.
@x0@devinprater@jookia Yep that's what I'm thinking too. As long as society will remain the same, it will leave us with such software stacks. A real shame if there ever was one.
@xogium have you ever tested macOS/iOS accessibility features? Apple does make a big deal out of it, so it would be interesting to know how does it contrast with reality
@sumek No, I don't own a mac. That being said, maybe I should make a vm to try it out... I've heard that qt programs are one of the things the mac screen reader can't handle.
@xogium Not sure if you can run macOS within a VM 🤔 There are apples stores or friends if thats an option for you. With Linux running in a VM you would have access to a Linux console dev environment and a *nix with accessible GUI applications
@xogium and I know Canonical had a blind developer on staff at that time. He split his time between audio and accessibility. (Can't have screen reading without audio.)
@maco Yes, I knew him a little. A few years ago, he left Linux behind and went back to windows after being tired that his advices and suggestions as accessibility dev were simply ignored by Canonical. Can't exactly blame him...
@maco Indeed. Pulseaudio and all the things that came before it were always problematic. Even alsa is! I should write a thread about audio vs accessibility...
@xogium feel free to contact him if you have any particular suggestions what to improve: Lukas Tyrychtr <ltyrycht@redhat.com>.
He's visually impaired, so he experiences all the problems of Linux desktop a11y himself, but he is always open to input from others, too.
@xogium thanks for this thread, I'm a sighted tech person who works primarily in Linux, and I've recently been wondering about exactly this - how accessibility is. It's 100% because of Mastodon and the accessibility community on here that I've started thinking about it.
I might not be the first to suggest this, but I wonder if testing for accessibility could be turned into automated tests for apps. The tests would require maintenance, but it'd be a place to start for developers (like me) who have no idea what good accessibility looks/sounds like.
@dizzyspiral While automated testing may sound like a good idea at first... I really don't think it's good. It might help, but it will never provide the accuracy and details a real user would. Same goes with testing accessibility of websites. Disabled people will always be a part of this equation.
@xogium oh, for sure, I didn't mean to imply that they shouldn't be. If automated testing is used as an excuse to exclude disabled users from giving feedback, then it's definitely worse. But if it can be used as a complement, by giving the developer immediate feedback before release, that sounds like an improvement to me.
It looks to me like the blind community does a LOT of yelling about accessibility getting broken, and that frankly looks exhausting. If I can lower that even fractionally by putting up pull requests with accessibility tests for open source repos, I will.
@dizzyspiral@xogium The maintanence piece is why automated testing for accessibility is not terribly robust: the ground under you keeps changing, forcing you to rewrite the test. It can work for some basic things like missing labels, but most of the smart people I have seen who try this on the web end up giving up. That said, maybe Linux is different!
Add comment