@onelesstweeter On Ivory for Mac, there's an "alt" button that's always visible. On Ivory for iOS, after tapping an image to view full screen, tap again, and you see controls like the close button, action button (Share, Save, Post, Copy), and alt button.
@gruber You give LLMs one toe in the door of Mastodon and they’ll knock the door down, barge in, and take over. Or more likely, while users breathlessly exclaim things like “amazing,” Mastodon’ll just swing open the door with welcome arms, invite the LLMs in, and the LLMs then barge in and take over. Either way, they barge in and take over.
@gruber I'll be honest: I hate this feature. I come to social media to interact with humans not LLMs. This is an optimization to make spammers’ lives easier.
@timbray@gruber building on this. One of the key pieces of advice when generating alt text is to NOT just describe what's in the picture, but to explain what's important about it. "a photo of a tabby cat on a lap" is descriptive, but it doesn't tell me what's notable about it. Is it adorable? Is it being derpy? Is it in a humorous position.
Without that, the post is pointlessly boring and context free for a blind person.
@masukomi@timbray@gruber And this ALT text guidance, I assume, carries to ANY photo used on the web? It’s a take I’d not heard of insofar as a possible best practice is concerned.
@AnthonyBaker@timbray@gruber yes. This applies everywhere. Imagine describing it to someone nearby who couldn’t see your screen. You’ll leave out a ton of factual things that can be described and focus on what’s important about it “an adorable tabby cat making a silly blep”
Note that you can leave out many things that have already been mentioned in the text preceding the image.
@masukomi@timbray@gruber Thanks for this. The bit about “leaving out many things that have already been mentioned” is interesting, as this means that if a single image or photo is used in multiple documents, the ALT text might need to be set on a per-instance basis (versus on the canonical item), to accommodate the context change for each placement.
to be explicit, that bit is from my time thinking about it and putting myself in the place of the blind folks who read my stuff. It'd be really annoying to read a mastodon post that was specifically commenting on the notable bits of the photo, and then have my time wasted with what's essentially the same info being repeated immediately afterwards.
i think when you get to image re-use in large systems it's better to play it safe and include everything so that it'll always be 👍
@timbray I wouldn't want to read AI-generated posts (and I hope I'd recognize it after a few and unfollow the account). But for alt text on images, this seems like a huge timesaver and benefit.
@gruber neat. Where this is going to fall down is providing context for the images; if two people are in a picture, and they're named in the story, the AI won't be able to know who is who.
@gruber iPhones can do something similar when reading text messages. If I’m on my headphones, it will give me descriptions like, “[your wife] sent a picture of two children smiling.” I’m not sure if the brevity is a choice on Apple’s part to keep the interruption short, limited ability of the phone, or both.
@gruber If alt text can be generated mostly accurately, why have it at all? For those who need/want it generate it on demand? Maybe an other feature for @IceCubesApp ?
@Mpwg@gruber@IceCubesApp being able to generate it on demand is fine, and increases overall accessibility. Relying exclusively on it risks missing the point the poster wanted to make. A bit like generated subtitles are fine — and better that not having them — but are not the same (in accuracy and quality) as proper subtitles.
The image shows a claymation-style character resembling a man with reindeer features, standing in a snowy landscape. He has antlers, a bright red nose, and is wearing glasses. He's dressed in a red coat with white fur trim, a green shirt, blue pants, and black boots. Beside him is a small, brown claymation reindeer with white detailing and a surprised expression. In the background, there are snow-covered trees and a few animal tracks in the snow.
@zkarj@gruber I was thinking to build an app just for that actually! For accessibility and other purposes. The result in Ice Cubes is short because I’ve asked it to be short. If you don’t put any limit it’s very verbose like the one you sent.
@courtcan@zkarj That is a better description, but I’m never going to bother switching out to another app to generate a description for each and every image I post here.
@gruber@zkarj I just don't post images unless I'm going to take the time to describe it in a way that will make it relevant to a person with vision issues. Until an AI can do it as thoroughly and as beneficially as a human, I don't think it's of much use to someone who can't see the images clearly or at all.
@gruber This seems like a great feature, but it’s odd that using it doesn’t cost anything. I may be missing something, but it looks to be using OpenAI API calls in the background. Those may be inexpensive, but they’ve got to add up!
Add comment