So what if we could get the current generation of component frameworks to expose IDs and maybe even class names (or equivalent) as well, in properties that can be accessed by assistive technologies? UI Automation has the ClassName and AutomationId properties. I don't know if Android has anything useful here. And of course, all of this would be useless on iOS unless Apple does something very out of character, like adding scripting to iOS VoiceOver.