Compatibility with Dragon NaturallySpeaking (MSAA)

[birke]

Title:
Compatibility with Dragon NaturallySpeaking (MSAA)

Area:
Link programming

Summary:
I suggest making links MSAA compatible. The dictation program Dragon NaturallySpeaking uses this to 'read' websites and click links using voice commands.

Description:
I'm coming to this with ignorance of the technical details. Dragon NaturallySpeaking, the foremost dictation program out there and one frequently used by people with hand/arm disabilities, allows me to follow links, enter text, switch tabs, jump from one text box to another... all by voice, in either Internet Explorer or Mozilla Firefox. But there are some popular sites where this breaks down, including Google, LJ, and DW. At DW, I can enter text and follow some links -- such as those on the main page -- by reading the link out loud, e.g. "Reading" or "Post." But on my reading list, if I tried to open a cut text or follow an external link by reading aloud, e.g., "More under the cut" or "nytimes.com", DNS wouldn't recognize that text as belonging to a link on the page.

I asked why at a speech-recognition forum, and got this answer:

"The reason Dragon cannot see some websites is simply because of the programming components that were used to create the website. Dragon uses mainly MSAA( Microsoft active accessibility) to pick up the text links, images, combo boxes, list boxes etc and if the MSAA API can't see the particular components (usually because they are customised components) used on the websites you mentioned them there will be no command access. So to give the short answer if the developers of the websites you mentioned make it MSAA compatible then DNS will be able to access it."

I don't know what you can do about this, but I feel it's important that you know about it. Google didn't know about it when they started, and now they have built what another forum member called "a largely inaccessible Web empire."

Thanks for listening!

Poll #4243 Compatibility with Dragon NaturallySpeaking (MSAA)

Open to: Registered Users, detailed results viewable to: All, participants: 31

View Respondents

This suggestion:

View Answers

Should be implemented as-is.
21 (67.7%)

Should be implemented with changes. (please comment)
0 (0.0%)

Shouldn't be implemented.
0 (0.0%)

(I have no opinion)
8 (25.8%)

(Other: please comment)
2 (6.5%)

Comments

[denise]

For the record, we absolutely put a high value on accessibiltiy -- in fact, we have a project team for it: dw_accessibility

I do not know enough about this to know whether the specific idea of MSAA implementation is the best way to make Dragon work well, but I will post a link to this suggestion over there, and you might want to join that comm too :)

[foxfirefey]

It looks like MSAA for web pages relies on WAI-ARIA elements, which we're already planning on adding/have added some:

http://en.wikipedia.org/wiki/Microsoft_Active_Accessibility

[birke]

I actually came here via the accessibility community, there seemed to be encouragement there to post suggestions directly to suggestions. And of course immediately after I posted I thought "Now really, it should have just remained a discussion at accessibility for now." Oh well! Thanks for posting it there.

[denise]

Oh, no worries! Suggestions is the place to put stuff so that it can get put into Bugzilla to work on; it's just that I'm the person who does the main migration to Bugzilla, and I don't know enough about this to know whether this would be the right way to go about achieving the goal (better Dragon compatability), so I called in the experts. Then I thought to let you know about dw-accessibility in case you hadn't found it!

[deborah]

*remembers telling allen that I'm not doing any more dreamwidth work in till fall semester is over*

*thinks that allen will understand why I find this particular suggestion important*

birke, I'm going to look into this ASAP. As you might guess from that lovely "in till" up there that I've decided not to correct, I'm a naturally speaking user, and I've done a lot of the WAI-ARIA work in the code so far. So I will do this investigation and see if this is the right move for us. Can I touch base with you off-line about other questions about what you think would make the site more accessible to you with speech recognition? And I do encourage you to join dw_accessibility as denise suggested above.

[feathertail]

Can I just say how awesome it is to see how seriously the people here take accessibility, and how quickly this is being dealt with? I've got the warm fuzzies now, srsly.

[birke]

Sure, I'm happy to talk about speech recognition to the extent I can. I am very much a user and not a developer, and have already found this thread educational. I've sent you my email address.

[giglet]

I use Dragon. Not all the time, but it's my second resort and sometimes my only option.

I would very much like you to be more Dragon compliant, but whether this is the best way to do it, I have no idea.

[zvi]

[PDF] Guidelines For Speech-Accessible HTML for Dragon 2009 and Guidelines For Speech-Accessible HTML for Dragon NaturallySpeaking 2007

My quick glance through suggests three problems that don't have to do with MSAA:

1) Dragon only processes the first 200 HTML elements in a page, by default. (this is a user controlled-setting)

2) Dragon has some difficulty with active controls (possibly this could implicate javascript stuff, like the read more. I wasn't at all clear.)

3) If multiple links have the same text, this makes using dragon to navigate links more difficult (because you have to specify which link you mean by number.) Although not impossible.

[deborah]

still, being able to process the first 200 elements, or having to say "reply, choose three" is still better than not having that access at all. NaturallySpeaking users have to have some kind of backup way of accessing links, but knowing that a specific site has the easier way would be a nice feature, especially if it is a site that is so heavily used, as dreamwidth is for many of us.

I have naturally speaking 11 professional on order, and as soon as it ships, I will get my copy. So it will be interesting to see what is different with that.

[jumpuphigh]

I chose Should be implemented as-is. However, what I really wanted to choose was Whatever needs to be done to make DW accessible to Dragon should be done.

[susanreads]

+1

[instantramen]

+1

[jazzfish]

+1

[denise]

Hey -- usually I don't comment when I send a suggestion to its eventual administrative fate, but I wanted to let you know in particular why this one is currently set to 'deferred', since accessibility is *so* important to us. Basically, we have a bug open already to add more WAI-ARIA elements to the site, which (judging by my reading) will improve Dragon compatability bunches. So, the 'deferred' means 'deferred until we can do more WAI-ARIA work and then see what else needs to be done, once we've got that groundwork complete'.

[pinesandmaples]

but I wanted to let you know in particular why this one is currently set to 'deferred',

Thank you.

[birke]

I never responded to this comment! Thank you.

[denise]

and, routing back to this suggestion as i go through the 'deferred' list to re-evaluate them:

we've done our first and second passes for WAI-ARIA implementation, so hopefully you've noticed things getting better! if there are still things that are inaccessible to you through Dragon, we still want to hear about it, though: instead of making a new suggestion and going through that whole long drawn-out process, just post about it to dw_accessibility and we can evaluate what still needs to be done to make things work for you.

[birke]

I have noticed things getting better, actually! The specific problems I mentioned are gone now: I can follow links, whether internal or external, that appear on my reading page. It's exciting. Thank you so much.