People often think of Silverlight as something that hinders, rather than helps, creating accessible sites. This post looks at a way in which Silverlight can make your standard HTML based web application more accessible by adding contextual audio help.
Here’s the scenario.
In order to help people use your web application you’d like to provide contextual audio help. Users should be provided with audible help as controls on the page receive focus. The approach should be easy to implement and to apply to any existing HTML based web application.
There are a few ways to incorporate speech into your Silverlight applications.
- Speech Application Programming Interface (SAPI) – the SAPI component is installed on all Windows operating systems since Microsoft Windows NT 4. From Silverlight you can instantiate a SAPI Voice object using ActiveX and JavaScript. The Voice class exposes methods to convert text to speech. This approach is used by ButtercupReader and is extremely fast since all the processing happens on the client. You also have access to all the functionality of the SAPI including controlling pitch and speed and being able to spell out words. However there are some limitations.
- It will obviously only work on Windows clients that have the sapi.dll installed
- As it requires you to use ActiveX object instantiation, it will only work in IE
- It requires your clients to relax some of the default IE security settings to allow the use of ActiveX object instantiation
- Server Based Text to Speech – this option involves sending the text to be spoken to the server to convert to an audio stream that can be used as the Source of a MediaElement. This option requires you to consider the load that will be placed on the server from all you clients and the (possibly lengthy) time required to convert the text. However, this option has the advantage that it will work on any client that supports Silverlight.
- MediaElement – you can of course record the help as individual audio files (e.g. mp3). These could be placed on the server and referred to by the controls in your application. This Option is pretty simple to implement but isn’t as flexible as the other options and requires you to author the audio files manually.
Options 1 and 2 could both be implemented to work against HTML that looked something like,
<input id="username" title="Your usename is the email address you registered with" />
In Silverlight you could hook into each control’s onfocus event, determine the value of the title attribute and convert/speak the text.
Option 3 could be achieved by adopting a naming convention that associated an HTML element with a media resource by its id attribute. For example, when the above input field received the focus the Source of a MediaElement could be set to usernameHelp.mp3.
I’ll go over the implementation details for Option 2 in more detail below and I’ll be posting the code for ButtercupReader on Codeplex shortly so you can see how to implement Option 1 there.
I came across this project on Codeplex that does a great job of converting text to speech. It includes a WCF service that accepts the text to convert and returns a WAV stream as a byte array. On the client the byte array is converted to a MediaStreamSource by the WAVMediaStreamSource class. For more details check out this article on MSDN.
So all you really need to do is create a Silverlight 3.0 application and change the Main.xaml UserControl to contain a single MediaElement.
<UserControl x:Class="SelfVoicing.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008" xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d" d:DesignWidth="640" d:DesignHeight="480">
<Grid x:Name="LayoutRoot">
<MediaElement x:Name="audioHelper"></MediaElement>
</Grid>
</UserControl>
In the code behind we need to hook in to the onfocus event of the HTML elements on the page that the Silverlight control is hosted within.
void MainPage_Loaded(object sender, RoutedEventArgs e)
{
// Hook up to the onfocus event of all HTML input elements
foreach (ScriptObject scriptObject in HtmlPage.Document.GetElementsByTagName("input"))
{
string id = scriptObject.GetProperty("id").ToString();
HtmlPage.Document.GetElementById(id).AttachEvent("focus", new EventHandler<HtmlEventArgs>(DoControlGotFocus));
}
}
The DoControlGotFocus method contains the code that calls the Speech service and sets the Source of the MediaElement.
private void DoControlGotFocus(object sender, HtmlEventArgs args)
{
HtmlElement element = sender as HtmlElement;
if (element != null)
{
SpeechWCFService.SpeechServiceClient client = new SelfVoicing.SpeechWCFService.SpeechServiceClient();
SpeechWCFService.CreateWavStreamRequest wcfRequest = new SelfVoicing.SpeechWCFService.CreateWavStreamRequest();
wcfRequest.Text = element.GetAttribute("title");
client.CreateWavStreamAsync(wcfRequest);
client.CreateWavStreamCompleted += new EventHandler<SelfVoicing.SpeechWCFService.CreateWavStreamCompletedEventArgs>(client_CreateWavStreamCompleted);
}
}
void client_CreateWavStreamCompleted(object sender, SelfVoicing.SpeechWCFService.CreateWavStreamCompletedEventArgs e)
{
MemoryStream ms = new MemoryStream(e.Result.SoundStream);
WAVMediaStreamSource ws = new WAVMediaStreamSource(ms);
audioHelper.SetSource(ws);
}
And that’s about it.
Add this block to any web page to magically get self voicing working – easy!
<object data="data:application/x-silverlight-2," type="application/x-silverlight-2" width="0px" height="0px">
<param name="source" value="ClientBin/SelfVoicing.xap"/>
<param name="onError" value="onSilverlightError" />
<param name="minRuntimeVersion" value="3.0.40624.0" />
<p class="help">
This form can be enhanced to provide audio feedback. All you need to do is <a href="http://go.microsoft.com/fwlink/?LinkID=124807">
install Silverlight</a> - it will only take a few seconds.</p>
</object>
Full source code here.