February 19, 2002
Purpose of Workshop:
To Explore the Societal Innovation Potential and Realities of Natural Language and Interaction Interfaces: Extending Government Web Services To Transcend Differences of Language, Print Literacy, Technical Literacy and Physical Ability
Welcome
Susan Turnbull, from the General Services Administration (GSA), opened the meeting by welcoming everyone to the continuing collaboration workshop series. The series began one year ago from initial discussions with George Brett, Internet2. The purpose of today's workshop is to explore the societal innovation potential and realities of natural language and interaction interfaces. Multiple channels of service delivery are needed to transcend differences of language, print literacy, technical literacy and physical ability.
The workshop series is co-sponsored by the Federal Architecture and Infrastructure Committee of the CIO Council and the National Coordination Office for Computing, Information and Communications to open dialogue among seven affinity groups representing science and service disciplines. All affinity groups share a common purpose in exploring how to ease digital access disparities.
The workshop series contributors have completed a guide for public administrators – Extending Digital Dividends: Public Goods and Services that Work for All. Discussion is underway about how to make the guide available in the audio ebook format. It is anticipated that GPO may publish the guide with the audio ebook as a CD insert.
John Andre, of the CIO Council's Knowledge Management (KM) Working Group introduced their new CD which also includes a copy of the Extending Digital Dividends guide. The KM Working Group is one of the seven affinity groups that participates in the workshop series.
Susan described the long-term, anticipated results of the workshop series:
1. Better appreciation of realities and potential for accommodating human difference
2. Faster innovation diffusion
3. Better marketplace discernment
Susan noted that the workshop will adjourn at 4 PM today allowing for additional networking timeand an introduction to the UA Expedition On-line conferencing capability. (Sign up for this through Karl.Hebenstreit@gsa.gov. The afternoon session continued the monthly exploration of Open Source including a presentation on SELinux, by Peter Loscocco, NSA and SNARE, a Linux auditing tool for SELinux by Mark Westerman, Westcam, Inc. The Cyberspace Policy Institute, George Washington University is a co-sponsor of the open source sessions. Tony Stanco is the primary contributor to this effort from CPI.
Susan introduced Tom Lofft from the Cohousing Network, one of the speakers for next month's workshop.
Demonstration of How to Create a Natural language Interface to Web Content at EPA: VoiceXML and the Chemical Emergency Preparedness and Prevention Website – Presented by Brand Niemann, Computer Scientist EPA, Jim Stilwell, Senior Systems Engineer, and Greg O'Connell, Senior Sales Consultant, Tellme Networks, Inc.
Susan introduced Brand Niemann, Computer Scientist and XML Web Services Evangelist from EPA and Jim Stilwell and Greg O'Connell, from Tellme. Susan noted that the UA Expedition guide addresses the importance of XML, voice applications and networked improvement communities to advancing citizen-centric government. All of these components are integral to the EPA pilot of Natural Language Interface to Web Content led by Brand Niemann. Greg explained that Tellme's VoiceXML site can be accessed by telephone. This EPA pilot has won recognition and will be demonstrated at FOSE in March at the CIO Council Showcase of Excellence. (Brand provided a handout describing the project). Brand learned about the VoiceXML developer portal available through Tellme at a previous UA Expedition meeting. He used the Tellme developers tools and studio to pilot a voice application for EPA. Demonstration today will include the State of Utah's 511 Travel Advisories Service and EPA's VoiceXML pilot.
Jim began by discussing what Brand needed to develop his voice application – a place to perform some testing – and how the VoiceXML website met this need. The website, studio.tellme.com, has a developer portal available for no charge. There is also an 800 number that can be used to access this portal. The developer portal provides a scratchpad to hold application work. There is no charge to develop a VoiceXML application at the site because Tellme wants people to get interested in the potential of this capability.
There is industry support for VoiceXML applications and it is a W3C standard. Database companies and Customer Relationship Management (CRM) companies are offering VoiceXML interfaces to their products. VoiceXML interfaces with all the standards for Internet, Voice, WAP and others.
This is how VoiceXML works. To use the Internet usually an organization has a web server backed up with some business logic and some data access source and this web server interfaces with the internet so that network users can use URLs to go to different Internet sites. Tellme can add an audio layer to this structure that allows the company to support voice applications. Most important is that Tellme provides the telephony hardware as a service. Extra layers can be easily added to an Internet site for voice applications. VoiceXML is closely related to HTML so web developers can easily learn to work with the application. Applications can easily be extended to multimedia. This enables the developer to concentrate on building very powerful applications.
Jim then provided more details on the VoiceXML website supported by Tellme:
· studio.tellme.com
· Access is free and the site provides a free ID and PIN
· Provides information on VoiceXML, sample code and a scratchpad
· Can use scratchpad to cut and paste sample code (uses standard text editor)
· Can run test applications on a network that includes an 800 number with ID and security PIN,
· Provides a trouble/error log on the test application after processing
· Touch-tone and voice recognition capabilities are built into VoiceXML
· Voice application grammars are available to assist with development
Next, Greg demonstrated the State of Utah VoiceXML Travel Advisory Application. Some of the specifics for this system are:
· It took only 3 months to develop
· Uses the N11 dialing codes – 511 for this application
· Uses concatenated speech – a series of words linked together obtained from recording studios
· All prompts are voice prompts
· Included information about Winter Olympic events
Greg called the Travel Advisory number and demonstrated how his voice commands in response to voice prompts resulted in real-time, naturally spoken information on road traffic around Salt Lake City, directions to Olympic venues and other transportation options. He noted that the application can accept voice commands using different dialects. The application is developed to keep the communication exchange flowing to provide a more positive experience for the user.
Discussion:
Q: What voice engines are used?
A: VoiceXML uses Nuance Recognition software – Nuance 7 - that supports many different speech patterns and dialects.
Q: What is the efficiency of this application for voice recognition?
A: The average is around 50%. It varies though in different situations - ATT was using VoiceXML for 800 number programs with a 14% rate of efficiency, then contracted with Tellme to run the program. With improved technology, Tellme is currently pushing rates of 90% with savings in the millions for AT&T.
Q: Is VoiceXML able to interface with TTY?
A: When TTY capabilities are integrated into the web server the same information on the web can be presented in text to the TTY user.
Q: If you go in by voice, can Google be used to do the search?
A: The 1-800-555-Tell number will do this.
After the demonstration, Brand discussed his experience with VoiceXML. EPA's Chemical Emergency Preparedness and Prevention Office (CEPPO) needed to develop an application for spatial data. CEPPO was looking for a "faster, better, cheaper" method of using the CEPPO website and accessing the information in the Local Emergency Planning Committee's (LEPC) database. Brand explained that the VoiceXML pilot represents the "faster, better, cheaper" he and his managers wanted. This represents a "Killer App" that meets CEPPO's needs and is 508 compliant.
The resources required for this application were FileMaker Pro (cost of $150), the NXT 3 P2P software platform (approximate cost $1,000) and 3 staff days. Using FileMaker Pro made the project quicker and cheaper. Brand had previously discussed this project with EPA web developers who told him a similar project developed through EPA's web development program would cost $50K to $75K and take 3 months. In addition, this method of web content development would not support access by phone. This approach also supports distributed content management of web content.
The pilot application is viewable at the EPA website and demonstrates the delivery of XML data from the LEPC contact database. When the pilot application is accessed, it maps your zipcode and then logs it as a query. The pilot application's database has 3000+ records with 10 parameters. The database can easily scale to larger sizes. Using VoiceXML, EPA has created a LEPC contact database that meets Section 508 requirements at a reasonable cost.
EPA is working right now on an XML application for accessing location-specific census data by typing in a longitude, latitude and radius. The pilot application search interface uses one or more of the identified parameters to search the database.
Brand then demonstrated the EPA VoiceXML application on the CEPPO site . First, he brought up the EPA CEPPO website application and typed in the zipcode as a search parameter. The application searched the database and brought up the names and data for the LEPC at that zip code. Next, Brand called the number for the website and demonstrated the VoiceXML application. When the zip code was spoken the system responded with information about the nearest LEPC location by voice. This was the same contact information displayed at the website. Brand also showed the scripting and XML markup language used.
Discussion:
Q: Who owns the data?
A: Congressional legislation requires EPA and the States to do two things – 1) States are to put the LEPCs together and, 2) EPA is responsible for keeping LEPC data current and accessible. In this environment EPA and the States co-own the data.
Q: Is Lexus-Nexus set up for voice capabilities?
A: Brand has heard that Lexus-Nexus has recently bought the NXT 3 platform to provide this channel of access. The value proposition of XML has three main elements: 1) it "future proofs" your information against periodic technology change; 2) it allows you to do XML web services; and, 3) it allows you to deliver XML content to alternate devices.
Q: Are there bilingual capabilities for voice applications?
A: Higher level software platforms can support bilingual capabilities.
Q: Is there use of radio?
A: This can happen with XML, but you need to determine what alternate devices the system will deliver information to and break the information into "nuggets" that can be used by those alternate devices.
Q: What is the status of voice application standards?
A: Applications developed are proprietary, but standardized tags are used by all companies developing these products. The companies want to avoid proprietary tags because they create problems in the future due to rewriting of code in these voice applications.
Comment: Voice applications would be great for updating federal government telephone directories and the Federal Blue
*****NOTE: The EPA VoiceXML application was one of the finalists at FOSE's Federal CIO Council Showcase of Excellence and was demonstrated at the FOSE exhibition March 19-21, 2002, award winners were recognized by the Quad Council on March 20, 2002. On March 20, this application was chosen by the Quad Council for top innovation honors. See http://www.gcn.com/vol1_no1/daily-updates/18224-1.html and www.xml.gov
Natural Interaction Interface: Bringing Web Content to Communities in a Manner that Transcends Differences of Language, Literacy and Physical Ability – presented by Neil Scott, Director of Archimedes Project, Stanford University
NOTE: Neil's presentation can be accessed at ftp://ftp-csli.stanford.edu/pub/Arch/ingmarie/ and then click on GSA_Pres_Feb19.ppt. Please be aware that it is large file and will take several minutes to download.
Susan introduced Neil Scott, Director of the Archimedes Project at Stanford University. The Archimedes Project seeks to promote equal access to information for individuals with disabilities by influencing the early design stages of tomorrow's computer-based technology. The project's website is http://archimedes.stanford.edu//arch.html.
Neil explained that the Archimedes Project was founded to provide leverage to disadvantaged individuals using existing and developing technologies. Many of these technologies have much broader uses than are currently employed. The project is primarily focused on discussing people's access needs as an important aspect of their overall well-being.
There are many people in the world and all are different in some way. Information technology tries to treat people as all the same. The project looks at people's goals, which are good health and well-being. There are problems in achieving these goals, but there are solutions and the solutions, in main part, are information.
Computers are artifacts - they get between you and the information. In the future, access devices like computers will get cleaner and more intuitive. Right now there are different types of computer users:
· Normal
· Disabled
· Aging
· Illiterate
· Poor
· Remote Locations
Sometimes providing the right incentives can get a certain user group going. In Palo Alto there is a group of elder Internet users called the Silver Surfers. With the appropriate incentives this elder user group became extremely active users of the Internet.
Well-being is the ability to manage daily issues effectively, having good health, a good education, and opportunities for leisure activities. Some people don't have this status, for example, disabled children. Problems are due to insufficient resources, illiteracy, and lack of a job or other skills. These are the things needed to just exist and when they are not present there are problems.
Sources of information are folklore, religion, schools, libraries, and NGOs (in other countries these can be very important). The Internet can be a source of information, but it is very unstructured. If you know what you are doing the Internet is a treasure trove. However, there are some issues with access for some – for those that have difficulty moving or seeing, those that are too remote or poor. Even with access, some users are faced with incomprehensible screens and an imposing amount of unstructured, poorly ordered information.
Trying to do something simple for access to all this information can be very difficult. We must change the way we interact with information. In many cases we are getting more information pumped at us than we need. There are three essential components to a better structure for obtaining information.
1. Simple Appliances
2. Intelligent User Interface
3. Structured Knowledge Repository (places to get knowledge, not just information)
Simple Appliances
· Have to be simple, such as a tablet computer (looks like an Etch-a-Sketch with icons) with point and speech – conventional software is too complex.
· Why tablet computer rather than PDA? PDA screens are too small right now
· Simple web browser with flash playback is usually sufficient
· As they develop simple appliances, we realize that they can be the prototype for many more appliances
· Simple does not mean dumb!
· Different input and output modalities become possible
· They need to be low cost and long lasting. Certain communities, such as people with disabilities, cannot repeatedly go back to the well to pay for constant upgrades
Intelligent User Interfaces
· Just starting to get there with voice recognition. Soon it will be done all in one sentence.
· Interface must adapt to the user rather than forcing the user to adapt to the computer.
· In Japan they have developed software that uses natural language – one sentence is spoken for the request.
· New ways are being found to incorporate natural interaction – body feedback, non-verbal signals etc.
· Look at: What is this person doing? What does this mean? What do we need to do? This is a distributive agent network.
Structured Knowledge Repository
· Put information in forms that can be used in intelligent ways.
· Have filters to take out clutter.
· Provide purpose-designed knowledge repositories.
Neil described a program sponsored by Reuters that asked nine scholars to identify a global problem and develop a solution. One of the scholars identified a situation in India involving villages in a certain region that needed information on health, agriculture and weather issues. The Indian government has laid 35,000 KM of fiber optic cable, but 75% of villages in India are not able to access this network to obtain information. These villages, on average, have a six-mile walk to get to an information source for help with a sick child or water buffalo (a long way to walk carrying a sick child, even longer carrying a sick water buffalo!). In these villages, 98% of the women, and 88% of the men, are illiterate.
The proposed solution was to start with local people who are domain specific knowledge repositories and have them use simple interfaces with the villagers based on speech icons and gestures. However, one of the major issues with this approach is the introduction of people and groups of people with different dialects, or even languages. When this occurs it is necessary to introduce a human mediator to help understand what the person needs and their information requirements. The mediator also can add new experiences to the knowledge repository to maintain its usefulness and relevance, which is very important in this situation.
The next step is to look at this local knowledge being identified and gathered to determine how to expand this into regional knowledge. Then the knowledge repository can be pushed higher to the country-wide level. At this level there will be holes in the repository. What they need to work at then, is filling gaps by finding a way to effectively link a local/regional/country knowledge repository with other communities (e.g. other countries) to expand and grow the knowledge repository base.
Neil introduced Bob Gauldie from the University of New Zealand so he could describe some of his experiences with the knowledge repository concept. Bob noted that New Zealand is very good at packaging information (e.g. agricultural information) and selling it to other individuals or groups. They adhere to Steaming's dictum "Survival is optional when change is not compulsory." With that in mind, New Zealand has become very proficient at controlled change at the bottom end of the worker cycle by doing things such as health and training upgrades, moving a worker from outdated technologies to capabilities with current technologies. Because of this ability, the country of Chile has asked New Zealand to help with its bottom economic work group, particularly on expanding technical capabilities. In Chile they have a ratio of five graduate level workers to one technician and they need to adjust that ratio to increase the denominator. New Zealand is also good at electrical system management and exports this capability by managing Finland's and Vietnam's electrical power systems.
Total Access System
Neil moved back to his presentation to discuss the concept of the Total Access System (TAS). A TAS covers people that need special tools to access information for a long period of time. Neil looked at the target systems – lights, computers, an appliance, etc. – and decided to leave these alone. Unfortunately, these target systems create a hostile environment in many ways. For example, computers are sold to make a profit and there is usually no attention paid to access issues. He has separated out these access issues and decided that the computers and software are someone else's concern.
Instead, the Archimedes Project focused on how the person would interface with the computer and developed the Total Access Ports (TAP) technology. TAPs are inexpensive devices that can be built-in or added on to existing computers. The access device (accessor) best suited for the individual links into TAP and the user can then access the information. Input from the accessor through the TAP emulates a keyboard and mouse. Using the TAP, video output from the computer can be displayed from the accessor in a manner appropriate to the user.
Neil described what universal access should be for everybody. Universal access is like glasses that allow the user to read and connect with the information. Prescription glasses are attuned to an individual and handle variations encountered by that individual (except for those extreme temperature changes that cause fogging!). Such glasses always work properly and are always with the individual. Any upgrades are done by choice, not because of arbitrary outside decisions.
Project Archimedes has focused on developing accessors – devices to allow all users access to information. Neil described several categories of accessors, such as:
· Speech Recognition – many advances being made
· Eye Tracking – was considered the "Holy Grail" for a time, but is very expensive even now; can be very useful for certain kinds of research
· Cartoons – somewhat like sign language – explores another method to communicate with people using different languages
· Haptic Output – allows the user to feel things in the display. Uses a "Moose" – a mouse with muscles to let a blind user to feel the screen layout and when combined with speech output, navigate to desired command icons.
· Musical Icons – musical chords are associated with different icons
TAP technology can be used with different machines – PCs, MACs, SUNs etc. Now, the project is looking at multiple ways to interface with each platform
There are three technologies used in the TAS project.
1. Neural Network Processors
2. Natural Language Processing (to learn the users' intention)
3. Distributed Agents (determine what information needs to go where)
With respect to the Neural Network Processors, they recently have worked with an inexpensive silicon recognition chip that mimics brain processing – the Zero Instruction Set Computing (ZISC) chip. The chip is very useful to identify subtle head and eye movements, that in turn, contribute to recognition of facial expressions. ZISC is a natural fit for command and control voice recognition. The ZISC chip will have an impact on all access technologies.
Distributed agents link accessors to TAPs and accessors interact with other computer-based devices through TAP. Using the distributed agent model, half of the network can be in TAP for whatever type of machine is used. The other half of the network can be used by the accessor. As opposed to single agent networks, distributed agent networks find the channel to perform the command. The distributed agent network automatically configures itself to the user's needs, abilities, and preferences.
One of the projects being developed currently is a TAS for the Sony Game system. Another uses tablet computers to help healthcare workers and elderly individuals in healthcare programs gain access to patient and health information.
Archimedes Access Research and Technology International (AARTI) - This group is focused on finding ways to fund the efforts of the Archimedes Project. Currently, there is distribution and cloning of the Archimedes Project in Japan, New Zealand, Ireland and England. They can use the strengths in each place to create complete solutions – developing a global collaboration network.
Discussion:
Q: For the musical chords accessor system, who decides the standards?
A: Let other people decide the standard formats. The people using the accessor can decide which "themes" to map to which icons using the standard formats.
Q: At nursing homes, residents must frequently wait for assistance and compassionate care is sometimes lacking. Part of this may be solved by technology – can this project contribute?
A: Hope to explore a similar scenario in Japan as part of a grant application to be submitted soon. The project would seek to advance the safety and well-being of nursing home residents.
Neil shared a story that was relevant to this question. There was a patient in a hospital who was refusing to eat and very despondent. The patient had severe physical ailments that did not allow him to talk or even gesture easily. The hospital staff only did the basics to maintain the patient because there was no way to figure out what his needs were. Neil was able to let him use the tablet computer accessor for communication. That subtle addition that transformed the situation. Now the patient, who had been a computer programmer, was able to communicate. The patient's sprit returned and he began eating again. Neil noted that the Archimedes Project wants to use and improve technologies to allow all people to gain and maintain independence.
Workshop Attendees and Comments:
Name and Organization
Position
E-Mail
Comments
Susan Turnbull – GSA
Senior Program Advisor
Susan.turnbull@gsa.gov
Would like to make the guide available using the Digital Talking Book technology.
Janina Sajka – American Federation for the Blind
Director of Research and Development
Janina@afb.net
Would like to enhance the design of environments (physical and cyber) that are responsive to need of individuals who are blind. Leader and expert in Digital Talking Book technology
Mark Westerman – Westcam, Inc.
mark.westerman@westcam.com
Will be speaking on an open source tool in the afternoon session
William K. Moore
Consultant
Wmoore@desc.dla.mic
President of a community-based group focused on computer and software training.
Greg O'Connell – Tellme Networks
Senior Sales Consultant
Grego@tellme.com
Provided presentation on VoiceXML.
Jim Stilwell – Tellme Networks
Senior Systems Engineer
Jstilwell@tellme.com
Provided presentation on VoiceXML.
John Huth – John J. Barcklow Foundation
Research Gerontologist
john.huth@verizon.net
Foundation is focusing on improving the level of compassion and quality of care in assisted living programs
Owen Amber – US Fish and Wildlife Service
CIO Council XML working group
Amber@erols.com
Focusing on the structure of a knowledge repository. Very interested in the issue of records management and how not to waste human knowledge.
Karl Hebenstreit – GSA
Systems and Accessibility Specialist
karl.hebenstreit@gsa.gov
Involved with access issues and the Telework project. Supporting the Quickspace effort for the workshop.
Shannon Collins – Veterans Health Administration
Shannon.collins@hq.med.va
Wants to explore and increase ways for Veterans to access health information.
John Andre – GSA
Knowledge Manager
john.andre@gsa.gov
Wants to find ways to enable knowledge creation and sharing.
Tom Lofft – The Cohousing Network
Tlofft@hotmail.com
Neil Scott – Stanford University
Director, Archimedes Project
Ngscott@arch.stanford.edu
Bob Gauldie – University of New Zealand
Associate Dean of Science
Rgauldie@actvix.gen.nz
Was struck by the nursing home example. He comes from a multi-cultural environment so has an interest in enabling diverse people to communicate well with one another
Mark Frautschi –
Independent Consultant
Frautschi@bigmindmedia.com
Likes to see groups making connections. Knowledge Management encourages linkages between two or more groups.
Peter Gallagher – devIS
Pgallagher@devis.com
Interested in internetworking applications.
Tony Stanco – George Washington University, Cyberspace Policy Institute
Tony@freedevelopers.net stanco@seas.gwu.edu
Very interested in open source software issues.
Ling Wan – Environmental Protection Agency
wan.ling@epa.gov
Involved with technology and applications agency will use to meet its mission
Nick Guzman – George Washington University, School of Medicine
Associate Professor of Medicine, Director, Diabetes and Kidney Care Program, GWU
Guzmann@gwu.edu
Exploring new and simple ways to deliver health information to minorities and Hispanic populations.
Bob Andrew – ICF Consulting
Bobandrew@icfconsulting.com
Has facilitated large groups up to 30,000 giving him a perspective on all types of communication issues. One of the large groups he facilitated resulted in the "idea into practice" of lasik surgery.
Craig Holcomb – NSA
Technical Director, CIO Office of ITMRA Governance
Lcholco@nsa.gov
Interested in Section 508 requirements and affect they will have on the NSA.
Peter Loscocco – NSA
Project Leader, SELinux
pal@epoch.ncsc.mil
Led development of security enhanced Linux
Jim Disbrow – US Department of Energy
jim.disbrow@eia.doe.gov
Involved in policy development and DOE's Kid's Page
Grant Wagner – NSA
gmu@tycho.nsa.gov
Interested in protecting information
William E. Smith – ODII
aic@adii.com
Interested in addressing and resolving large complex problems.
Lowell Christy – Chairman, City of Mind
lowell@cityofmind.com
Exploring cross-cultural communications as a learning tool for youth
Mark Addleson – GMU
Director MNPS of Organizational Learning, (703) 993-1142
Exploring reframing how we think about organizations and management and the need to look at relational approaches rather than transactional approaches.
Paul Mrochinski – EDS
Managing Consultant, E-Government Community of Practice
paul.mrochinski@eds.com
Lisa Xantus – EDS
Consultant
lisa.xantus@eds.com
Dion Anderson – EDS
Consultant Specialist, E-Government Community of Practice
dion.anderson@eds.com
Providing note-taking support to the workshop