D3 - Assessment of Communication Function

Task Leaders: Bambang Parmanto, Ph.D. and Corinna Lathan, Ph.D.

Co-Investigators: J. Scott Yaruss, Ph.D. CCC-SLP and Ellen R. Cohn, Ph.D. CCC-SLP

Other participants: Patricia Ourand, M.S. CCC-SLP


The overall aim of D3 is to create a telerehabilitation (TR) environment to allow Speech/Language Pathologist (SLP) to remotely train, facilitate, and evaluate the child performing speech/language "homework" exercises using computer software and interface devices while the child's parent is providing additional encouragement, reinforcement, rewards, as trained by the therapist.

The objectives of the project are:


Using the infrastructure developed through D1 Project, the aim of D3 project is to provide telemonitoring via a store-and-forward method over the Internet where the therapist monitors the child upon completion of a therapy session. Store-and-forward monitoring provides the therapist with a means to evaluate the child's progress in performing at-home therapy exercises. Telemonitoring provides a solution for no therapy or infrequent therapy due to location, transportation, or clinician availability issues. This approach enables the therapist to accurately and objectively assess the child's ability, dedication, and interest in performing therapy exercises at home, as opposed to relying on the child's and/or the parents' reporting of therapy homework. Furthermore, this type of monitoring allows for the therapist to add to the body of evidence-based practice and to adjust therapy to better suit the needs and interests of the particular child.

The use of centralized database can also be used to collect data on users over time or aggregate data across users. This is important in light of the increased pressure for clinicians to perform and confirm subjective results with detailed data collection and analysis.


CosmoBot: An Innovative Computer-Based System for Play-and-Learn Speech-Language Therapy

The CosmoBot system (developed by AnthroTronix Inc., Silver Spring, MD) is designed for use in rehabilitation and educational settings. The system's primary software is Cosmo's Play and Learn computer game featuring a robot character named Cosmo. Children of developmental ages 3-5 years work on their developmental goals while playing interactive software games featuring Cosmo. A primary objective, particularly for children with severe disabilities, is to encourage them to interact as much as possible with their environment. CosmoBot system's Play and Learn is designed to work on pre-literacy and pre-numeracy, with particular learning objectives including receptive and expressive language goals, listening and attending, spatial awareness, recognizing sets and numbers, cause & effect, word recognition, recognizing symbols, and active working memory.

Figure 1

Figure 1. Cosmo's Play and Learn Screen Shots.

The game's main screen (upper left) is the "hub"-here, the child chooses which planet Cosmo will visit (remaining screens). There are various activities and levels of difficulty on each planet, providing a variety of skill levels, targeted goals, and interests.

The interface to the CosmoBot system is Mission Control, an interactive control station. Mission Control is a child-friendly, fun computer interface system with four large pressure-sensing buttons and a microphone. Mission Control plugs directly into the USB port of a PC, and can be used to play any joystick, keyboard, or mouse-based game or use any switch-enabled software. Mission Control is expandable through external ports - four mono-plug jacks for external binary switches and two USB ports for external USB devices. CosmoBot can be controlled through gestures and voice with the use of the Mission Control interface device and modular, gestural components that can be plugged into Mission Control.

Cosmobot Mission Control

Figure 2. Cosmobot Mission Control

Telemonitoring Architecture

Expanding the CosmoBot system into a telerehabilitation service is challenging due to the limited network support provided by the CosmoBot system's platform. The CosmoBot system was developed using a software platform that emphasizes graphical interaction and animation capabilities, but has minimal network communication capabilities. To overcome the limitation, we divide the data transfer into two components sitting in both ends of the client and the server, and by using XML (eXtensible Markup Language) document as a container for transferring data. ?The desktop-based component is a local application which interacts directly with CosmoBot and sends the data to the remote server. The server-based component is a server application which waits and receives the data. Upon receipt, the server component processes the data and stores the data into the database. The server-based component is also responsible for creating a dynamic interface that allows a therapist to access the data remotely.

Figure 3

Figure 3. Cosmobot Telemonitoring Infrastructure.

The server hosts a database system that stores the educational database, demographic and clinical data of the child, and individual progress of the child. With the aggregator system working on the application on the server, therapists can assess the progress of the child and adjust the game settings to his/her progress and abilities. Based on the progress, the therapist can tailor the individual course of therapy for each child. He/she can also send encouragements, suggestions, and other educational materials to the child over the connection channel. Once completed by the therapist, these settings are sent from the server to the child's computer and presented the next time the child has a play session with CosmoBot.

Figure 4

Figure 4. Telemonitoring Portal.

Secure Monitoring Protocol: Store-and-Forward

The goal of the telemonitoring system is to provide automatic and secure data transmission between the CosmoBot systems in children's homes with the server. To ensure that the transmission is secure, the client and the server have established an XML schema for the transmission. In addition, the XML files are transferred on top of secure HTTP.?A valid XML file is required to exchange the data between the CosmoBot Desktop and server applications. An XML file is considered valid if it conforms to its schema. Figure 4 shows the pre-defined XML schema for data exchange purposes.



Completion of the 'tele'-architecture. With the addition of the 'tele' aspect into CosmoBot, we were able to modify the system into a tele-rehabilitation system


Finished developed "Patch" for Cosmo's Learning Systems to allow automatic data upload after each session


Presented findings from 2007 at the 2008 Assistive Technology Association (ATIA)

2009/Future Projects

In Fall of 2009, we will implement a similar testing protocol as utilized in 2007 with Cosmo's Learning Systems. The difference in protocol is that the technology will be tested in a school environment. In 2007, we recorded the source code of the Cosmo's Learning Systems software to send data collected during game play to the secure University of Pittsburgh server. The data transfer system allows us to automatically send the data from the children's classroom environment to the center's server via internet. Children who receive speech language therapy one or two times weekly will use Cosmo's Learning Systems during the school day. Data will be transferred to the secure server. A speech therapist will be able to remotely analyze data on days when she is not present at school and recommend changes to the software based on this data. Two potential locations have been identified and the IRB is pending based on location confirmation.

AnthroTronix, Inc. will continue to participate in research related to telerehabilitation using Cosmo's Learning Systems as well as new technology products. For example, we have launched the AcceleGlove, a gesture capture glove, and plan to submit proposals to interested funding sources to validate the technology for use in the home environment.


Study with kids with Autism in July/August 2006


  1. AD-02 13 years old, some single word comprehension and production
  2. AJ-04 12 years old, echolalia, nonverbal, inconsistent comprehension
  3. AI-05 11 years old, some visual matching, inconsistent comprehension
  4. AV-06 10 years old, verbal, working memory impairment
  5. AZ-07 9 years old, some single word comprehension and production
  6. AC-01 6 years old, verbal

Pilot Study Result with Children with Autism in 2006

Figure 5. Pilot Study Result with Children with Autism in 2006

Summary of Results:

Study with children with disabilities at home Jun/Jul/Aug 2007


Summary of Results:

We evaluated the tele-assessment portal using Cosmo's Learning Systems in a home environment. Several issues were identified that were specifically related to the technology product (CLS) used. Several families were not able to employ CLS on a regular basis which was important to the tele-monitoring portion of the project. We believe that the addition of an off the shelf software program with Mission Control as an interface device with children who have been diagnosed with autism is an appropriate next step.



Lathan, C. & Safos, C. (2007). From research to market: Commercializing a user-centered project. Human Factors and Ergonomics Society Bulletin, 50 (1), 1-2.

Parmanto, B., Saptono, A., Murthi, R., Safos, C., & Lathan, C. (2008). Secure telemonitoring system for delivering telerehabilitation therapy to enhance children's communication function to home. Journal of Telemedicine and eHealth, 14 (9), 905-11.


Lathan, C., Boser, K., Safos, C., Frentz, C., & Powers, K. Using Cosmo's Learning System (CLS) with Children with Autism, Presentation in Learning with Disabilities 07 Conference, July 19 - 20, 2007, Ohio.

Lathan, C. Cosmo's Learning Systems, an educational tool for children with disabilities, Demonstration in 2008 ATIA conference, January 2008.

Saptono, A., Parmanto, B. An Integrated Telerehabilitation Infrastructure to Support Speech-language Therapy, Proceeding of American Telemedicine Association 12th Annual International Meeting, May 13-15, 2007, Nashville, Tennessee.[S3]


Video of AnthroTronix, Inc - Assessment of Communication Function


The RERC on Telerehabilitation State of the Science Conference


D3-01: Assessing "Off-the-Shelf" Software for Telerehabilitation: Guiding Principles and Best Practices

Ellen R. Cohn PhD, J Scott Yaruss PhD, Corinna Lathan PhD, Amy Brisben PhD, Bambang Parmanto PhD, David Brienza PhD

University of Pittsburgh, Pittsburgh, PA

D3-02: Pandemic Preparedness: Is there a Role for Telerehabilitation?

Ellen R. Cohn, PhD

School of Health and Rehabilitation Sciences, University of Pittsburgh, Pittsburgh, PA

D3-03: Virtual Public Service: Dissemination of Health and Telerehabilitation Content

Ellen R. Cohn, PhD

School of Health and Rehabilitation Sciences, University of Pittsburgh, Pittsburgh, PA

D3-04: Telerehabilitation E-Dissemination Opportunities: Three Vehicles for Academic Public Service

Ellen R. Cohn PhD, David Brienza PhD, Michael McCue PhD, Michael Pramuka, PhD, CRC, Timothy S. Deliyannides, MSIS

University of Pittsburgh, Pittsburgh PA

D3-05: Telerehabilitation Practice: A Curricular Imperative for Future Audiologists

Ellen R. Cohn, PhD, [1] Mark Krumm, PhD, [2] Deborah Theodoras, PhD, [3] Michael McCue, [1] and Barbara Vento, PhD, [1]

University of Pittsburgh [1], Kent State University [2], and University of Queensland [3]

Return to RERC TR 2