[Home]AutoTypeIntroduction

HomePage | RecentChanges | Preferences | My Website home page

Showing revision 46

AutoTypeIntroduction - Soak testing the Text Relay App

To make phone calls, the deaf community use a textphone to TYPE and READ what that want to say. The relay operator SPEAK and HEAR with the VOICE telephone user.

A user of the Relay UK App types what they want to say while the Relay Advisor types the responses from the VOICE user.

This call should work for as long as the callers want to talk. It could be a over an hour.

Long duration soak test calls needs typing approximating real Relay calls.

This webpage is about how to use some automated USB keyboards to automate typing during a long duration test call testing the Text Relay App.

Set up a test call manually and then enable automated typing for the duration of the test call.

This solution is a "SMART" automation tool ( adapted from SMART goals. )

 SMART stands for:
 * Specific: - A solution to enable text to be typed back and forth exists.
 * Measurable: - repeating text that can be seen in call logs
 * Achievable: - Tools are available so a realistic goal that can be accomplished.
 * Relevant: - fullfils the need to send a stream of characters back and forth
 * Time-bound: Set a deadline for achieving the goal. The proof of concept can take as little as two hours to demonstrate.

Some limitations are discussed later.

The automated keyboards can use Arduino Leonardos, DigiSparks, or Raspberry PICO.

They can be programmed as a USB Keyboards to type a repeating set of phrases during long duration Text Relay test calls.

The deaf community use a textphone to TYPE and READ and a relay operator SPEAK and HEAR.

Users can now use a smartphone App to TYPE and READ.

Some users use a combination of TYPE and READ and SPEAK and HEAR.

The Relay Advisors use a Workstation and READ off an LCD screen and TYPE using a keyboard and mouse

For the TextRelay, the TYPE and READ user and Advisor use a keyboard to communicate, hopefully in turn.

Long duration calls would normally have typing back and forth between the users via the Relay Advisor.

The users look at the screen to select the input box where the keystrokes go.

When it is their turn they type a phrase, pressing GA at the end.

The Advisor reads out the typing and types what they hear, word by word.

Quite how a real conversation takes turn, requires the parties to "listen" for a clue that it is their turn to type.

To simplify this, the users can type GA - "Go Ahead".

Automation has been a goal, but it is not that easy.

How does your simple automation tool read a smartphone screen or press buttons on the on screen keyboard?

How does the automation "hear" the GA?

What is available?

The Arduino Leonardo can be programmed as a USB Keyboard.

https://docs.arduino.cc/language-reference/en/functions/usb/Keyboard/

The Arduino Leonardo can also be programmed as a USB Keyboard and mouse, and it has digital inputs and outputs and serial ports.

A keyboard cannot see the screen. Use a wire to enable and co-ordinate blind typing

The USB keyboard cannot "hear" the other end or read what they are typing to wait for GA or a pause.

They have inputs and outputs which can be used to enable and co-ordinate when they are typing.

So the AutoType devices asserts an output low while it is typing and removes this assert when it stops typing and goes IDLE. It holds off starting to type until it sees IDLE.

Use one for the User and another one for the Advisor.

An Arduino Leonardo, Raspberry PICO, and DigiSpark can be programmed to press keys on a keyboard

You need to use the mouse to select the input box that you type into, then enable typing.

It is possible to use an external keyboard and OTG cable plugged into a smart phone.

Automating this is possible

Automating these is near impossible

Automation could be simplified if an agreed sequence of keystrokes triggered test events.

Below is a collection of Atmel 32u4 bases Arduino Leonardo and DigiSparks.

A Raspberry Pico can also be used.

history

The Arduino Leonardo has Keyboard examples.

https://github.com/arduino/ArduinoCore-mbed/blob/main/libraries/USBHID/examples/Keyboard/Keyboard.ino

I brought some DigiSparks off eBay for 1.69 GPB - possibly for a model railway project.

They could be programmed using the Arduino IDE and came with examples.

An OTG cable can be used to plug an external keyboard into a smart phone.

Here are a collection of DigiSparks, Arduino Leonardo and other boards that use the Atmel 32u4 chips that support USB.

There was a need to soak test the App with what every typing could be acheived, and the DigiSparks and example were available.

These pages write up some notes and ideas.

AutoType - Automating pressing keystrokes using DigiSparks , Arduino Leonardo and Raspberry Pico

DigiSparks - DigiSpark - https://www.instructables.com/Digispark-DIY-The-smallest-USB-Arduino/

The Arduino documentation had an example that sent keyboard keystrokes.

One example was:

https://github.com/arduino/ArduinoCore-mbed/blob/main/libraries/USBHID/examples/Keyboard/Keyboard.ino

The DigiSpark also came with a KeyBoard example

https://github.com/digistump/DigisparkArduinoIntegration/blob/master/libraries/DigisparkKeyboard/examples/Keyboard/Keyboard.ino

I modified it, and here is a version just for the DigiSparks.

DigiSpark sketch ( 2025-09-04 ): http://www.dougrice.plus.com/dev/DigiSpark/digi3/digi3.ino

It types slowly, and a switch connected to input P2 can be used to enable / disable typing the next phrase. Shorting P0 to GND HALTS typing

After you position the cursor and select the input box where you normally type, plug the device in and it starts typing the list of phrases.

You can optionally use two or more DigiSparks connected together so they take turns typing.

They wait for P2 to go IDLE before typing their next phrase.

Getting the code to work on the different boards has made it a bit cluttered and over complicated.

Example text typed, It took about 100 seconds to type the text below

 ts: 00009761 ms GA
 This is BeeTea Engineer, DHR, making a test call.GA
 Please hang up the B-leg GA
 I am testing the new Relay UK app GA
 Hello Doug here GA
  Can you see my typing? GA
  The Quick brown fox jumps over the lazy dog GA
  I am testing the new Relay UK app GA
   ts: 00113746 ms GA

A Telephone call Script

Caller A dials B and has a conversation using TYPE and READ.

A script

Caller A decides to call B

Caller A picks up phone and hears Dial tone

They dial the number and hear ring ring

B heads the ringer and pick up the phone

A hears the ringing stop and B says hello

A says hello and they talk back and forth.

For the TextRelay, the TYPE and READ user and Advisor use a keyboard to communicate, hopefully in turn.

The script could be coded into a computer and send commands to the dialler and text to speech units.

It would be possible to write a Script and run this on a computer that sent commands to Arduino or Raspberry PICO programmed as Keyboards and the mouse.

To prove the point the Arduino Keyboard example is a good place to start:-

https://docs.arduino.cc/language-reference/en/functions/usb/Keyboard/

At its simplest, a switch to turn the typing on and off and a loop of typing was required. This was acheivable.

The more complicated senario are awaiting developement.

conclusion

For the TextRelay, the TYPE and READ user and Advisor use a keyboard to communicate, hopefully in turn.

The automated keyboards cannot read the screens.

Using an Arduino or Raspberry Pico as a Keyboard and a bit of wire to synchronise the two, some automation is possible.

Soak testing of a set up call was possible and this could be left to load up the scroll buffers on the App and Advisor workstations.

The conversation is between two humans, so their needs limit the maximum duration required.

http://www.dougrice.plus.com/dev/DigiSpark/indexImages.htm

Below is a picture of various Atmel 32u4 based Arduino Leonardo variations AutoTyping into the App.

Possibly using this sketch: http://www.dougrice.plus.com/dev/DigiSpark/KeyboardRP2/KeyboardRP2.ino

Using a USB modem as a Textphone

A textphone using V.21 can be emulated using a USB Modem.

Putty does not provide macros to simplify sending modem commands and reacting to inband text.

However, a web page can be used for test.

https://homepages.plus.net/dougrice./dev/DigiSpark/simpleterm/simpletermModem.html

https://homepages.plus.net/dougrice./dev/DigiSpark/simpleterm/simpletermModemOne.html

The Arduino Leonardo USB keyboard could be used with the webpage driving the modem. The Webpage could also be enhanced to AutoType, and respond to text from the service.

Using Web Audio can be used to generate modem signals

It would be possible to dial up an announcement and encode the announcement to send modem tones.

The web pages below document some of this.

http://www.dougrice.plus.com/dev/UART/UART2_V18.htm

https://www.dougrice.co.uk/cgi-bin/wiki.pl?WebAudio

Here is a Textphone sending BAUDOT, Frequency Shift Keying is used to send the 5bit code.

http://www.dougrice.plus.com/dev/UART/UART2_V18.htm has a section "Generate samples tools" that can generate the modem squark used by TextPhones. It can be used to acoustic couple into a telephone and a TextPhone can decode it.


HomePage | RecentChanges | Preferences | My Website home page
This page is read-only | View other revisions | View current revision
Edited September 15, 2025 6:29 am by dougrice.plus.com
Search: