This web page is about how you could use two Arduino Leonardos, DigiSparks and Raspberry PICO programmed as USB Keyboards to type during long duration Text Relay test calls.
The deaf community use a textphone to TYPE and READ and a relay operator SPEAK and HEAR.
Users can now use a smartphone App to TYPE and READ.
Some users use a combination of TYPE and READ and SPEAK and HEAR.
The Relay Advisors use a Workstation and READ off an LCD screen and TYPE using a keyboard and mouse
For the TextRelay, the TYPE and READ user and Advisor use a keyboard to communicate, hopefully in turn.
Long duration calls would normally have typing back and forth between the users via the Relay Advisor.
How does your simple automation tool read a smartphone screen or press buttons on the on screen keyboard?
What is available?
The Arduino Leonardo can be programmed as a USB Keyboard.
https://docs.arduino.cc/language-reference/en/functions/usb/Keyboard/
The Arduino Leonardo can also be programmed as a USB Keyboard and mouse, and it has digital inputs and outputs and serial ports.
They have inputs and outputs which can be used to enable and co-ordinate when they are typing.
Use one for the User and another one for the Advisor.
An Arduino Leonardo, Raspberry PICO, and DigiSpark can be programmed to press keys on a keyboard
You need to use the mouse to select the input box that you type into, then enable typing.
It is possible to use an external keyboard and OTG cable plugged into a smart phone.
Automating this is possible
Automating these is near impossible
Automation could be simplified if an agreed sequence of keystrokes triggered test events.
Below is a collection of Atmel 32u4 bases Arduino Leonardo and DigiSparks.
A Raspberry Pico can also be used.
A textphone using V.21 can be emulated using a USB Modem.
Putty does not provide macros to simplify sending modem commands and reacting to inband text.
However, a web page can be used for test.
https://homepages.plus.net/dougrice./dev/DigiSpark/simpleterm/simpletermModem.html
https://homepages.plus.net/dougrice./dev/DigiSpark/simpleterm/simpletermModemOne.html
The Arduino Leonardo USB keyboard could be used with the webpage driving the modem. The Webpage could also be enhanced to AutoType, and respond to text from the service.
It would be possible to dial up an announcement and encode the announcement to send modem tones.
The web pages below document some of this.
http://www.dougrice.plus.com/dev/UART/UART2_V18.htm
https://www.dougrice.co.uk/cgi-bin/wiki.pl?WebAudio
Here is a Textphone sending BAUDOT, Frequency Shift Keying is used to send the 5bit code.
http://www.dougrice.plus.com/dev/UART/UART2_V18.htm has a section "Generate samples tools" that can generate the modem squark used by TextPhones. It can be used to acoustic couple into a telephone and a TextPhone can decode it.
I brought some DigiSparks off eBay.
They could be programmed using the Arduino IDE and came with examples.
An OTG cable can be used to plug an external keyboard into a smart phone.
Here are a collection of DigiSparks, Arduino Leonardo and other boards that use the Atmel 32u4 chips that support USB.
These pages write up some notes and ideas.
AutoType - Automating pressing keystrokes using DigiSparks , Arduino Leonardo and Raspberry Pico
DigiSparks - DigiSpark - https://www.instructables.com/Digispark-DIY-The-smallest-USB-Arduino/
The Arduino documentation had an example that sent keyboard keystrokes.
I brought some DigiSparks off eBay for 1.69 GPB - possibly for a model railway project.
One example was:
Caller A dials B and has a conversation using TYPE and READ.
Caller A decides to call B
Caller A picks up phone and hears Dial tone
They dial the number and hear ring ring
B heads the ringer and pick up the phone
A hears the ringing stop and B says hello
A says hello and they talk back and forth.
For the TextRelay, the TYPE and READ user and Advisor use a keyboard to communicate, hopefully in turn.
The script could be coded into a computer and send commands to the dialler and text to speech units.
It would be possible to write a Script and run this on a computer that sent commands to Arduino or Raspberry PICO programmed as Keyboards and the mouse.
To prove the point the Arduino Keyboard example is a good place to start:-
https://docs.arduino.cc/language-reference/en/functions/usb/Keyboard/
At its simplest, a switch to turn the typing on and off and a loop of typing was required. This was acheivable.
The more complicated senario are awaiting developement.
For the TextRelay, the TYPE and READ user and Advisor use a keyboard to communicate, hopefully in turn.
The automated keyboards cannot read the screens.
Using an Arduino or Raspberry Pico as a Keyboard and a bit of wire to synchronise the two, some automation is possible.
Soak testing of a set up call was possible and this could be left to load up the scroll buffers on the App and Advisor workstations.
The conversation is between two humans, so their needs limit the maximum duration required.
http://www.dougrice.plus.com/dev/DigiSpark/indexImages.htm
Below is a picture of various Atmel 32u4 based Arduino Leonardo variations AutoTyping into the App.
Possibly using this sketch: http://www.dougrice.plus.com/dev/DigiSpark/KeyboardRP2/KeyboardRP2.ino
