Language Learning Tool



This study, led by Aydin will use the portable kit to for the implementation and evaluation of a tool for teaching individuals foreign expressions that have been associated with specific sensor activations in the home.

Useful Pages


In Progress


  • Implement sensor install on Motorola phone
  • Incorporate sensor setup with language learning app
  • Let participants review/quiz on recently played phrases
  • Create remote control for prefs and word review

To Do


  • Remove or restructure question tree - 100%
  • Implement smart search - 100%
  • Re-implement the network protocols - 100%
  • Container sensitivity - 100%
  • Worn-out words - 100%
  • Adding associations - 100%
  • Create debug mode where every activation produces a phrase - 100%
  • Restructure UI - 95%
  • Debugging the networking code - 70%
  • Playing sounds - 0%

Future


  • Let participants elaborate sensor labeling (table + breakfast, refrigerator + snack + "What do you want for dinner?")
  • Recruit multilingual n_gineers to record phrases into clearer and more sound files
  • Design a "universal clip" that will all user to pick specific attachment types
  • Decide what to do about items that are not in our original list (implement way to add translations?)
  • Add needed words to list for new sound files
  • Fire Comrade Vlad (be article aware)
bathroom
soap dispenser
shower door
can
side table
canned food
fruit bowl
study
office
sofa
spoon rest
globe
study chair
office chair
window seat
dumbbell
desk
cordless phone
sofa
countertop
furnace
dining table
bench
trash
hamper

Design decisions


  • Will map sensors to play at a specified kiosk
  • Phone may flash written translations for spoken words
  • Phone may be used to update labels for objects
  • Must be able get data onto the computer that has an internet connection
  • Check if AudioVox still compiles okay
  • Phone connection will be okay for the data transfer
  • If subject adds an item that's not on the list, it won't play a sound, but they can add a label
  • Will not allow elaboration of object associations
  • Will not initially use RFID tags

Protocol specification

  • PhraseTrigger (Main Kiosk)
Port
Command
Parameters
Comments
8002
MUTE_ON
DURATION (minutes)

8002
MUTE_OFF
none

8002
MUTE_ACK
none
Phone acknowledges mute status change
8002
MUTE_STATUS
none
Request the mute status be sent to the phone
8002
GET_LAST_PHRASES
NUM_PHRASES (to return)

8002
CHATTERBOX_ON

Speak phrases for all sensors
8002
CHATTERBOX_OFF



  • SensorMapEditor (Main Kiosk)
Port
Command
Parameters
Comments
11552
ADD_SENSOR
SENSOR_ID | OBJECT_NAME | KIOSK #

11552
DEMOTE_SENSOR
SENSOR_ID | OBJECT_NAME


  • Phone
Port
Command
Parameters
Comments
9020
LAST_PHRASE_LIST
Comma-delimited list of phrases in English/their translations
e.g. "bed/cama,chair/silla/,..."
9020
SENSOR_SHAKING
SENSOR_ID | ACTIVITY_COUNT

9020
MUTE_OFF
none

9020
MUTE_STATUS
"ON" or "OFF"

Note: Eventually it'd be nice to have a way of querying the status of various settings on the
kiosks. For example:
GET_STATUS| status_type |
where status_type can be things like "MUTE" or "DEBUG"
also:
SET_STATUS| status_type | value
which the kiosks would reply with:
STATUS_CHANGED| status_type | value

....again, for another deploy I suppose

Notes on Language Learning Application


  1. Announcements in some cases appear to be random. This is likely to be due to building vibrations, and has been addressed by adding a filter to discard firings wherein SensorVal < 5
    • We will review recordings to see if this strategy worked
  2. Participant recommends playing English and Italian words 2x with 2s intervals. Patio door ... Patio door ... [POR - to] ... [POR to]
    • Change has been implemented and received well
  3. Participant reports that Italian phrases are unclear, and recommends using a teacher voice rather than conversational voice
    • "Sink" -- LA-van-DI-no.
  4. Reports total silence Sunday afternoon (data suggest otherwise)
  5. Reports "telephone" not correct in Italian
  6. Reports "stapler" repeats itself (is really staple vs stapler)
  7. Reports "arm chair - arm chair - arm chair" (Italian not said)
  8. Reports "stove" is being spoken often (5x for each of 5 sensors)
  9. Suggests not putting sensor in room with no speaker (or else add speaker to that room)
  10. Mentions "soap dispenser" in front bathroom but forgot what his comment was going to be