Language+Learning

Language Learning Tool
This study, led by Aydin will use the portable kit to for the implementation and evaluation of a tool for teaching individuals foreign expressions that have been associated with specific sensor activations in the home.

Useful Pages

 * Developing for multiple architectures
 * Setting up your development environment
 * Setting up the application

In Progress

 * Implement sensor install on Motorola phone
 * Incorporate sensor setup with language learning app
 * Let participants review/quiz on recently played phrases
 * Create remote control for prefs and word review

To Do

 * Remove or restructure question tree - 100%
 * Implement smart search - 100%
 * Re-implement the network protocols - 100%
 * Container sensitivity - 100%
 * Worn-out words - 100%
 * Adding associations - 100%
 * Create debug mode where every activation produces a phrase - 100%
 * Restructure UI - 95%
 * Debugging the networking code - 70%
 * Playing sounds - 0%

Future

 * Let participants elaborate sensor labeling (table + breakfast, refrigerator + snack + "What do you want for dinner?")
 * Recruit multilingual n_gineers to record phrases into clearer and more sound files
 * Design a "universal clip" that will all user to pick specific attachment types
 * Decide what to do about items that are not in our original list (implement way to add translations?)
 * Add needed words to list for new sound files
 * Fire Comrade Vlad (be article aware)
 * bathroom || soap dispenser || shower door || can || side table ||
 * canned food || fruit bowl || study || office || sofa ||
 * spoon rest || globe || study chair || office chair || window seat ||
 * dumbbell || desk || cordless phone || sofa || countertop ||
 * furnace || dining table || bench || trash || hamper ||

Design decisions

 * Will map sensors to play at a specified kiosk
 * Phone may flash written translations for spoken words
 * Phone may be used to update labels for objects
 * Must be able get data onto the computer that has an internet connection
 * Check if AudioVox still compiles okay
 * Phone connection will be okay for the data transfer
 * If subject adds an item that's not on the list, it won't play a sound, but they can add a label
 * Will not allow elaboration of object associations
 * Will not initially use RFID tags

Protocol specification

 * **PhraseTrigger** (Main Kiosk)
 * **Port** || **Command** || **Parameters** || **Comments** ||
 * 8002 || MUTE_ON || DURATION (minutes) ||  ||
 * 8002 || MUTE_OFF || none ||  ||
 * 8002 || MUTE_ACK || none || Phone acknowledges mute status change ||
 * 8002 || MUTE_STATUS || none || Request the mute status be sent to the phone ||
 * 8002 || GET_LAST_PHRASES || NUM_PHRASES (to return) ||  ||
 * 8002 || CHATTERBOX_ON ||  || Speak phrases for all sensors ||
 * 8002 || CHATTERBOX_OFF ||  ||   ||


 * **SensorMapEditor** (Main Kiosk)
 * **Port** || **Command** || **Parameters** || **Comments** ||
 * 11552 || ADD_SENSOR || SENSOR_ID | OBJECT_NAME | KIOSK # ||  ||
 * 11552 || DEMOTE_SENSOR || SENSOR_ID | OBJECT_NAME ||  ||

Note: Eventually it'd be nice to have a way of querying the status of various settings on the kiosks. For example: GET_STATUS| status_type | where status_type can be things like "MUTE" or "DEBUG" also: SET_STATUS| status_type | value which the kiosks would reply with: STATUS_CHANGED| status_type | value
 * **Phone**
 * **Port** || **Command** || **Parameters** || **Comments** ||
 * 9020 || LAST_PHRASE_LIST || Comma-delimited list of phrases in English/their translations || e.g. "bed/cama,chair/silla/,..." ||
 * 9020 || SENSOR_SHAKING || SENSOR_ID | ACTIVITY_COUNT ||  ||
 * 9020 || MUTE_OFF || none ||  ||
 * 9020 || MUTE_STATUS || "ON" or "OFF" ||  ||

....again, for another deploy I suppose

Notes on Language Learning Application

 * 1) Announcements in some cases appear to be random. This is likely to be due to building vibrations, and has been addressed by adding a filter to discard firings wherein SensorVal < 5
 * We will review recordings to see if this strategy worked
 * 1) Participant recommends playing English and Italian words 2x with 2s intervals. Patio door ... Patio door ... [POR - to] ... [POR to]
 * Change has been implemented and received well
 * 1) Participant reports that Italian phrases are unclear, and recommends using a teacher voice rather than conversational voice
 * "Sink" -- LA-van-DI-no.
 * 1) Reports total silence Sunday afternoon (data suggest otherwise)
 * 2) Reports "telephone" not correct in Italian
 * 3) Reports "stapler" repeats itself (is really staple vs stapler)
 * 4) Reports "arm chair - arm chair - arm chair" (Italian not said)
 * 5) Reports "stove" is being spoken often (5x for each of 5 sensors)
 * 6) Suggests not putting sensor in room with no speaker (or else add speaker to that room)
 * 7) Mentions "soap dispenser" in front bathroom but forgot what his comment was going to be