Visualizer


 * BoxLab Visualizer**

The BoxLab Visualizer can be installed on the Microsoft Windows XP (Service Pack 2) operating system or later (tested on Windows Vista, Windows 7). At least 1GB of system RAM and a Pentium-4 or newer processor is recommended. Currently installation on Macintosh, Linux, or other platforms is not supported.
 * System requirements**

A sample version of the visualization/annotation application is available here: [|BoxLabVisualizerInstaller] Version 1.1.3 (500KB)
 * Download**



A sample dataset is available here. Download and unzip to a desired location on your local hard drive. You will later use the BoxLab Visualizer to open the dataset from this location. You may delete the zip archive after its contents have been extracted to the desired location. [|JulyDemoData100701.zip] (1.7GB)
 * Sample Data**

The BoxLab Visualizer is a tool for viewing and annotating BoxLab datasets. For more information, please visit **boxlab.wikispaces.com**.
 * I. About**

To start the Visualizer, launch the application from the Windows Start Menu under a folder called MIT.
 * II. Getting Started**

After starting the application, you will be prompted for a dataset to view.
 * III. Loading Data**




 * Click on the folder icon** to bring up a folder browser dialog. **Choose** the folder corresponding to the data you wish to view.
 * If you want to view **multiple days** of data, choose the folder that is **above the Day Folder(s)** that you wish to view. (See example below.)
 * If you want to view **one day** of data, choose that **Day Folder**.
 * If you want to view **one hour** of data, choose that **Hour Folder** in the Day Folder it corresponds to. (Note: This will load the entire day.)




 * Click** "OK" once you have selected the proper folder. The directory should appear in the text box.




 * Click** on "Start Show" to begin viewing the data.


 * IV. Visualizer Components**

a. __Data Source Selection__

Begin by **selecting the day** you wish to view from the panel in the top left.



The **hours** of data associated with that day appear under it. The numbers on the right of each line indicate the hour in a 24-hour format. If a number is missing, there is no data folder for that hour. You may select a "**data colorizer**" that will be used to reflect sensor activations or amount of activity present in each hour of data. Choose a colorizer with the buttons below the playhead control. Some of the colorizer options are illustrated below:


 * Colorizer || **Motion** || **Annotation** || **Object MITes** || **Motion MITes** ||
 * Description || The amount of activity recorded by the video sources as inferred from the variable frame rate recording software. || The presence of activity labels for a given dataset is conveyed through the use of orange markers that indicate the start time for an activity annotation. || Object movement (acceleration) sensor activation detected. || Passive infrared sensor activation ||
 * Coloring || **White** indicates no activity.
 * Yellow** indicates little activity.
 * Red or purple** indicates lots of activity. || Hours with a substantial number of **orange** lines suggest a lot of varied activities occurred during that time. || **Purple** indicates sensor activation || Red indicated movement in proximity to a PIR motion sensor ||
 * Notes || The method of data collection may cause this to appear almost completely orange even when there is not continuous activity. ||  ||   ||   ||
 * Image || [[image:Step_5A.gif width="204" height="439"]] || Annotation image goes here || [[image:Step_5B.gif width="206" height="430"]] || Object MITes image goes here ||


 * Click on the hour** you wish to view to bring up the video.

b. __Video Screen and Selector__


 * Selecting an hour** will bring up all the corresponding **video views**. (Any unused space for views will be blue).




 * Double-clicking** on a view will initiate a **single-view layout**.



You may **change the** **view layout** with the buttons in the top right corner of the video screen.



As currently implemented, when switching from a small number of video displays to a larger number, you must reassign the lost video streams to their former places (as described below). A strategy for maintaining the placement of video streams will be implemented in a future revision.


 * Clicking** on the video screen will **toggle pause and play**, depending on your currently selected play speed and direction (see the "Playhead Control" section for more information). **Right-clicking** will bring up the **annotation menu** (see the "Annotation" section for more information).

At the bottom of the Visualizer are the available **video and audio streams for the selected hour**. Only **one** audio stream may be selected at a time, and currently it will **only play** when the video is playing **forward at normal speed**.

To **change the placement** of individual video views, **hover** your mouse pointer over the view you want to move. An **overlay** of the current view layout will appear.




 * Click** **on the space** you wish to move the view to. You can also **change the view layout** by **right-clicking** on an individual video view. To apply a new view layout, click on a space in the new layout to place the current video view. (Note: Occasionally the overlay will stick to the video view after you move your mouse away. This is a known bug.)

c. __Playhead Control__

The three buttons on the left are used to **fast-backward**, **play**, and **fast-forward** the video, respectively. **Right-clicking** on the **play** button will **toggle its direction** between play forward and play backward.



The **timeline slider**, along with the Time display, indicates the **current time** of the video view(s).


 * Clicking** on a button will give it **color** (green for forward play, orange for backward play) and change its symbol to "pause". **Click** on it again to **pause** the video.



The **small numbers** on the left and right indicate the **speed** of fast-backward and fast-forward. They can be **changed** individually by **right-clicking** on the **number** and selecting a speed. **Clicking** on the number will **change its speed** to the **next** in the list.




 * Drag the small circle** along the slider to **change the current time**, or **click** on the slider to make the circle jump to a time. The selected colorizer scheme will appear in the background of the slider. **Right-click** on the slider to **magnify** a section of the timeline (in the large circle below) for more fine-grained time selection.



The **volume slider** can be used to adjust the volume of the audio.

d. __Annotation Management__

Annotation management is on the right side of the Visualizer.




 * **Right-clicking** on an annotation will bring up an **annotation menu** (see the "Annotation" section for more information).
 * **Double-clicking** on an annotation tab will **jump the video time to the beginning of the annotation**, if that start time is within the current hour. If it’s not within the current hour, it will change the current video time to the start or end of the current hour, depending on which is closer to the start time of the annotation. Subsequent **double-clicking** on an annotation tab will **jump the video time between the beginning and end** of that annotation. The **current time** is highlighted in **white**
 * **[CURRENTLY DISABLED] Pressing CTRL + Z** (undo) from anywhere in the application will cause the last change made to an annotation to be reversed.


 * Click on the small yellow rectangle** on the right side of any annotation to **toggle between expanded and collapsed**. Holding the **CTRL key** and expanding or collapsing an annotation will cause all annotations to assume that state. **[CURRENTLY BROKEN]**

The expanded version lists the certainty ratings of the annotation next to the times they begin.



The annotations shown in the topmost portion of the annotator interface are those including at least one time stamp with a stop time (i.e. "Not happening"). They are listed under the time they begin, in chronological order. Only the current annotations with a stop time in the current hour will appear in the list. For annotations to appear in the list, they must belong to a currently selected annotation set.

The color of the annotation is determined by the Annotation Set selection interface. **Right-clicking** on one of the annotation sets will allow you to change the color used for the annotations associated with it. During editing and playback, the annotations will reflect their state using variations on their base color:
 * A **lighter shade** of the annotation color indicates that the label is currently happening.
 * A **darker shade** of the annotation color indicates that the label is **not** currently happening.
 * A **red** color indicates that the current time of the video currently at the start time of the annotation.
 * A **gray** color indicates that the annotation has a logical error (such as multiple "Not happening" or two of the same rating in a row).




 * Partial annotations** are those without a stop time. They are listed in chronological order based on their start time. Only the partial annotations from the current hour or previous hours will appear in the list.
 * A **red** color indicates that the current time of the video is the start time of the annotation.
 * A **light orange** color (see the first annotation below) indicates that the annotation is currently happening (its start time is before the current time of the video).
 * A **dark orange** color (see the last two below) indicates that the annotation is not currently happening (its start time is after the current time of the video).



e. __Keyboard Shortcuts__


 * Step Forward/Backward a Frame || RIGHT/LEFT ARROWS ||
 * Set Play Forward || UP ARROW ||
 * Set Play Backward || DOWN ARROW ||
 * Play/Pause/Resume || SPACEBAR ||
 * Fast Forward/Pause || CTRL + RIGHT ARROW ||
 * Fast Backward/Pause || CTRL + LEFT ARROW ||
 * Adjust Fast Forward Speed || SHIFT + RIGHT ARROW ||
 * Adjust Fast Backward Speed || SHIFT + LEFT ARROW ||
 * Volume Up/Down || +/- ||


 * V.** **Annotation**

a. __Creating an Annotation__


 * Pause** the video at the time you wish to rate an activity or state. **Right-click** on the **video screen** to bring up the annotation menu. (Note: Right-clicking while the video is playing will automatically pause the video.)
 * **Partial annotations** are those under your "Partial Annotations" list on the right side of the Visualizer.
 * **Recent annotations** are those you have used selected recently, excluding those on the "Partial Annotations" list.
 * **New annotation** is where you can create a new annotation.

To make a new annotation, **navigate** the middle or last section of the menu until you find the activity or state you wish to add a certainty rating to. (Activities or states that are currently listed under "Partial Annotations" are grayed out.) **Highlight** it and select the desired value.



The new annotation should appear under the "Partial Annotations" list on the right.

b. __Adding a Certainty Rating to an Annotation__


 * Pause** the video at the time you wish to rate an annotation. **Right-click** on the **video screen** to bring up the annotation menu.


 * Choose** an annotation from the first section (//Partial annotations//). **Highlight** it and select the desired value. (Note: Adding "Not happening" will move a partial annotation to the "Current Annotations" list.)



You may also add a rating by **right-clicking** on the annotation, choosing "**Add Rating**" and selecting the desired value.



To add a rating to a current annotation, **right-click** on the annotation, choose "**Add Rating**" and select the desired value.



c. __Changing or Removing a Certainty Rating__

To **change** a certainty rating, **right-click** on the annotation that contains the rating you want to change and choose "**Change Rating**". **Select** the rating and the replacement value.



To **remove** a certainty rating, **right-click** on the annotation that contains the rating you want to remove and choose "**Remove Rating**". **Select** the rating you want to remove. (Note: If you remove all "Not happening" ratings, the annotation will be moved to the "Partial Annotations" list.)



d. __Changing or Deleting an Annotation__

To **change** an annotation’s label, **right-click** on the annotation that you want to change and choose "**Change activity label**". **Select** the new label.



To **delete** an annotation, **right-click** on the annotation that you want to delete and choose "**Delete annotation**".



e. __Saving Annotations__

The annotation data files will be saved when you **make a change to an annotation** or **close** the Visualizer.


 * VI. VideoMap Data Visualization**

The **VideoMap** data view provides a time-locked visualization of sensor activations as they occur. As the playhead is moved, markings on the VideoMap change state to indicate where in the environment a sensor activation has occurred. Sensors are illustrated as simple shapes overlaid onto a floor plan on the environment where the data was recorded.



An **Ekahau Position** sensor is represented with a **fuzzy semi-transparent colored circle** where the system estimates the tag to be located at the present time.

The **Object Movement MITes** are represented on the map as small blue circles. Each time an ObjectMITes records an activation, the circle briefly flashes yellow. Passive infrared **Motion MITes** are shown as large **gray semi-transparent circles** that flash yellow when they detect motion within the approximate range represented on the map.


 * Double-clicking** on one of the sensor icons will cause the visualizer playhead to move to the **next activation** time for that sensor, assuming it occurs within the current hour.




 * Right-clicking** on the sensor icon will generate a pop-up timeline for the current day, with **red tick marks** whenever the sensor has been activated. Clicking on a region in the timeline will cause the visualizer to open the hour of data represented by that region.


 * VII. Time Series Data Visualization**

In the area below the video display port, a zoomable time series visualization is used to present scaled values over time.


 * VIII. Bug Reports**

Please report bugs to **mit.boxlab@gmail.com**.


 * IX. Credits**

The BoxLab Visualizer was created by the House_n consortium at MIT.


 * X. Acknowledgments**

Work on BoxLab and generation of shared sensor datasets using BoxLab is supported by National Science Foundation grant #0708375 (CRI: CRD Development of Longitudinal Home Activity Datasets as a Shared Resource). Equipment used in this work was donated by Intel Corporation, Microsoft Research, and Mitsubishi Electric Research Laboratory (MERL).