From Computational Memory Lab
Revision as of 16:34, 12 August 2011 by Pcrutchley (Talk)

Jump to: navigation, search

/!\ Note: this page is a work in progress.

Guide to Annotation with TotalRecall


TotalRecall is a program for marking word onsets when scoring free recall and recognition data. The program displays the wordpool below the speech waveform to facilitate this process. Below are instructions for how to properly annotate data. When annotating, it is crucial to remain consistent; therefore, refer to these instructions whenever you are unsure of a word onset.


The most recent version of the program can be found through our TotalRecall page. Download the `.dmg` file (if you are working on a Mac) and drag the app file into your Applications folder. (If you are not an Administrator on your machine, ask an RA to authenticate this action.)


In normal operation, the three file types associated with audio annotation are as follows.

! Kahana Lab members: A script called will download and organize the following types of files for each session you annotate.  See below for its operation.

Wordpool file

This must be a `.txt` file containing all possible normal vocal responses from the subject. All words found in the `.lst` files (see below) must be contained in this wordpool. Usually, this is all words that may be presented to any subject for a given experiment.

! Kahana Lab: `` puts the wordpool for each subject in a given experiment in that subject's folder for said experiment; e.g., in `parse_files/ltpFR/LTP046/wordpool.txt`

.wav files

These are the audio files that are fed into the annotation program.

! Kahana Lab: For our current set of experiments, each audio file consists of a subject's vocal responses for one "trial", i.e., one list of words that they are recalling/recognizing. (An exception is `ffr.wav`, which is not present in all sessions. In this file, the subject is recalling every word presented to him/her throughout the session.)

.lst files

A `.lst` file displays the words that the participant viewed during the particular trial that you are scoring. There is a .lst file for every trial, every session. To be read correctly by TotalRecall, `.lst` files must have the same time as their corresponding `.wav` file; e.g., `15.lst` and `15.wav` would correspond to the same trial within a session. The `.wav` and `.lst` files must be in the same directory.

When you open an audio file in TotalRecall, the software highlights the words that are contained in the corresponding `.lst` file in bold in the "Wordpool" window.

.ann files

These are the output files of the annotation program. They display the words recalled with the respective times of recall (in milliseconds) within the recall period. .ann files can be edited like text documents, but it is best to avoid changing any of the text in a `.ann` file.

/!\ If a participant does not make any recalls, vocalizations, or intrusions during a recall period, ie. the annotator did not make any entries in the entire period, a `.ann` file will not be created for that trial. If this occurs, the annotator will need to create a blank .ann file manually. The best way to do this is from the terminal, in the session folder, to type `touch 10.ann` to create the blank `.ann` file for whichever empty `.wav` file exists in that session. This is necessary in order to run several post-processing analyses scripts.

.tmp files

Before an annotated or partially-annotated file is marked complete, it exists as a `.tmp` file, with the same name as the `.wav` file

{i} If onset-detection or word-recognition scripts are used (not described here for sake of generality/brevity), they should generate a `.tmp` file which is automatically loaded with the corresponding audio file.


Starting an annotation session

There are several ways to load a multi-trial session into TotalRecall to begin annotation. This is one example.

{i} The examples that follow assume Mac keybindings. That is, they make reference to the Command key, ⌘. Use the modifier key appropriate to your operating system, e.g., Ctrl on Windows.

1. Open the application TotalRecall

  • If you are using a Mac and followed the procedure in "Installation" above, it will be in your Applications folder.

2. Open a session folder with the command "Open Audio Folder..." in the "File" menu.

  • You can also use the keyboard shortcut Option + Command + O, or ⌥⌘O.
  • If you only have one audio file to annotate, you may also use the "Open Audio File..." command from the File menu, or use ⌘O.

3. Open the corresponding wordpool, from the File menu (or using Shift + Command + O, ⇧⌘O).

You are now ready to start annotating!

Marking Onsets

Select an audio file from the "Audio Files" list by double-clicking on it. You will see the waveform appear in the top half of the window, and it is now available for playback.

You must now find and mark the onset of each vocal response. The space bar starts and stops playback. When you get to a response, stop playback and navigate to the beginning of the word using the arrow keys.

The default settings are as follows:
  • arrow keys alone, ← and →, will move the cursor 5 milliseconds
  • command + arrow, ⌘← and ⌘→, will move the cursor 50 msec
  • shift + command + arrow, ⇧⌘← and ⇧⌘→, will move the cursor 500 msec
These may be modified in the Preferences window.

Command + Z (⌘Z) is an extremely useful command. It will replay the 200 msec prior to the cursor position. When you move the cursor to the beginning of the word, press ⌘Z to check the position. Listen carefully: you should just barely hear the beginning of the word. What this sounds like exactly will depend on the first phoneme of the word in question (i.e., consonants will sound different from vowels, etc.). The cursor is in the correct position when ⌘Z yields a very brief voice sound, but when you press ← then ⌘Z you cannot hear any part of the word.

Once the cursor is in the correct position, type the vocal response. If it is in the wordpool, you may begin typing, then press "tab" to complete. Note that tab will complete with the top response in the list below the text field.

Marking Intrusions

Intrusions (responses that are not in the entire wordpool) are rare. If you think that someone made an intrusion, first check the .lst file and then look through the wordpool to see if there are any responses that sound really similar to what you heard. If not, you can score the word as an intrusion by typing the response and then pressing Command + Shift + Enter (⌘⇧⏎).

Marking Vocalizations

Sometimes people will mumble to themselves or say things like "No...wait...wrong word" or "umm..." These should not be scored as intrusions, but as vocalizations. Simply press Command + Shift + Enter (⌘⇧⏎) without typing anything at the start of the string of speech, just as if you were scoring an intrusion. (Instead of a word appearing by the annotation mark, you will see `"<>"`.)

Do not ignore vocalizations because EEG data during vocalization periods should not be examined. For vocalizations lasting longer than 1 second, score a new ⌘⇧⏎ at every full second of the vocalization. (If default settings are used, pressing ⇧⌘→ twice will move the cursor forward one second.)

Finishing a Session

When all responses, intrusions, and vocalizations have been marked, and the cursor has moved tot he very end of the file, you may mark the session as complete. Simply press the space bar to stop playback, then click the "Mark Complete" button. (You may also press Command + Shift + D, ⌘⇧D.)

Adding files to SVN repository

! Kahana lab annotators will have to add completed .ann files to our parse_files repository. Adding these two aliases to your `.bash_profile` or `.bashrc` on separate new lines will make this much easier:

  • `alias fradd='svn add {[0-9]*.ann,ffr.ann}; svn commit -m "[zzz] added ${PWD##*parse_files/} fr"'`
  • `alias radd='svn add r*.ann; svn commit -m "[zzz] added ${PWD##*parse_files/} rec"'`

You should change "`[zzz]`" to your initials. Make sure to source your `bash_profile` or `bashrc` after editing it. Ask a more senior lab member if you have questions about editing your `bash_profile`/`bashrc`.

Annotation commands

The following are commands central to using TotalRecall. Note that time intervals are default values, and may be changed by the user.

  • Space: Start and stop playback. (Must also be pressed when the cursor reaches the end of the audio file before marking it as complete.)
  • Enter: Commit text as normal annotation
  • Command + Shift + Enter, ⌘⇧⏎: Commit text as intrusion or mark vocalization
  • Arrow keys, ← and →: Move the cursor 5 milliseconds back and forward, respectively
  • Command + arrow, ⌘← and ⌘→: Move the cursor 50 msec back and forward, respectively
  • Shift + Command + arrow, ⇧⌘← and ⇧⌘→: Move the cursor 500 msec back and forward, respectively
  • Shift + arrow, ⇧← and ⇧→: Move the cursor between existing annotation marks
  • Command + Backspace or Command + Delete: Delete selected annotation (use ⇧← and ⇧→ to navigate and select)
  • Command + L, ⌘L: Undo play, i.e., go to cursor position before playback

All other key bindings may be found in the program drop-down menus, and under "Help" ➔ "Key Bindings"

Annotation / TotalRecall FAQ

Have a question? Email `memory - software @ psych . upenn . edu` (remove all the spaces).