![]() |
Computational Memory Lab | None | ||||||||||||||||||||||||||||||||||||||||||||||
Experiment Programming Library for LinuxProgrammed by: Josh Jacobs(based on work by Daniil Utin & Roger Khazan)Table of Contents
1. INTRODUCTIONThis document provides an overview of the development kit for creating computerized experiments. Every experiment consists of two stages: examination and scoring. In a computerized version both appear as two separate modules. The examination module consists of audio/visual tests given to a subject. Responses are stored as raw text (for example, digitized voice). The scoring part assists in interpretation of this data (e.g., correctness, inter-response time, etc.). The results are then recorded in a cumulative data file, according to a pre-specified format suitable to further analysis (e.g. statistical, graphical, etc.).Designing experiments that include video and audio components can be simplified with the use of high level routines developed for this purpose. The knowledge of C and at least a very light exposure to C++ are necessary. See Appendices I and II to learn how to set up an experiment. Development kit consists of several C++ development libraries:
2. LIBRARY FUNCTIONS2.1 InitializationInitialization library allows runtime initialization of variable values which are obtained from experiment's config.ini file. Here are a few guidelines on how to use this library: Before programming an experiment edit a config.h file containing all variables to be initialized at runtime. The advantage of initializing certain variables at runtime is that an experiment doesn't need to be recompiled every time values of these variables change. Runtime variables also increase an experiment's flexibility and make it easier and faster to use. A good example of runtime variable is the number of trials in the experiment. After config.h is finished, run the makeini utility; it will take care of generating the config.ini file and all necessary C++ code to parse that file at runtime. Don't forget to declare runtime variables (along with the rest of the variables) in the main experiment file by including the extern.h file produced by makeini, or the experiment won't compile. Use function load_config() in the main experiment to invoke runtime initialization. It should return OK in case of success. if (load_config()!=OK) return(-1);
Word Pool library provides means of operating with collections of words (e.g., a set of 18 groups of words called blocks, each consisting of four words). The set (pool) can be loaded, shuffled, searched and saved. It is possible to operate on several independent words at the same time. At the beginning of the program an instance of the word pool should be declared in the following way: POOL mypool;. The pool library consists of the following functions:
Usage: mypool.get_pool(filename, number_of_blocks, size); [HEADER: long get_pool(char *fname, long nblocks = 0, long size=1);] Creates a pool containing words from a specified word file with specified number of blocks of the specified size. For example, the function call mypool.get_pool("wordpool.txt", 18, 4) would get 72 words from the file "wordpool.txt" as 18 groups of 4 words. The default size of the block is 1, so if it is not specified like in mypool.get_pool("wordpool.txt", 72) a linear word pool is created, i.e. each block consists of only one word. If the second argument (number of blocks) is 0, the entire file is used. In order to get a uniform sampling of words in a word pool it is advisable to load an entire pool, shuffle it (see below), and then work with the desired number of words. This way words from the entire spectrum of the dictionary are obtained. Usage #1: mypool.get_str (block_number, item_number); [HEADER #1: char *get_str(long block, long item);] Usage #2: mypool.get_str (item_number);
Returns pointer to a specified word in a word pool. block gives a block number a word should be taken from while item specifies a zero base offset form the firs word of that block. For example: char *word; word = list.get_str(9, 2); set word pointer to point to the third word in tenth block of a list
pool. It is also possible to use this function in linear mode, i.e.
word = list.get_str(0, 38) will yield the same result assuming that one
block consists of four words.
Usage: mypool.shuffle_items(number_of_items, starting_item); [HEADER: void shuffle_items(long nitems, long from=0);] The usual thing to do after the pool is loaded is to shuffle all the words so the random grouping is created. The function shuffle_items is designed for this purpose, as in mypool.shuffle_items(72), which shuffles the first (since from is equal to 0) 72 words. When the program is executed, this function will shuffle the items the same way each time. To generate a random shuffle, you must call rnd.set_seed(nRandom) and pass it a randomly generated number. Usage: mypool.shuffle(number_of_blocks, starting_block); [HEADER: void shuffle(long nblocks, long from_block=0);] This function shuffles entire blocks of the pool preserving individual block structure. That is only blocks, not words within them are shuffled. The function call mypool.shuffle(18) shuffles the first 18 blocks of the pool. As in shuffle_item function, there is an optional second argument that specifies from which block to start shuffling. In general, this routine is convenient for shuffling the study list between the successive presentations. When the program is executed, this function will shuffle the items the same way each time. To generate a random shuffle, you must call rnd.set_seed(nRandom) and pass it a randomly generated number. Usage: mypool.save_list ("list.txt", &pool, starting_block, number_of_blocks); [HEADER: int save_list(char *fname, POOL *pool, long from_block, long nblocks);] Sometimes it is convenient to save a (shuffled) list presented to a subject in order to have it available for the scoring part. The function save_list makes this possible. It saves specified part of pool in a file fname as another pool.. For example, save_list("list.txt", &pool, 0, 18) saves the first 18 blocks of pool in "list.txt". The advantage of saving them in the pool format is that they can be easily retrieved and operated on as a pool in the scoring part of the experiment, e.g.: POOL list; list.get_pool(list.txt", 18, 4); Usage: mypool.length(); [HEADER: long length();] This function returns the number of blocks in the pool. Continuing our previous example, mypool.length() returns 18. Usage: mypool.search("word_being_searched_for", starting_point); [HEADER: long search(char *str, long from=0);] This functions can be used to search for a word in a pool. For example,
i = pool.search("table", 7) would search for the word "table" starting
from the 8th word in the pool. The returned value specifies
index (zero based) of the word table in the pool. Finding which block and
item i is can be done in a following way: block = i / blocksize;
item = item % blocksize. search returns -1 if word is not present
in the pool.
index = searchexact(thestring); // Attempt a perfect match first if(index==-1) index=search(thestring); // attempt a partial match if(index==-1) cerr << "no match"; // the word just ain't in there Usage: mypool.searchexact("word_being_searched_for", starting_point); [HEADER: long searchexact(char *str, long from=0);] Like search (see above for description), this functions can be
used to search for a word in a pool. For example, i = pool.search("table",
7) would search for the word "table" starting from the 8th
word in the pool. The returned value specifies index (zero based) of the
word table in the pool. Finding which block and item i is can be
done in a following way: block = i / blocksize; item = item % blocksize.
search returns -1 if word is not present in the pool.
2.3 SB16 Audio The following environment variables affect EPL sound output:
This library consists of a number of high level functions which allow recording and playback of digital sound through Sound Blaster 16 sound card. To start, an object of class SB16 should be declared (SB16 audio;). There are two functions that allow recording of digital sound. The first one:
SB16 audio; // declare an instance of class SB16 TIMER timer; // declare an instance of class TIMER char fname[] = "myvoice.voc"; ... // start the timer timer.start(); // start recording in a file specified by fname audio.record_voice(fname); // wait while timer has not reached X milliseconds while (timer.get() < X); // stop recording upon exiting from the while loop audio.stop_voice();The function record_voice should be used to record voice continuously for a certain period of time (e.g. free recall). Usage: record_voice_start("my_voice.voc", number_of_milliseconds_to_record) [HEADER: long record_voice_start(char *fname, long length)] or Usage: record_voice_start("my_voice.voc", number_of_milliseconds_to_record, recording_beginning_function) [HEADER: long record_voice_start(char *fname, long length, void (*)())] or Usage: record_voice_start("my_voice.voc", number_of_milliseconds_to_record, recording_threshold) [HEADER: long record_voice_start(char *fname, long length, short)] or Usage: record_voice_start("my_voice.voc", number_of_milliseconds_to_record, recording_beginning_function, recording_threshold) [HEADER: long record_voice_start(char *fname, long length, void (*)(), short)] This function records sound into a voice file for a specified number of milliseconds. This functions starts saving sound from microphone only after subject begins speaking. It will just wait until subject produces a sound that exceeds the given recording level threshold. The sound level threshold can be set by using a version of this function that lets you specify it (see the declarations above). The default value (200) is used if the function is called without specifying it. The sound level threshold should be set by the programmer to customize the epl to the specific environment that the experiment is being done in (ie. the characteristics of the sound card and microphone, and mixer settings). Valid values for the sound threshold range from 0 - (2^16-1). The programmer would probably want to make this value a run-time modifyable variable via the config.ini file so that it is not necessary to recompile the experiment to change this threshold. This function also can be passed a function that it will call when recording begins. The function that the function is passed in should be a function that returns a void and takes no parameters. This can be used, for instance, to have the program flash "Recording" on the screen when the recording actually begins. In order to use this function, the programmer will probably have to pass in a wrapper function which encapsulates the functionality that is wanted inside of a function that returns void and takes no parameters. Since the amount of recording time is specified as a second argument, no function to stop recording is necessary. A typical example of using this function is presented below: SB16 audio; // declare an instance of class SB16 char fname[] = "myvoice.voc"; ... // start recording in a file specified by fname for 2 seconds audio.record_voice_start(fname, 2000);This function is useful for recording short responses, one at a time. It is appropriate for measuring the correctness of the responses, as opposed to something like interresponse time. This function returns the length of time (in milliseconds) that the
program had to wait while there was silence before it began recording sound.
Usage: audio.play_voice("my_voice.voc"); [HEADER: int play_voice(char *fname)] This function plays the specified voice file.
Usage: beep(freq,msec); [HEADER: void beep(int freq, int msec)] Where freq is the desired frequency in Hz, and msec is an amount of time in milliseconds. Function beep produces a freq Hz tone (beep) of msec millisecond duration.
2.4 Video This library contains a collection of functions which allow displaying text in high resolution video mode. At the beginning an object of class Svga256 should be instantiated: Svga256 video() and then video mode should be initialized by calling initialize_video(). Default width and height can be changed by specifying different values in the first and second arguments to Svga256 instantiation: Svga256 video(3,4) will create set up 3x4 screen. Note: Try using default 2x2 width and height. It will make the program less confusing (especially for others).
Usage: video.initialize_video(); [HEADER: void initialize_video(void)] This function initializes graphics hardware and switches screen to the graphics mode. Call this function after the text based part of the experiment (like asking for a patient's name or number, etc.) is completed. Usage: video.close_video(); [HEADER: void close_video(void)] Call this function to switch back to text mode. Usually, it is a function
to call at the end of experiment before exiting. Calling it before initialize_video()
makes no sense.
Usage: video.set_pointsize(size); [HEADER: void set_pointsize(int size)] This function allows you to set the text size that will be used by the
EPL. The value of size that it is passed should be the point size
(according to X window's idea of point sizes) that you want all text to
be.
Usage: video.set_textsize(size); [HEADER: void set_textsize(int size)] Note: This function is used for backwards compatibility
with the old version of the EPL. For new experiments, use the video.set_pointsize
function. set_textsize sets the text size according an unusual metric
used in the dos version of EPL.
Usage: video.write_text("My Text", x-position, y-position, horiz-justification, vertical- justification); [HEADER: void write_text(char *s, double x, double y, int h = 0, int v = 0)] This function writes a text string *s at a location specified
by x and y on the screen. For example, video.write_text("Hello,
Screen!!!", -.5, .5) will write the specified string in a left upper
quarter of the screen. The text is drawn in a white color. Optional arguments
h and v allow to alter (h)orizontal and (v)ertical justifications
of the text around x and y. The possible values for h
are: 0 (LEFT_TEXT), 1 (CENTER_TEXT), and 2 (RIGHT_TEXT). The possible values
for v are: 0 (BOTTOM_TEXT), 1 (CENTER_TEXT), 2(TOP_TEXT). By default,
text is left justified (0) horizontally, and centered (1) vertically.
Usage: video.clear_text("My Text", x-position, y-position, horiz-justification, vertical- justification); [HEADER: void clear_text(char *s, double x, double y, int h = 0, int v = 0);] This function erases a text string *s at a location specified by x and y on the screen. For example, video.clear_text("Hello, Screen!!!", -.5, .5) will erase the specified string in a left upper quarter of the screen. Use this function to undo the results of previous write_text call. Usage: video.clear_screen( ); [HEADER: void clear_screen()] This function clears the screen with the current background color. Note:
Since this function takes lots of resources and is comparativelyslow; do
not use it excessively (don't call it instead of clear_text(), for
example).
Usage: video.change_fore(char *); [HEADER: void change_fore(char *color_name)] This function changes the foreground color used by all video functions
to the color specified in color_name. For a list of valid colors
and their names, refer to the file /usr/lib/X11/rgb.txt. The default
foreground color is white.
Usage: video.change_back(char *); [HEADER: void change_back(char *color_name)] This function changes the background color used by all video functions
to the color specified in color_name. For a list of valid colors
and their names, refer to the file /usr/lib/X11/rgb.txt. The default
background color is black.
Usage: video.show_file("my_file"); [HEADER: void show_file(char *filename)] Use this function to display a text file on a screen which has been
previously initialized by the initialize_video() function. The
user may scroll up and down using the arrow keys, page up and page down,
and space (same as page down). When the bottom of the file is in view
the user may press enter to finish viewing. The user will see
context-sensitive instructions at the bottom of the screen. Long lines
will be word-wrapped. Usage: video.show_xpm_file("my_file", x, y); [HEADER: void show_xpm_file(char *filename, double x, double y)] where filename is a string containing path (optional) and file name of 256-color bitmap file. x and y are horizontal and vertical position of the .bmp file respectively. NOTE: This function is outdated and is only included here for backwards compatibility. Please use load_xpm and display_xpm and display_xpm_center below. Function show_xpm_file displays xpm files with left upper corner of the bitmap being in (x, y) coordinate. Example: Svga256 video();
void main(void)
{
video.initialize_video();
video.show_xpm_file("apple.xpm",-1, 1); // Show apple.xpm file in left upper corner of the screen
}
The preferred way of displaying graphics in the EPL is to use this suite
of functions. By separating the process of loading the bitmap from
that of displaying it, it will increase performance by allowing the X server
to cache the images internally. In order to use these functions,
at the beginning of the program (after the initialize_video call, but before
any real work is done) call initxpms with the maximum number of images
that you plan on having the program display throughout its execution.
Then, load each image that you plan on having the program display throughout
its execution with the load_xpm function. From this point on each
image can be referred to by its index within the EPL's internal list of
images. You can have the EPL automatically assign each image an index
(by just calling load_xpm with an image name and nothing else), or you
can specify the exact index for an image (by specifying the image index
in the call to load_xpm.). Then, to display an image, call display_xpm
with the index of the image that you want to display, and the coordinates
that you want the image displayed at. This method is faster than
using the show_xpm_file function because after all the images are loaded
with the load_xpm functions, they will be much faster to access than if
they had to be loaded from disk each time. Here is an example
of how the code should be used:
vid.init_xpms(10); //maximum of 10 xpms allowed vid.load_xpm("image0xpm"); //puts image0.xpm in index 0 vid.load_xpm("image1xpm"); //puts image1xpm in index 1 vid.load_xpm("image2.xpm"); //puts image2.xpm in index 2 vid.load_xpm("image3.xpm"); //puts image3.xpm in index 3 vid.load_xpm("image1-new.xpm",1); //puts image1-new.xpm in index 1, replacing image1.xpm vid.display_xpm(1,0,0); //displays image1-new.xpm at position 0,0 vid.display_xpm(2,1,1); //displays image2.xpm at position 1,1 Usage: video.kbclear ( ); This function clears the keyboard buffer. This is a very useful way of preventing the program from going on by itself when a subject holds down a key too long inadvertantly filling the buffer. Function wait_CTRL_pressed( ) waits for the user to press one of the two CONTROL keys on the bottom of the keyboard. This function returns which control key was pressed (1=right; 0=left). The function wait_CTRL_depressed( ) waits for the user to release one of the two CONTROL keys. This function returns nothing. To get a return value, use the function get_user_response(int yes_left) to get a yes or no response using the control keys. If yes_left=0, the function will return 1 if the right CTRL key is pressed and 0 if the left CTRL key is pressed. If yes_left=1, the function will return 0 if the right CTRL key is pressed and 1 if the left CTRL key is pressed. To be certain that the buffer is kept clear, it is usually best to use get_user_response, followed by wait_CTRL_depressed(int yes_left) and then kbclear() to clear the buffer. video.kbclear(); // clear any previous keypresses
Here is a typical example of using video library: // Instantiate video object
Svga256 video(SVGA1024x768x256);
?
// Set up video hardware and switch to graphics mode
video.initialize_video();
// Present introduction
video.show_file("txt\intro.txt");
// Clear screen
video.clear_screen();
// Write something on the screen
video.write_text("Hello, screen!!!", -.5, .5);
// Erase that something.
video.clear_text("Hello, screen!!!", -.5, .5);
// Switch back to text mode
video.close_video();
Usage: video.close_video(); This function closes the window created for the video object returns
the program to text mode. This function should be called at the end
of an experiment.
Usage: Video.get_key(); [HEADER: char get_key()] This function should be used by the user to get a character from teh
keyboard. This function serves the same function as the normal C
getch function under the EPL X windows implementation. get_key waits
until a key is hit (if one is not already present in the input buffer before
it is called) and then returns the character value of the key that is pressed.
Usage: Video.get_keyevent(XKeyEvent *) [HEADER: void get_keyevent(&xkeyevent)] This function serves the same purpose as get_key above, except that it places a copy of the actual XKeyEvent object into the XKeyEvent pointer that is passed to it (rather than simply extracting the character value as get_key does). This can be used if you want to determine the values of the modifier keys (alt, shift, control....) at the time that the key is pressed. Usage: Video.xkbhit() [header int xkbhit()] This function checks the event queue and determines if it contains a keyboard event that can be obtained via either get_key or get_keyevent above. This allows the program to do background processing while waiting for the user to hit a key, as in the following example: while (!vid.xkbhit()){
Usage: Video.kbclear(); This function flushes the keyboard input buffer of the program.
This function is if you want to have the program only count key presses
after a certain point, so that those presses before this function call
are ignored.
Usage:Video.mouse_show(); This function makes the mouse pointer visible on the screen in preparation
for having the user use the pointer for part of the experiment.
Usage: Video.mouse_hide(); This function hides the mouse pointer. This function should be
called after a mouse_show function call in order to put the EPL in the
state that it had previous to the mouse pointer being shown.
Usage: Video.mouse_get_event(double *xpos, double *ypos, int *buttons); This function is called by the program to get the current state of the
mouse pointer. The pointers that are passed in will contain the correct
values corresponding to the currentt status of the mouse: xpos will contain
the x coordinate of the mouse pointer, ypos will contain the y coordinate
of the y pointer, and buttons will contain a constant that corresponds
to the status of the buttons on the mouse that are pressed (see below table).
2.5 Voice Segmentation The words in recorded voice files are usually identified during scoring and their timing is recorded for further processing. Parse, the voice segmentation program, is rather large standalone function written to semi-automate and improve a laborious task of manual word segmentation. It is declared in the following way:
Usage: parse_file(fname, pname, data, datasize, parsename); [HEADER: int parse_file(char *fname, char *pname, DATA *data, int data_size,char *parsename);] where fname is a voice file name containing words to segment, pname is a pool file name containing words that can occur in fname voice file, *data is a pointer to an array of data structures containing information on words found in a voice file, and data_size is a number of elements in data. parsename is the filename that the parse program will write the parse file (the parsefile is not always necessary, as its contents is duplicated in the data parameter.) A brief example should clarify everything: DATA wdata[100];
int i=0;
wparse_file("my_exp/free_1.voc", "my_exp/free_1.txt", wdata, 100, "/tmp/parse.tmp"); //parsefilename doesn't matter, so it is stored to /tmp
while (wdata[i].pool_index>=0)
cout << "Pool Index: " << wdata[i].pool_index << "Interresponse Time: " <<
wdata[i].interresponse << "Word started at: " << wdata[i].voice_start ;
In above example, wparse_file was given voice file free_1.voc
in my_exp directory to segment, free_1.txt word pool in the
same directory containing all words that could be in free_1.voc
and a wdata array of 100 "blank" data structures of type DATA
which could accommodate information on up to 100 words. Upon return, data
structure wdata is filled with sequential information on segmented
words. Each entry in DATA data structure has the following fields:
typedef struct
{
int pool_index; // Index of the word in a word pool
long interresponse; // Interresponse time between beginnings of two adjacent words
long voice_start; // Offset of the word from the beginning of the voice file
} DATA;
Now, suppose subject was given five words to memorize from a word pool
free_1.txt:
(0)DUTY, (1)ACCORD, (2)PLACE, (3)REVENGE, (4)MUTE Subject recalled the following words that were recorded in a file free_1.voc: DUTY, MUTE, ACCORD After the call to parse_file the pool_index of wdata structure should be equal to the following values (assuming that scoring was done correctly): wdata[0].pool_index 0 Index of DUTY in free_1.txt
Instructions for the parse program:When the program starts, it will try to select what it thinks is the first word. If it finds a word, it will highlight the waveform that it thinks it is. If not, you will have to what you think is the waveform yourself. You can scroll through the waveform using either the horizontal scroll bar on the bottom of the screen, or using the left ( [ ) or right ( ] ) brackets. You can select a waveform with the mouse (hold down the button to start the selection and move the mouse to the end of it. The start and and of a selection can be changed easily via the arrow keys. The left and right arrow will move what is selected as the onset of the word, if alt or control is held down while the arrow is pressed, the onset will move even more. If shift is held while the arrow is pressed, you will change the offset of the word selection instead of the onset. Once you have a selection, you can press the spacebar to have it play the sound you selected. Typically, after having it play the sound, you will change the onset/offset so that it more closely matches the word. Once you have a word selected, you can type in the word. the parse program will display the letters that you typed in the bottom of the screen. It will only let you type in letters that fit words that are present in the word pool (so as to prevent spelling errors). Once the correct word is entered, press enter to have the program mark it down and go to the next word. If the actual word that is spoken by the user is not in the word list (in other words, an intrusion), press '!' (shift 1) and a dialog box will pop up that will let you enter the word. Type the word in this dialog box and hit enter to have the program register it. Once you enter in a word, it will try to find the next word in the file. The program will play all of the sound from the end of the current word to the end of what it thinks is the next word. In this way, you can hear if the program made a mistake in parsing the sound file and missed a word. If the program cannot any next word, you will have to do it manually in the same manner as before. This same basic pattern contines, and once you reach the end of the program, press the "quit" button to quit. You can press the help button to have the program give you a small help box that will give you a short reminder of how the program works. You can press the up and down arrow to zoom in and out of the file. This helps in parsing large data files. You can also zoom in and out using the slider at the bottom of the screen. The parameters that the program uses in parsing the sound file for words are set at defaults inside the program, but they can be changed by changing the parse.ini file. There is a sample version of this file in the parse source code directory. In order to use this file to change the parsing parameters, copy this file to the directory where the experiment that you will be doing parsing is, and then edit the file so that the parameters that it contains are what you would like to use. There is more information available on how the word parsing algorithm works and how to change the parameters in the source code for the parsing program. Parse Key Bindings
2.6 Timer A timer library provides two exact timing routines . The first one, timer.start [HEADER: void start(void)], starts the timer, the second, timer.get [HEADER: usigned long get(void)] returns the time, in milliseconds, expired from the call to start(). A second call to start() resets the timer:
2.7 Synchronization Object The synchronization protocol works as follows:
g++ experiment_name.cc -o experiment_name -L/usr/X11R6/lib -lX11 -lXpm -lpthread -lepl You should change experiment_name to whatever the actual name of the files in your experiment are. The system libraries that the EPL needs to be linked with to work are the X11, Xpm and pthreads. chmod a+r /dev/dsp chmod a+w /dev/dsp Note that this does present a potential security problem.... Tips for porting experiments to Linux EPL from DOS EPL As one may expect, there are a number of differences that must be taken into account when trying to move an experimetn that was programmed under the DOS EPL to the Linux EPL. These differences stem from both differences between the actual implementation of the EPL on these platforms (the API is not completely compatible), and lower-level differences between DOS and Linux programming. Here is a summary of some of the things that you should keep track of when porting an experiment to the Linux EPL.
| ||||||||||||||||||||||||||||||||||||||||||||||||