Interfacing
experiments with an external environment
There are certain cases in which you may need to have more
on-line control of what Runner presents to subjects. For
example, you may want to determine on-line what the next
stimulus presented to subjects is based on their current
performance in the task, as is often the case in psychophysical
experiments (and many other situations). To accomplish this,
MouseTracker can interface on-line with an external application
or programming environment (e.g., MATLAB, Python, Java, etc.) to
receive information trial-by-trial on what stimulus (or stimulus
sequence) and response options to present to subjects, and then
also send back information on what the selected response was,
whether it was correct/incorrect, and what the trajectory's
initiation and response times were (so that the external
environment can then generate information for the next trial,
for MouseTracker to receive).
The interface is accomplished through the Windows
Clipboard. Although of course the Clipboard is a crude way to
interface, it is the simplest and easiest and therefore likely
able to be taken advantage of by the most researchers. More
sophisticated approaches, e.g., COM or ActiveX, may be developed
in the future, but the Windows Clipboard should do the trick for
now while being broadly exploitable by all. Thus, so long as
your external environment (e.g., MATLAB) has access to the
Clipboard, it should be able to interface with MouseTracker. The
only constraint is that other applications should not be writing
to the Clipboard while an experiment is underway (although it'd
be unlikely for this to be the case anyway).
Setup: Activating Clipboard Interface (cbinterface) and
specifying trials to be interfaced
Add into your experiment .CSV file a new experimental
parameter, cbinterface, and specify an ID number (e.g.,
123). The ID number is arbitrary, but just makes sure that
MouseTracker and the external environment are on the same page.
Optionally, you can also add a cbinterfaceexec
parameter to have MouseTracker launch an executable after the
subject ID has been entered in at the beginning of an
experiment. For example, you can automatically launch a
particular MATLAB script that you are interfacing with using
this parameter. See the Adaptive MATLAB example below.
In your stimuli/response list, you need to specify which
trials you would like to be specified by the external
environment on-line. To let MouseTracker know you'd like a trial
to be "to be determined", use trial-type 7 and you can simply
leave the rest of the information blank (as it will be
determined later by the external environment). Using the
randomize-block column ("rand" here) does not make a whole lot
of sense using cbinterface, because the external
environment is determining all the trials. Thus, randomization
should be done by the external environment, not MouseTracker.
For example, if you'd like an instruction screen to first
appear, followed by 10 trials to be determined by the external
environment, then a break screen, and then another 10 trials to
be determined by the external environment, you could specify the
following:
Note that because the external environment can specify
on-line a trial-type 0 as well, that you could in theory make
all these trials trial-type 7 (i.e., change the instruction and
break slides to trial-type 7), and then have the external
environment specify those as well. Whichever way works.
MouseTracker and the external environment will now
communicate by placing comma-separated messages on the Clipboard
in a specific sequence (the sequence is maintained by a message
ID). When MouseTracker receives information on the Clipboard, it
should always start with:
MT[interface ID]_[message
ID]_IN
e.g., MT123_1_IN
When MouseTracker sends information to the Clipboard (to be
received by an external environment), it will start with:
MT[interface ID]_[message
ID]_OUT
e.g., MT123_2_OUT
[interface ID] = The ID number
specified in the cbinterface parameter of the .CSV file
[message ID] = This is simply
the current index of how many messages have been sent/received
by MouseTracker. It is important so that MouseTracker and the
external environment make sure they are at the same step of the
experiment's sequence (e.g., to make sure that MouseTracker did
not miss a message intended for it from the external
environment). The very first message sent by MouseTracker has a
message ID of 1, and then MouseTracker waits for an incoming
message with a message ID of 2. It will then send out a message
with a message ID of 3, and then wait for a message with a
message ID of 4, and so on and so forth.
Step
1: Initial WAITING signal from MouseTracker
When MouseTracker has loaded the experiment successfully,
it places an initial waiting signal on the Clipboard:
(The arbitrary number of 123
will be used to denote the interface ID for the remainder of the
article, for simplicity)
MT123_1_OUT,WAITING
This now lets the external environment know that
MouseTracker is loaded up and ready to begin.
Step 2:
START signal from external environment
MouseTracker will now wait for the external environment to load
up and be ready. When it is ready, the external environment
should place the following on the Clipboard:
MT123_2_IN,START
Step 3:
START confirmation from MouseTracker
MouseTracker will confirm that it has received the START signal
from the external environment by next putting the following on
the Clipboard:
MT123_3_OUT,START
(Note how the message ID is continuously incrementing, to keep
pace with one another.)
At this point, any time that MouseTracker encounters a
trial-type 7 in the experiment .CSV stimuli/response list, it is
going to wait and probe the Clipboard for the appropriate
message ID. In the example above, the first trial is trial-type
0, the instructions screen. Once that finishes, the next trial
is trial-type 7, at which point MouseTracker will wait for a
message from the external environment. The first trial to be run
through the interface should therefore have a message ID of 4
(as the previous message--the START confirmation--had an ID of
3).
Step 4: External environment sends trial information
To tell MouseTracker how to run the trial (with trial-type 7),
the external environment should place on the Clipboard the
typical header plus the trial's comma-separated-value
information exactly as it would appear in the experiment .CSV
file (see
Creating Experiments | Stimuli and response list for
information on how to format this). For example:
MT123_4_IN,2,MANGO,less_common-word,interface,1,2,FRUIT,VEGGIE
Thus, MouseTracker is going to load the current trial
(trial-type 7) as if the experiment .CSV had specified the
following comma-separated values:
2,MANGO,less_common-word,interface,1,2,FRUIT,VEGGIE. And
thus, MouseTracker will now treat this trial as a trial-type 2
(letter-string stimulus), will display the string "MANGO", will
assign the trial to condition "less_common-word", will display
"FRUIT" and "VEGGIE" response options, with the correct response
being "FRUIT" and incorrect being "VEGGIE". Note that
"interface" has been put in the randomize-block column.
Typically, without using cbinterface, a number would be
put here to tell MouseTracker to randomize the trial with a
certain block. Randomization is pretty meaningless here because
the external environment is the one deciding what each trial is.
Thus, the randomize-block column is not used and "interface" has
been put only because it is arbitrary (anything else could be
used).
Step 5: MouseTracker sends external environment response
information
Once the subject clicks on a response, MouseTracker places a
message on the clipboard to let the external environment know
how the subject did. It sends it in the following format (using
comma-separated values):
Interface Header Selected Response #
Incorrect?
Initiation Time
Response Time
For example, when the subject completes the trial above,
MouseTracker would send a message such as:
MT123_5_OUT,1,0,194,1024
MouseTracker is letting the external environment know that on
the last trial the subject selected response #1, which was
correct (if incorrect, the value would be a 1, not 0), with an
initiation time of 194 ms, and a response time of 1024 ms. And
to be sure MouseTracker and the external environment are talking
about the same trial, the current message ID (5) would mean that
the last trial this is in reference to would have been specified
by a message ID of 4 (as in Step 4 above).
Now, on the next trial-type 7 encountered by MouseTracker
(e.g., the very next trial), it will wait again for more trial
information (repeat Step 4 above). In such a way, the external
environment can use previous trials' performance to guide what
the next trials will be (and Step 4 and Step 5 repeat until
there are no more trials with trial-type 7).
If you'd like to terminate the experiment adaptively, you
can have the external environment send an "END" signal to
MouseTracker. Just use the correct message ID and have the
external environment send "END" (e.g., MT123_119_IN,END),
and MouseTracker will send an "END" signal back to the external
environment (see Step 6 below), save the data, and end.
Otherwise, MouseTraker will terminate after the last trial of
the experiment (see below).
Step 6: END
signal from MouseTracker
Once the last trial of the experiment runs, MouseTracker lets
the external environment know that the experiment has finished
by sending a final END signal. It then saves data and ends.
Using an arbitrary message ID of 120 (assuming there are more
messages sent/received across the experiment), MouseTracker
sends:
MT123_120_OUT,END
Viewing the cbinterface
log
While an experiment is running, you can view the current
cbinterface log, which contains all the messages
successfully sent and received through the interface. Just press
F9. This is especially useful for debugging purposes.
MATLAB
Example: Adaptive Task
An example of an experiment interfaced with MATLAB is
provided in the sample folder. The Runner experiment is at
C:\MouseTracker\sample\adaptive_matlab.csv, and the
corresponding MATLAB script file is at C:\MouseTracker\sample\adaptive_matlab.m.
The MATLAB script is commented in detail, so that you can easily
understand how to generate your own interface using MATLAB (or
some other external environment).
In the experiment .CSV file, the cbinterfaceexec
parameter specifies the path for the MATLAB executable, as well
as the path of the adaptive_matlab.m script, so that it can
automatically be launched in the background when the Runner
experiment begins. Please update the parameter with the correct MATLAB path if you would like Runner to automatically launch the
MATLAB script. (Because the path involves quotation marks, I
suggest editing this parameter in Notepad, rather than Excel).
If an invalid path is specified when running the .CSV file, an
error will be generated and the experiment will not start. Note that this parameter is by no means required,
and you can simply launch
your external environment (e.g., MATLAB) on your own. Once the
subject ID is set in the Runner experiment, it will just wait
for the START signal (whether you run MATLAB on your own or
automatically through Runner). This is just to make things
easier for you and not have to switch windows and such while a
subject is waiting to start the experiment.
This example task is silly and doesn't make a whole lot of
sense, but it provides a good demonstration of how to create
adaptive MouseTracker paradigms, where future trials are
determined by subjects' past responses or performance. Here,
subjects go through 10 trials. On each trial, a technology brand
(e.g., Microsoft) or a fashion brand (e.g., Gucci) is presented
as a word stimulus, and subjects need to decide how valuable the
brand is. On the first trial in the experiment, the two response
alternatives start as $100 and $200. If the subject selects the
larger alternative (e.g., $200), then on the next trial the two
response alternatives double in size ($200 and $400). If the
subject selects the smaller alternative (e.g., $100), then on
the next trial the two response alternatives are cut in half
($50 and $100).
Again, this experiment makes absolutely no sense and should
never be run on real subjects. It just was a fast way off the
top of my head to show how to create an adaptive MouseTracker
paradigm, using subjects' prior data to determine future trials.
For example, you could use past responses to determine future
stimuli, a current measure of accuracy/performance to determine
how easy/difficult future trials will be, or change future
stimuli/responses based on subjects' current initiation times or
response times, among other possibilities.
To understand the interface, it is best to check out the
experiment .CSV file and thoroughly go over the MATLAB script
file, looking at the code in tandem with the comments. You
should be able to generate an interface of your own in no time.
If it is helpful, here are the Clipboard transmissions from
the cbinterface log (F9 key in Runner) for a run of this
experiment:
8/23/2011 5:31:00 PM >>> MT123_1_OUT,WAITING
8/23/2011 5:31:02 PM <<< MT123_2_IN,START
8/23/2011 5:31:02 PM >>> MT123_3_OUT,START
8/23/2011 5:31:04 PM <<<
MT123_4_IN,2,Microsoft,technology,,1,2,$100,$200
8/23/2011 5:31:08 PM >>> MT123_5_OUT,1,0,158,1327
8/23/2011 5:31:08 PM <<<
MT123_6_IN,2,Gucci,fashion,,2,1,$50,$100
8/23/2011 5:31:12 PM >>> MT123_7_OUT,2,0,195,970
8/23/2011 5:31:12 PM <<<
MT123_8_IN,2,Apple,technology,,1,2,$100,$200
8/23/2011 5:31:16 PM >>> MT123_9_OUT,1,0,352,1480
8/23/2011 5:31:16 PM <<<
MT123_10_IN,2,Sony,technology,,1,2,$50,$100
8/23/2011 5:31:20 PM >>> MT123_11_OUT,2,1,165,863
8/23/2011 5:31:20 PM <<<
MT123_12_IN,2,Prada,fashion,,2,1,$100,$200
8/23/2011 5:31:23 PM >>> MT123_13_OUT,2,0,135,620
8/23/2011 5:31:23 PM <<< MT123_14_IN,2,Marc Jacobs,fashion,,2,1,$200,$400
8/23/2011 5:31:27 PM >>> MT123_15_OUT,2,0,95,805
8/23/2011 5:31:27 PM <<<
MT123_16_IN,2,IBM,technology,,1,2,$400,$800
8/23/2011 5:31:30 PM >>> MT123_17_OUT,2,1,199,907
8/23/2011 5:31:30 PM <<<
MT123_18_IN,2,Versace,fashion,,2,1,$800,$1600
8/23/2011 5:31:53 PM >>> MT123_19_OUT,1,1,116,910
8/23/2011 5:31:53 PM <<<
MT123_20_IN,2,Toshiba,technology,,1,2,$400,$800
8/23/2011 5:31:56 PM >>> MT123_21_OUT,2,1,367,835
8/23/2011 5:31:57 PM <<<
MT123_22_IN,2,Valentino,fashion,,2,1,$800,$1600
8/23/2011 5:32:01 PM >>> MT123_23_OUT,1,1,172,807
8/23/2011 5:32:01 PM >>> MT123_24_OUT,END
Good luck!
