Page 1 of 1
How to make VecStim (vecevent.mod) thread safe?
Posted: Fri Feb 13, 2015 5:00 pm
by aaronmil
I find VecStim very useful for driving spikes with a list of spike times from python. However, I'm trying to make all my .mod files thread safe and cvode-compatible. Compiling vecevent.mod results in:
Code: Select all
Translating vecevent.mod into vecevent.c
Notice: VERBATIM blocks are not thread safe
Any ideas? Here is the entire .mod file for convenient reference:
Code: Select all
: Vector stream of events
NEURON {
ARTIFICIAL_CELL VecStim
}
ASSIGNED {
index
etime (ms)
space
}
INITIAL {
index = 0
element()
if (index > 0) {
net_send(etime - t, 1)
}
}
NET_RECEIVE (w) {
if (flag == 1) {
net_event(t)
element()
if (index > 0) {
net_send(etime - t, 1)
}
}
}
VERBATIM
extern double* vector_vec();
extern int vector_capacity();
extern void* vector_arg();
ENDVERBATIM
PROCEDURE element() {
VERBATIM
{ void* vv; int i, size; double* px;
i = (int)index;
if (i >= 0) {
vv = *((void**)(&space));
if (vv) {
size = vector_capacity(vv);
px = vector_vec(vv);
if (i < size) {
etime = px[i];
index += 1.;
}else{
index = -1.;
}
}else{
index = -1.;
}
}
}
ENDVERBATIM
}
PROCEDURE play() {
VERBATIM
void** vv;
vv = (void**)(&space);
*vv = (void*)0;
if (ifarg(1)) {
*vv = vector_arg(1);
}
ENDVERBATIM
}
Re: How to make VecStim (vecevent.mod) thread safe?
Posted: Sun Feb 15, 2015 2:50 pm
by ted
VecStim is not going to be thread safe. It uses VERBATIM blocks, and as per the message emitted by nrnivmodl (or mknrndll), VERBATIM blocks are not thread safe. Which isn't really much of a loss because multithreaded parallel execution is not as efficient as mpi-based parallel execution.
Strong suggestion: use the latest implementation of VecStim--see
viewtopic.php?f=2&t=2147&p=13614#p13614
Re: How to make VecStim (vecevent.mod) thread safe?
Posted: Sun Feb 15, 2015 6:09 pm
by aaronmil
Thanks for the updated vecevent.mod
So even if the VecStim object is not threadsafe, if a single cell model was split into pieces to run in multiple threads, could VecStim objects still be used as sources for NetCon objects targeting point processes in the multisplit cell, or not? Artificial cells aren't even associated with sections, so maybe?
Also, I have another .mod file that uses VERBATIM blocks that compiles as thread safe. What makes the difference?
Code: Select all
NEURON {
POINT_PROCESS Pr
RANGE P, P0, random, f, tau_F, d1, tau_D1, d2, tau_D2, F, D1, D2, tlast
THREADSAFE
POINTER randObjPtr
}
PARAMETER {
: the (1) is needed for the range limits to be effective
P0 = 0.5 (1) < 0, 1 > : basal release probability
: these values are from Fig.3 in Varela et al. 1997
f = 0.917 (1) < 0, 1e9 > : additive facilitation per spike
tau_F = 94 (ms) < 1e-9, 1e9 > : rate of decay back to baseline following facilitation
d1 = 0.416 (1) < 0, 1 > : multiplicative fast depression per spike
tau_D1 = 380 (ms) < 1e-9, 1e9 > : rate of decay back to baseline following fast depression
d2 = 0.975 (1) < 0, 1 > : multiplicative slow depression per spike
tau_D2 = 9200 (ms) < 1e-9, 1e9 > : rate of decay back to baseline following slow depression
}
ASSIGNED {
P : instantaneous release probability
randObjPtr : pointer to a hoc random number generator Random.uniform(0,1)
random : individual instance of random number
F : current level of facilitation
D1 : current level of fast depression
D2 : current level of slow depression
tlast (ms) : time of last spike
}
INITIAL {
P = P0
random = 1
F = 1
D1 = 1
D2 = 1
}
NET_RECEIVE(weight) {
INITIAL {
tlast = t
}
F = 1 + (F-1)*exp(-(t - tlast)/tau_F)
D1 = 1 - (1-D1)*exp(-(t - tlast)/tau_D1)
: D2 = 1 - (1-D2)*exp(-(t - tlast)/tau_D2)
: if (P0*F*D1*D2 > 1) {
if (P0*F*D1 > 1) {
P = 1
} else {
: P = P0*F*D1*D2
P = P0*F*D1
}
random = randGen()
if (random <= P) {
net_event(t)
}
tlast = t
F = F + f
D1 = D1 * d1
: D2 = D2 * d2
}
VERBATIM
double nrn_random_pick(void* r);
void* nrn_random_arg(int argpos);
ENDVERBATIM
FUNCTION randGen() {
VERBATIM
if (_p_randObjPtr) {
/*
:Supports separate independent but reproducible streams for
: each instance. However, the corresponding hoc Random
: distribution MUST be set to Random.uniform(0,1)
*/
_lrandGen = nrn_random_pick(_p_randObjPtr);
}else{
hoc_execerror("Random object ref not set correctly for randObjPtr"," only via hoc Random");
}
ENDVERBATIM
}
PROCEDURE setRandObjRef() {
VERBATIM
void** pv4 = (void**)(&_p_randObjPtr);
if (ifarg(1)) {
*pv4 = nrn_random_arg(1);
}else{
*pv4 = (void*)0;
}
ENDVERBATIM
}
Re: How to make VecStim (vecevent.mod) thread safe?
Posted: Mon Feb 16, 2015 1:54 am
by hines
Adding the keyword, THREADSAFE, to the NEURON block in a mod file, is a claim by the author that the model is, in fact, thead safe. The only automatic help that the nocmodl translator
will provide in making this statement true is to promote all GLOBAL variables that are on the left hand side of assignment statements, into threads specific instance variables. I.e. instead
of thread shareing one memory location, there will be pc.nthread() memory locations devoted to the variable so a thread reading and writing to the variable will not cause a race condition.
The translator cannot determine if POINTER variables or the code in VERBATIM blocks is thread safe or not and so, conservatively assumes it is NOT thread safe and refuses to allow
an instance of the model into any thread other than thread 0. Upon a review of the code as well as the intended use of any POINTER variable, the author may determine that the
model is, in fact thread safe and so force that interpretation by adding the THREADSAFE keyword.
It is an accident that the THREADSAFE keyword does not turn off the "Notice: Use of POINTER is not thread safe." unless it appear before the POINTER declaration. So merely
put the THREADSAFE at the beginning of the NEURON block.
It turns out that vecevent.mod is, in fact, threadsafe if the POINTER is only used to reference a Vector object from Hoc. So THREADSAFE should be added to the beginning of the NEURON
block in vecevent.mod.
Re: How to make VecStim (vecevent.mod) thread safe?
Posted: Mon Feb 16, 2015 12:02 pm
by aaronmil
hines wrote:
It turns out that vecevent.mod is, in fact, threadsafe if the POINTER is only used to reference a Vector object from Hoc. So THREADSAFE should be added to the beginning of the NEURON
block in vecevent.mod.
This actually results in a series of errors:
Code: Select all
vecevent.c:282:23: error: parameter named '_p' is missing
Prop *_prop; double* _p; Datum* _ppvar; Datum* _thread;
^
vecevent.c:282:34: error: parameter named '_ppvar' is missing
Prop *_prop; double* _p; Datum* _ppvar; Datum* _thread;
^
vecevent.c:282:49: error: parameter named '_thread' is missing
Prop *_prop; double* _p; Datum* _ppvar; Datum* _thread;
^
vecevent.c:281:8: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
static _destructor(_prop)
~~~~~~ ^
vecevent.c:281:8: error: conflicting types for '_destructor'
vecevent.c:129:14: note: previous declaration is here
static void _destructor(Prop*);
^
vecevent.c:284:2: error: use of undeclared identifier '_thread'
_thread = (Datum*)0;
^
vecevent.c:285:2: error: use of undeclared identifier '_p'
_p = prop->param; _ppvar = _prop->dparam;
^
vecevent.c:285:7: error: use of undeclared identifier 'prop'
_p = prop->param; _ppvar = _prop->dparam;
^
vecevent.c:285:20: error: use of undeclared identifier '_ppvar'
_p = prop->param; _ppvar = _prop->dparam;
^
vecevent.c:290:21: error: use of undeclared identifier '_ppvar'
void* vv = (void*)(_p_ptr);
^
vecevent.c:44:16: note: expanded from macro '_p_ptr'
#define _p_ptr _ppvar[2]._pval
^
1 warning and 9 errors generated.
Re: How to make VecStim (vecevent.mod) thread safe?
Posted: Mon Feb 16, 2015 1:00 pm
by hines
is your version of the mod file the same as at
http://www.neuron.yale.edu/hg/neuron/nr ... cevent.mod
If so, what is your version of NEURON?
Re: How to make VecStim (vecevent.mod) thread safe?
Posted: Mon Feb 16, 2015 1:36 pm
by aaronmil
Yes.
NEURON -- VERSION 7.3 ansi (975:433096832316) 433096832316
Re: How to make VecStim (vecevent.mod) thread safe?
Posted: Mon Feb 16, 2015 2:08 pm
by hines
The THREADSAFE keyword did not cause compiler problems when I checked it today with the latest mercurial trunk version. I'm guessing the
problem will disappear if you use that or one of the 7.4 alpha distribution files at
http://www.neuron.yale.edu/ftp/neuron/versions/alpha/
Let me know if that is not the case.
Re: How to make VecStim (vecevent.mod) thread safe?
Posted: Tue Feb 17, 2015 11:29 am
by aaronmil
hines wrote:The THREADSAFE keyword did not cause compiler problems when I checked it today with the latest mercurial trunk version. I'm guessing the
problem will disappear if you use that or one of the 7.4 alpha distribution files at
http://www.neuron.yale.edu/ftp/neuron/versions/alpha/
Let me know if that is not the case.
Hmm. Is it true that if I want to use anaconda python or any python other than cython that I am stuck building from the ansi branch? Doesn't look like that has been updated recently.
I followed your advice and built from the latest trunk. It generated quite a few warnings and had a long pause during 'make install', but it completed and appears to be functioning properly. I still get the same compile errors when adding 'THREADSAFE' to the NEURON block in vecevent.mod
Re: How to make VecStim (vecevent.mod) thread safe?
Posted: Tue Feb 17, 2015 11:50 am
by hines
I assume you want to launch anaconda's version of python and import neuron. What machine are we talking about?
If mswin then use:
http://www.neuron.yale.edu/ftp/neuron/v ... -setup.exe
if mac os x then use:
http://www.neuron.yale.edu/ftp/neuron/v ... 10.7.5.dmg
if linux then people often build from sources, but
http://www.neuron.yale.edu/ftp/neuron/v ... x86_64.deb
is supposed to work. In any case, for launching python, it should be sufficient to
export PYTHONPATH=/where/you/installed/neuron/lib/python # but check to see if lib is where the python folder exists or if it is lib64
Re: How to make VecStim (vecevent.mod) thread safe?
Posted: Tue Feb 17, 2015 12:27 pm
by aaronmil
hines wrote:I assume you want to launch anaconda's version of python and import neuron. What machine are we talking about?
Mac. Has that image been built --with-paranrn to work with mpi?
Re: How to make VecStim (vecevent.mod) thread safe?
Posted: Tue Feb 17, 2015 2:38 pm
by hines
Yes. All the distributions are built using --with-paranrn=dynamic --with-nrnpython=dynamic .
On the mac, you can use mpi if you install openmpi .
Re: How to make VecStim (vecevent.mod) thread safe?
Posted: Tue Feb 17, 2015 2:48 pm
by aaronmil
Thanks for all your help, Michael. I installed from this image, but still get the same compile errors when attempting to make vecevent.mod thread safe.
Going back to the original question, does it matter? If vecevent is an artifical cell used to drive a netcon, won't it be agnostic to whether or not the target of the netcon has been split into more than one thread?
Re: How to make VecStim (vecevent.mod) thread safe?
Posted: Thu Feb 19, 2015 7:47 am
by hines
My apologies. There was a bug in the nmodl translator which produced uncompilable code for the combination of DESTRUCTOR and THREADSAFE.
http://www.neuron.yale.edu/hg/neuron/nr ... 534c11dd4b
A new distribution for the mac is at
http://www.neuron.yale.edu/ftp/neuron/v ... 10.7.5.dmg
I must earlier had only looked at the elimination of the translator warning when THREADSAFE was prepended to the NEURON block.
Going back to the original question, does it matter? If vecevent is an artifical cell used to drive a netcon, won't it be agnostic to whether or not the target of the netcon has been split into more than one thread?
It likely does not matter in practical terms. The number of VecStims would have to be huge to make a performance difference as to whether they were all handled by thread 0 or were carefully placed
on the threads that contained the cells that received the input from VecStim. The vecevent is agnostic to whether it is the source of many netcons that have targets on other threads or other mpi
processes. It would be an experimental performance queston whether one would ever see a performance improvement in the case where many VecEvents each send spikes to many targets on
different threads.