Strange behavior Updating HOC Vectors inside Python+mpi deff
Posted: Thu Jan 22, 2015 8:31 pm
I have written an analysis to calculate transfer entropy on a large data set. The data set is so large I am using MPI such that I have access to more memory.
I am having trouble updating HOC Vectors inside Python definition, whenever I am using mpi4py as well, which is why I have made tautology statements like: 'h.py.j=j', the same is true for h.py.k=k, h.py.i=i. Strangely a statement like that will make update the python variable j, k, and i and if I did not make such a statement the values of j k i would not be updated. Which is catastrophic when they required to act as iterators in loops.
Sometimes k does get updated and sometimes it doesn't, it seems to be influenced by how many statements come before it. Is it possible that this is caused by blocking behavior of synchronous MPI calls like broadcast? Maybe the HOC object in Python is not designed to work with mpi4py.
Thanks for any help or suggestions. I think there will definitely be ways of simplify the code, I am open to any suggestions about how to do this. I think the whole thing should be rewritten as a more general python definition that performs a broadcast inside an iterator and returns a matrix.
I am having trouble updating HOC Vectors inside Python definition, whenever I am using mpi4py as well, which is why I have made tautology statements like: 'h.py.j=j', the same is true for h.py.k=k, h.py.i=i. Strangely a statement like that will make update the python variable j, k, and i and if I did not make such a statement the values of j k i would not be updated. Which is catastrophic when they required to act as iterators in loops.
Sometimes k does get updated and sometimes it doesn't, it seems to be influenced by how many statements come before it. Is it possible that this is caused by blocking behavior of synchronous MPI calls like broadcast? Maybe the HOC object in Python is not designed to work with mpi4py.
Code: Select all
def t_e(SIZE, NCELL, RANK, trentm):
h('storval = new Vector()')
h('objref vec')
h('vec=new Vector()')
for k in xrange(0, SIZE):
for j in xrange(0, int(NCELL)):
if RANK == k:
histogram = []
test = 0
h.py.j=j
h('py.test=py.int(pc.gid_exists(py.int(py.j)))')
if test != 0:
h('source=pc.gid2cell(py.int(py.j))')
h('vec=source.recvec1')
h('vec.x[0]=py.j')
histogram = h.source.recvec1.to_python()
if test == 0:
histogram = None
else:
histogram = None
histogram = COMM.bcast(histogram, root=k) # ie root = rank
h.py.k=k
h('py.gidn=int(vec.x[0])')
h('vec.x[0]=0')
gidn = int(gidn)
h('py.test2=int(vec.sum())')
test2 = h.vec.sum()
if int(test2) != 0:
print 'cell number ', gidn
for i in xrange(0, int(NCELL - 1)):
h.py.i=i
test3 = 0
h('py.test3=py.int(pc.gid_exists(py.int(py.i)))')
if test3 != 0:
h('target=pc.gid2cell(py.int(py.i))')
h('storval=normte(target.recvec1,vec,20)')
trentm[i][gidn] =float(h.storval.x[2])
return trentm
trentm=t_e(SIZE, NCELL, RANK, trentm,np)
Thanks for any help or suggestions. I think there will definitely be ways of simplify the code, I am open to any suggestions about how to do this. I think the whole thing should be rewritten as a more general python definition that performs a broadcast inside an iterator and returns a matrix.