Question regarding multisplit and rank assignment
Posted: Sun May 10, 2020 2:24 pm
Hello, I am attempting to convert my working network model from serial to parallel to cut down the run time. It is implemented on a windows server edition with python code and biophysical single cell neuron models with complex morphologies. It will be run on a multi-processor computer with 16 cores, and 32 logical processors, and 128gb of RAM. This current test network is not large (6 neurons). This network is comprised of 2 subnetworks of 3 neurons each. Only 1 neuron of each subnetwork (I'll call this cell type 1) synapses onto the other 2 cells (cell type 2). Cell type 2 only receives input. Cell type 1 has 3 spike detectors. There is also some external poisson input via vecstim to all the neurons. The two subnetworks are also interconnected via cell type 1.
I have read through all the documentation for parallel context and read a few of Michael Hines' papers from 2008 and 2009. I've also watched a nice youtube video from Michael Hines from a few years ago (HBP education). The code is almost there, but I'm struggling with a few aspects that seem simple, but perhaps I am not understanding it well.
My first question is about set_gid2node(). I believe have enough processors to have one gid per processor so a "round-robin" style of assignment is not necessary. All of my gids in the network are unique (0 through 9). Could I simply create the assigments in this fashion:
self is the python class object: subnetwork1 or subnetwork2
mpiexec -n 10 python diffrun.py <-my script to run a simulation. 10 processors for 10 gids (3 for cell type 1 + 2 for cell type 2)*2 subnetworks. I put this code is the class object scripts and so I'm guessing I would have to offset the rank numbers (pc.id()) each time I instantiate a subnetwork. i.e. (rank 0-4 for subnetwork1 and rank 5-9 for subnetwork2). I have already tried this and I am getting errors like: 2 NEURON: gid=0 has not been set on rank 2. I'm confused because I don't understand how gid 0 is being forced to be on rank 2. I must be missing something...
My next question is regarding multiple spike detectors in one cell and multisplit. I am having trouble understanding why multisplit is necessary when I have created unique gids for each spike detector and placed these gids on different processors (assuming I can get my issue above to work). It seems to me that it is a method to create another id (sid) for a split node location, but why must this split node location be created? If it is necessary to include pc.multisplit() in my code, I am unsure where to place it in my code.
Any help is much appreciated. Thanks a lot and thank you to the moderators for creating these tools for scientists.
I have read through all the documentation for parallel context and read a few of Michael Hines' papers from 2008 and 2009. I've also watched a nice youtube video from Michael Hines from a few years ago (HBP education). The code is almost there, but I'm struggling with a few aspects that seem simple, but perhaps I am not understanding it well.
My first question is about set_gid2node(). I believe have enough processors to have one gid per processor so a "round-robin" style of assignment is not necessary. All of my gids in the network are unique (0 through 9). Could I simply create the assigments in this fashion:
self is the python class object: subnetwork1 or subnetwork2
Code: Select all
self.gidlist = list(range(pc.id()+self.offset, (pc.nhost()/2)+self.offset))
for gid in self.gidlist:
pc.set_gid2node(gid+self.offset, pc.id())
pc.cell(self.celltype_1._gid, self.celltype_1._spike_detector)
pc.cell(self.celltype_1._gid2, self.celltype_1._release_detector)
pc.cell(self.celltype_1._gid3, self.celltype_1._release_detector2)
for cell in self.celltype2cells:
pc.cell(cell._gid, cell._spike_detector)
My next question is regarding multiple spike detectors in one cell and multisplit. I am having trouble understanding why multisplit is necessary when I have created unique gids for each spike detector and placed these gids on different processors (assuming I can get my issue above to work). It seems to me that it is a method to create another id (sid) for a split node location, but why must this split node location be created? If it is necessary to include pc.multisplit() in my code, I am unsure where to place it in my code.
Any help is much appreciated. Thanks a lot and thank you to the moderators for creating these tools for scientists.