Network Parallelization

General issues of interest both for network and
individual cell parallelization.

Moderator: hines

Post Reply
Krishna Chaitanya
Posts: 60
Joined: Wed Jan 18, 2012 12:25 am
Location: University of Pavia

Network Parallelization

Post by Krishna Chaitanya » Mon May 14, 2018 9:44 am

Hi,

I have followed your example of ring network in the paper "Translating Network Models to Parallel Hardware". I see that for the NetStim connection to the first cell is made through NetCon and the rest of the cells are connected through gid_connect.

I am not sure why NetCon was used for making connection from NetStim to the first cell. If I have around 20-30 NetStims and wanted to connect to cells distributed over multiple processes. How do I achieve it? The NetStims as well as the cells are distributed over multiple processes using round robin distribution.

Can I use pc.gid_connect in this scenario? The reason I am asking is when I printed the netcons created using gid_connect, I see that some show NetCon statement with numbers and the rest of them show as None.

I don't understand if there is a mistake there.

Can you kindly let me know.

Thank you.

ted
Site Admin
Posts: 5266
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: Network Parallelization

Post by ted » Mon May 14, 2018 10:53 am

The NetStim is indeed a spike source, so in principle it could be assigned a gid. But before doing that, consider the three leading reasons why one would assign a gid to a spike source.
1. At least one of its targets exists on a different host.
2. Load balance.
3. Convenience of recording the events that it generates in the context of parallel simulation, i.e. if all spike sources have gids, then it is simpler to just iterate over all of them and execute nc.record(tvec, idvec, i) than to have to do something special for the spike sources that don't have gids.

Does any of these pertain to this NetStim?

It targets only one synaptic mechanism, so placing it on the same host as its target synapse avoids the need to broadcast its events to all hosts. Furthermore, a NetStim poses negligible computational overhead, so its effect on load balance is inconsequential--which means there is no need to place it on any host other than the one where its target exists. Finally, for this toy example, this NetStim's spike time is completely predetermined, so there is no need to record the time of the event that it generates.

Krishna Chaitanya
Posts: 60
Joined: Wed Jan 18, 2012 12:25 am
Location: University of Pavia

Re: Network Parallelization

Post by Krishna Chaitanya » Mon May 14, 2018 4:42 pm

Hi Ted,

Thank you for the reply.

Is it possible to place source gids and Target gids on same host? I have a predetermined connection matrix. There is no problem in serial execution. But in parallel, I would have preferred a way where both source and Target gids exist on the same host. Thereby avoiding the need to search for target gid bcoz targets can only receive events from source gids on the same host. Isn't it?

Kindly let me know.

Thank you.

Krishna Chaitanya
Posts: 60
Joined: Wed Jan 18, 2012 12:25 am
Location: University of Pavia

Re: Network Parallelization

Post by Krishna Chaitanya » Thu May 17, 2018 7:14 am

Hi,

I am sorry for asking this question. I didn't understand the use of pc.gid_connect properly for connecting source and target. Now I could use it to connect neurons.

When we run the network as 'python init.py' and as 'mpiexec -n 1 python -mpi init.py', should generate same number of spikes, right? In my case, the number of spikes generated by these two commands for the same networks is significantly different.

Is it correct that the spikes should be same for both the commands shown above?

Thank you.

ted
Site Admin
Posts: 5266
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: Network Parallelization

Post by ted » Thu May 17, 2018 8:26 am

Is it possible to place source gids and Target gids on same host?
Properly implemented parallel code should work regardless of the host on which any particular spike source or target is located. The purpose of distributing cells over multiple hosts is to achieve load balance, which is critically important for performance. A little bit of load imbalance makes all other hosts wait for the slowest one to catch up, and that can take much longer than the time required for spike exchange between hosts.

If a network combines biophysical model cells (implemented with sections, therefore need numerical integration) with event-based artificial spiking cells like NEURON's built-in IntFire classes (which do not have differential equations that require numerical integration), some network architectures might theoretically benefit by strategic placement of the artificial spiking cells. But somehow nobody seems to be implementing hybrid models for which the placement of artificial spiking cells matters much to performance.

What does matter is load balance, and that's why model setup code often includes clever strategies such as cell splitting (so that the equations that describe different parts of a biophysical model cell are solved on different processors). Another important factor is the programmer's own time and effort required to implement and debug model setup code. That's why the simplest cell distribution algorithm (so-called "round-robin"; think of dealing out cards to players) is the starting point for many network setup strategies.

ted
Site Admin
Posts: 5266
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: Network Parallelization

Post by ted » Thu May 17, 2018 8:40 am

When we run the network as 'python init.py' and as 'mpiexec -n 1 python -mpi init.py', should generate same number of spikes, right?
That is one indication of properly parallelized code. Another is that results produced by parallelized code should be identical to the results of a serial implementation of the same network, regardless of how many hosts are used. To help achieve this goal, it is useful to create analysis code that produces reports you can use to verify that connectivity (including weights and delays) is correct. A strategy for diagnosing and fixing particular cases is to identify the first spike that is incorrect, then verify the properties of the cell that generated that spike, and finally verify all of the projections to that cell's synaptic mechanisms (i.e. verify the identities of the spike sources that send events to the target cell, the thresholds of all of the NetCons that detect the presynaptic spikes, and the weights and delays of all of the NetCons that implement the connections).

Post Reply