Parallel simulations of ephaptic connections

General issues of interest both for network and
individual cell parallelization.

Moderator: hines

Post Reply
mcapllonchj

Parallel simulations of ephaptic connections

Post by mcapllonchj »

Hello,

I am using a model that connects axons ephaptically using LinearMechanism. The ephaptic connections are modelled as "transverse" resistors connecting the opposite nodes of two interacting cells (so it's more like a network, not at all using a mean-field model; the longitudinal extracellular resistors are modelled with xraxial).

I would like to run my simulations in parallel because a serial simulation with many axons may last a good number of hours. To do so, I have attempted an equivalent to your network from "Translating network models to parallel hardware in NEURON" (Hines & Carnevale, 2008) in its Pythonic version. I have adapted my code accordingly, so I have created a class EphapticNW (even if it's not really a network) which intends to be the equivalent to your Ring.

In the serial simulations, each individual ephaptic connection uses an instance of a class Ephap (a template using LinearMechanism) to which I pass the two sections to be connected. Then it connects a resistor between the respective extracellular nodes.

The problem is that I can't do this in parallel because hosts don't share sections. I've read about how pc.source_var and pc.target_var are used for gap junctions. Could use that to connect one end of the resistor (Ephap._ref_y.x[]) to its corresponding cell. However, they copy v from source onto target, so that seems to be a one-directional coupling to me, which is not what I want. Please let me know if I am wrong.

Is there a way in which I can use some equivalent to pc.gid_connect that can handle a LinearMechanism instead of a synapse? Or else, is there a way in which I can connect (so, sharing the state variable vext, not just copying) two nodes belonging to objects (one cell's Section and one instance of LinearMechanism) from different hosts? I'm suspecting that learning how to program mod files can give me clues about the latter, but not sure if that might be a cul-de-sac.

Any ideas on this will be much appreciated.
mcapllonchj

Re: Parallel simulations of ephaptic connections

Post by mcapllonchj »

Hello,

I would like to rephrase this question in much simpler terms to make things easier. It would be very helpful if someone could provide some hints.
So I have a large resistor network implemented with LinearMechanism (which models the 3D extracellular space of a bundle of axons). For a simulation of a certain size (about 300 axons of 100 nodes each; axons are interconnected with resistors) it takes about 12 hours of computation to run one single time step. So the simulations I intend to run could take a week.
May I have made some mistake, or is this to be expected given the size of the system? Is there a better way I can use to do this?

Many thanks.
ted
Site Admin
Posts: 6287
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: Parallel simulations of ephaptic connections

Post by ted »

Referring back to your post from March 13
I've read about how pc.source_var and pc.target_var are used for gap junctions. . . . they copy v from source onto target, so that seems to be a one-directional coupling to me
which is why two source_var - target_var pairs are used to parallelize gap junction coupling--one pair for each direction.

Now back to your latest question:
For a simulation of a certain size (about 300 axons of 100 nodes each; axons are interconnected with resistors) it takes about 12 hours of computation to run one single time step.
How many couplings exist? 300*299/2?
mcapllonchj

Re: Parallel simulations of ephaptic connections

Post by mcapllonchj »

which is why two source_var - target_var pairs are used to parallelize gap junction coupling--one pair for each direction.
Thanks for that. It was so obvious and still I didn't see it.
There is a point that I'm still not sure if I understand. Is copying the value of a variable, say, vext, from source to target and viceversa, equivalent to sharing the state variable vext at a point between two processors? Am I misled if I think that this method does the former? How are the equations of the system solved when the information is passed in this way?
How many couplings exist? 300*299/2?
Luckily not. Connections are given by a triangulation between axons, which is repeated nseg times (so each compartment adds a new layer of connections), so it's not all with all.
I have run some short tests and I have seen that the time cost not only depends on the total number of connections, but also on the number of axons and nseg. For varying number of axons, the curves time cost vs no. of connections are different.
Also, there's some good news: the large time cost I mentioned was for the first time step, but subsequent time steps don't take so long.
ted
Site Admin
Posts: 6287
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: Parallel simulations of ephaptic connections

Post by ted »

Is copying the value of a variable, say, vext, from source to target and viceversa, equivalent to sharing the state variable vext at a point between two processors? Am I misled if I think that this method does the former? How are the equations of the system solved when the information is passed in this way?
It can affect stability because the system of equations is no longer fully implicit.
How many couplings exist? 300*299/2?
Luckily not. Connections are given by a triangulation between axons, which is repeated nseg times (so each compartment adds a new layer of connections), so it's not all with all.
This I don't understand. Are you representing the extracellular medium as a 3D resistive mesh? That's a lot more expensive than calculating the potential at each point of interest as the weighted sum of membrane currents (where the weights are the transfer impedances between the point of interest and all other current sources and sinks. In a model with N nodes of Ranvier, even if one ignores the internodes, there are N points of interest, and for each point of interest there are N-1 other current sources and sinks (the other nodes of Ranvier). Extracellular potential at node i is
SUMMA zij Ij
i!=j
where i and j lie in the range 1..N,
Ij is the membrane current at the jth node of Ranvier,
and zij is the transfer impedance between the ith and jth nodes of Ranvier.
The impedance matrix Z = |zij| is NxN--symmetric, but still NxN.
there's some good news: the large time cost I mentioned was for the first time step, but subsequent time steps don't take so long.
I can't account for that.
mcapllonchj

Re: Parallel simulations of ephaptic connections

Post by mcapllonchj »

Are you representing the extracellular medium as a 3D resistive mesh?
Yes. The xraxial of every axon accounts for the extracellular space in the longitudinal direction (z-axis, parallel to the axons), while the connections are between the xraxial of neighboring axons, and they are on the x-y plane.

I tried the method you mention before this, but I got numerical instabilities and I didn't know how to fix them. I was using

phi(x) = (1/4*pi*sigma) * summa[over N-1 nodes]{I(node) / dist(x, node)}, (so, the typical for LFPs if I am not mistaken)

for the extracellular potential at a node on x. Is that perhaps not the best way to model the weights?
Even if I re-implemented that using LinearMechanism, which would ensure numerical stability, I guess that would be more expensive since there's more connections.
I can't account for that.
It's odd, yes. But I've seen it happen. I will have a closer look at this.
ted
Site Admin
Posts: 6287
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: Parallel simulations of ephaptic connections

Post by ted »

mcapllonchj wrote: Thu Jun 21, 2018 6:05 am
Are you representing the extracellular medium as a 3D resistive mesh?
Yes. The xraxial of every axon accounts for the extracellular space in the longitudinal direction (z-axis, parallel to the axons), while the connections are between the xraxial of neighboring axons, and they are on the x-y plane.
extracellular is not a complete representation of extracellular space. As noted in the Programmer's Reference, its utility is limited to the following situations:
simulating the stimulation with extracellular electrodes, response in the presence of an extracellular potential boundary condition computed by some external program, leaky patch clamps, incomplete seals in the myelin sheath along with current flow in the space between the myelin and the axon. It is required when connecting LinearMechanism (e.g. a circuit built with the NEURON Main Menu ‣ Build ‣ Linear Circuit) to extracellular nodes.
Which is why I ask: are you assuming that
1. the axon bundle is contained inside a structure that prevents radial spread of current away from the bundle
and
2. only nearest neighbor interactions need to be taken into account
?
mcapllonchj

Re: Parallel simulations of ephaptic connections

Post by mcapllonchj »

The answers to the questions are:
1. No. Radial spread happens through the transverse resistors. And the walls of the bundle have a current path to a 'distant' ground.
2. Yes. Only nearest neighbours are connected using resistors. This doesn't mean, however, that non-nearest neighbours are electrically disconnected, since nearest-neighbour connections form a resistive grid.

extracellular is not used as a complete representation of the extracellular space. The 'complete' representation of it is done by combining extracellular with the transverse (interaxonal) resistors.

Is there another way of communication in which I can send you more details about the model?
Post Reply