Splitting on a single machine.

[Krishna Chaitanya]

Hi,

I have a very basic question on splitting a group of cells over 'n' processes on a single machine. I have 4000 cells which were distributed over 3 processes. I have used the usual syntax "for (i=pc.id; i<$1; i+=pc.nhost)". And the gid is assigned with "pc.setgid2node(i, pc.id)" to that process.

When I use this print statement "{printf("I am %d of %d and have %d cells\n", pc.id, pc.nhost, cells.count())}". It shows me that "I am 0 of 3 and have 1334 cells". Does this mean that only 1334 cells were assigned gid and created because it is done on a single machine?

I am testing this on 8 core machine with 8GB RAM. What about the remaining cells? If they are created and distributed, how do I know through printf statements?

Kindly let me know.

Thank you.

[ted]

When I use this print statement "{printf("I am %d of %d and have %d cells\n", pc.id, pc.nhost, cells.count())}". It shows me that "I am 0 of 3 and have 1334 cells". Does this mean that only 1334 cells were assigned gid and created because it is done on a single machine?

It means only that the printf statement was executed by only one host: host 0. Examine the code that surrounds the statement and see if you can revise it to iterate over each host, one at a time.

[ted]

You might find it helpful to check out the code from
Hines, M.L. and Carnevale, N.T.
Translating network models to parallel hardware in NEURON.
J. Neurosci. Methods 169:425-455, 2008.
The PDF is available from http://www.neuron.yale.edu/neuron/nrnpubs, and the code is available from ModelDB via accession number 96444.

In particular you might be interested in this bit

Code: Select all

// reports distribution of cells across hosts
proc report_gidvecs() { local i, rank
  pc.barrier()  // wait for all hosts to get to this point
  if (pc.id==0) printf("\ngidvecs on the various hosts\n")
  for rank=0, pc.nhost-1 {  // host 0 first, then 1, 2, etc.
    if (rank==pc.id) {
      print "host ", pc.id
      gidvec.printf()
    }
    pc.barrier()  // wait for all hosts to get to this point
  }
}

from ran3par.hoc



from

[Krishna Chaitanya]

Dear Ted,

Thank you very much for the answer. Another problem I have is with pc.gid2cell. I have gone through the reference you quoted and found that for pc.gid2cell to work, it needs connect2target function. If I don't have connect2target function in my cell template, it is giving me this error:

1 /home/krishna/nrn-7.3_installed/x86_64/bin/nrniv: vv :not right number of subscripts

Is there any way I can bypass the usage of connect2target and still use pc.gid2cell? Please let me know.

Thank you.

[ted]

If I don't have connect2target function in my cell template, it is giving me this error:
. . .
Is there any way I can bypass the usage of connect2target and still use pc.gid2cell?

Association of a cell with its gid is best done at the time that the cell is created. An example of a convenient idiom for doing this is provided by proc mkcells() in our JNM paper's ring network example:

Code: Select all

// creates the cells and appends them to a List called cells
// argument is the number of cells to be created
proc mkcells() {local i  localobj cell, nc, nil
  cells = new List()
  // each host gets every nhost'th cell,
  // starting from the id of the host
  // and continuing until all cells have been dealt out
  for (i=pc.id; i < $1; i += pc.nhost) {
    cell = new B_BallStick()
    cells.append(cell)
    pc.set_gid2node(i, pc.id)  // associate gid i with this host
    nc = cell.connect2target(nil)  // attach spike detector to cell
    pc.cell(i, nc)  // associate gid i with spike detector
  }
}

I strongly suspect that ParallelContext's cell() method sets up a "table" of
gidnumber, netconobjref
pairs, and that its gid2cell method works by looking in this table to find the netconobjref that corresponds to a user-provided gid number, then uses that NetCon's precell() method to return an objef that points to the cell to which the NetCon's spike detector was attached. If that is the case, then all you have to do is make sure that when you create each cell instance for your network, you also attach a NetCon to it and then use ParallelContext's cell() mehod to associate the gid with the NetCon's spike detector. The for loop that creates the cell instances and associates each cell with its host and gid becomes

Code: Select all

  for (i=pc.id; i < $1; i += pc.nhost) {
    cell = new B_BallStick()
    cells.append(cell)
    pc.set_gid2node(i, pc.id)  // associate gid i with this host
//    nc = cell.connect2target(nil)  // attach spike detector to cell
    // assumes that cell has a section called soma
    cell.soma nc = new NetCon(&v(0.5), nil)  // attach spike detector to cell
    pc.cell(i, nc)  // associate gid i with spike detector
  }