Copy a nrn object in Python?

When Python is the interpreter, what is a good
design for the interface to the basic NEURON
concepts.

Moderator: hines

Post Reply
igor
Posts: 5
Joined: Tue Mar 03, 2020 11:33 am
Contact:

Copy a nrn object in Python?

Post by igor »

I need to run some training of networks built for nrn using Python. In particular, at a given moment I need to **deepcopy** an object, already trained up to some moment, to check the outcome of two possible ways of subsequent training, in order to select a better one according to some measure.

The object is a Python built Agent, which is a population of neurons with several cells of different types, synapses, and outputs, with some current states, several described with mod files.

It seemed to me very natural to use a

Code: Select all

new_agent = copy.deepcopy(agent)
Python function from the

Code: Select all

copy
module.

UNFORTUNATELY, this all ends up with (only last lines copied here)
File "/Users/igor/anaconda3/envs/pytorch/lib/python3.7/copy.py", line 280, in _reconstruct
state = deepcopy(state, memo)
File "/Users/igor/anaconda3/envs/pytorch/lib/python3.7/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/Users/igor/anaconda3/envs/pytorch/lib/python3.7/copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/Users/igor/anaconda3/envs/pytorch/lib/python3.7/copy.py", line 169, in deepcopy
rv = reductor(4)
TypeError: can't pickle nrn.Segment objects
Deas it mean that it is not possible to copy whole nrn objects??? I DO NOT want to pickle and serialize the object here for saving it, but only to have a full copy in memory, that would be working, if selected later.

Does any of you know the answer? I would be most grateful in these times of the virus plague... Hope you are well.

/igor

BTW, is there any other thread on the NEURON blog that this post should be put into?
hines
Site Admin
Posts: 1682
Joined: Wed May 18, 2005 3:32 pm

Re: Copy a nrn object in Python?

Post by hines »

You're right, there are no __deepcopy__ method implementations for NEURON objects like sections, segments, mechanisms, point processes, NetCon, or other potentially relevant info such as gid in parallel models. In some cases, a copy is not valid such as for top level hoc Section (constructed by the
hoc "create" statement) as two top level hoc Section cannot have the same name. I think an adequate implementation of deepcopy can be accomplished since for every object mentioned above there is a way to visit all its contents..
ramcdougal
Posts: 267
Joined: Fri Nov 28, 2008 3:38 pm
Location: Yale School of Public Health

Re: Copy a nrn object in Python?

Post by ramcdougal »

If you're running a single process simulation on Linux or Mac, an alternative strategy is to just do an os.fork() once everything has been run for however long, and have each fork test a different variant afterwards.

e.g. You might simulate for a 1000 * ms, then os.fork(), then in one fork simulate as-is for another 1000 * ms. while the other fork changes a parameter before resuming simulation.

This may still require some form of serialization to compare the end results of the two forks.
igor
Posts: 5
Joined: Tue Mar 03, 2020 11:33 am
Contact:

Re: Copy a nrn object in Python?

Post by igor »

Thank you for a prompt answer,

although I am sorry to hear that ;-( I understand, that a workaround I thought about to start with two (or a short list of, I wouldn't need many) identical objects, then try each on an event selecting different action, compare and select the better, and then __copy__ all local parameters that are different from the "better" to the other one, and loop like that, would not be an option? It would, naturally need a careful copying of parameters. Would then the restriction of "two top level hoc Section cannot have the same name" apply too?

Thanks,

/igor
hines wrote: Fri Apr 03, 2020 5:44 pm You're right, there are no __deepcopy__ method implementations for NEURON objects like sections, segments, mechanisms, point processes, NetCon, or other potentially relevant info such as gid in parallel models. In some cases, a copy is not valid such as for top level hoc Section (constructed by the
hoc "create" statement) as two top level hoc Section cannot have the same name. I think an adequate implementation of deepcopy can be accomplished since for every object mentioned above there is a way to visit all its contents..
igor
Posts: 5
Joined: Tue Mar 03, 2020 11:33 am
Contact:

Re: Copy a nrn object in Python?

Post by igor »

Thnx a lot!

Maybe that would be an answer. In our small team I am usin a Mac, a friend is using Linux, thus all shall be well. I really haven’t thought of that even though I am actually a computer scientist. I simply thought that since nrn works in a single thread and Python has a GIL mechanism, that would be really troublesome.

Fortunately, I shall only need to compare some efficiency measures (for now...).

Thank you,
/igor
ramcdougal wrote: Fri Apr 03, 2020 5:56 pm If you're running a single process simulation on Linux or Mac, an alternative strategy is to just do an os.fork() once everything has been run for however long, and have each fork test a different variant afterwards.

e.g. You might simulate for a 1000 * ms, then os.fork(), then in one fork simulate as-is for another 1000 * ms. while the other fork changes a parameter before resuming simulation.

This may still require some form of serialization to compare the end results of the two forks.
ramcdougal
Posts: 267
Joined: Fri Nov 28, 2008 3:38 pm
Location: Yale School of Public Health

Re: Copy a nrn object in Python?

Post by ramcdougal »

Here's a simple example of using os.fork() to do the first part of a simulation once and then explore two different options; this launches a new process not a new thread, so the GIL does not apply:

Code: Select all

from neuron import h
from neuron.units import ms, mV
import matplotlib.pyplot as plt
import os

h.load_file('stdrun.hoc')

soma = h.Section(name='soma')
soma.diam = soma.L = 10
soma.insert('hh')

ic1 = h.IClamp(soma(0.5))
ic1.delay = 5 * ms
ic1.amp = 0.5
ic1.dur = 0.1 * ms

ic2 = h.IClamp(soma(0.5))
ic2.delay = 15 * ms
ic2.dur = 0.1 * ms

t = h.Vector().record(h._ref_t)
v = h.Vector().record(soma(0.5)._ref_v)

h.finitialize(-65 * mV)
h.continuerun(10 * ms)

# everything above here happened once
# we split into two cases here
if os.fork():
    case = 1
    ic2.amp = 0.5
else:
    case = 2
    ic2.amp = 1

h.continuerun(20 * ms)

plt.plot(t, v)
plt.xlabel('t (ms)')
plt.ylabel('v (mV)')
plt.savefig(f'sim{case}.png')
(The above assumes Python 3.6+ and NEURON 7.7+, but can easily be adapted for older versions.)

Incidentally, note that NEURON optionally supports multithreaded simulation and MPI simulation.
Post Reply