Segmenting a simulation using SaveState

Anything that doesn't fit elsewhere.
Post Reply
Prokopiou

Segmenting a simulation using SaveState

Post by Prokopiou » Tue Feb 21, 2012 7:54 am

Hello,

What I've been trying to do is use Matlab to generate stimulus data and NEURON to generate the excitation data. How the "collaboration" between the two programs works is by Matlab generating data, saving them to a stimulus.txt file, NEURON is called, reads the stimulus.txt file, generates an exchitation.txt, then Matlab reads that and moves on with further calculations.
This works perfect for a single piece of stimulus data.

I want now to make this stimulus data into segments so I can process larger chunks of data without having the computer drink up all the RAM and crash.

My thoughts are since I already setup this Matlab calling NEURON to grab its results, what if I store the neuron state and use that to initialize the neuron parameters within NEURON for the next piece of stimulus data what Matlab with deal with.

To avoid the problem of the first set of data and their initialization I have the file called "init.hoc" which is:

Code: Select all

...
proc initialize(){
	t = 0
	finitialize(v_init)
	fcurrent()
}
...
proc integrate(){ 

	while (t<tstop){  
		fadvance()	
	}

}
...
objref svstate
objref f5

proc go(){
	initialize()
	integrate()
	
	
    svstate  = new SaveState()
	f5 = new File()
	f5.wopen("neuron_state.dat")
	svstate.save()
	svstate.fwrite(f5)
	
}
This runs once with the first segment of the stimulus data and then a for loop enters in matlab that calls the "batch_run.hoc" which deals the the rest of the data.
"batch_run.hoc" look like:

Code: Select all

...
objref ldstate
objref f6
ldstate  = new SaveState()
f6 = new File()
f6.ropen("neuron_state.dat")
ldstate.fread(f6, 1)

proc initialize(){
	ldstate.restore()
}
...
proc integrate(){ 

	while (t<tstop){  
		fadvance()	
	}

}
...
objref f5
objref svstate

proc go(){
	initialize()
	integrate()
	
	f5 = new File()
	svstate = new SaveState()
	
	f5.wopen("neuron_state.dat")
	svstate.save()
	svstate.fwrite(f5)
	
}
...

This code doesnt work...

init.hoc works just fine so the saving the state into a file I assume it works ok.
My next step was to try and see if this code:

Code: Select all

proc initialize(){
	ldstate.restore()
}
was messing up by replacing the ldstate.restore() with finitialize() and fcurrent() , which resulted in the same error.

So now I assume the error must be within

Code: Select all

objref ldstate
objref f6
ldstate  = new SaveState()
f6 = new File()
f6.ropen("neuron_state.dat")
ldstate.fread(f6, 1)
and the error I get is a popup window that says:

"Failed assertion
i < prl_->count()
at line 5454 of file
"/home/hines/neuron/nrnrel/src/nrniv/../nrncvode/netcvode.cpp",
in function: PlayRecord* NetCvode::playrec_item(int)
"

which to be honest I don't know how to deal with.

Any help will be greatly appreciated,
Thank you,
Andreas.

ted
Site Admin
Posts: 5566
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: Segmenting a simulation using SaveState

Post by ted » Tue Feb 21, 2012 10:28 am

In order to suggest a solution, I'll have to be able to work with your code and reproduce the problem. If you zip up the necessary hoc, ses, and mod files, plus whatever data files and Matlab source code is relevant, and email it to ted dot carnevale at yale dot edu, I'll let you know what I discover.

Prokopiou

Re: Segmenting a simulation using SaveState

Post by Prokopiou » Fri Feb 24, 2012 8:51 am

Hello Ted,

thank you for your email, I sorted out the code so now it runs as it supposed to.

Some comments though on the operation of the code if anyone else wants to do something similar.

First of all my code which runs in hoc:

Code: Select all

/*=======================================*/
/*  Simulation Control                  */
/*=======================================*/


objref f1
f1 = new File()
f1.ropen("simulation_data.txt")

dt = f1.scanvar()
tstart = f1.scanvar()
tstop = f1.scanvar()

v_init = -65  // this could go into matlab
 
objref ldstate
objref f6

proc initialize() {
  finitialize(v_init)
  
  
  if (firstrun==0) { // this is not the first run in a series
    
	
	ldstate  = new SaveState()
	f6 = new File()
	f6.ropen("neuron_state.dat")
	ldstate.fread(f6, 1)
	
	ldstate.restore(1) // don't clear the event queue
    t = tstart // t is one of the "states"
    
	if (cvode.active()) {
		cvode.re_init()
		
		print "cvcode is active"
	  
		} else {
		
		print "cvcode is not active"
		fcurrent()
	}
    frecord_init()
  }
}

...

proc integrate(){ 

	while (t<tstop){  
		fadvance()	
	}

}

objref f5
objref svstate

proc go(){
	initialize()
	integrate()
	
	f5 = new File()
	svstate = new SaveState()
	
	f5.wopen("neuron_state.dat")
	svstate.save()
	svstate.fwrite(f5)
	
}

...
This recieves input from matlab which specifies the dt, tstart and tstop of the simulation.

One thing that I have noticed on the output when segmenting is that there was a big problem (possibly a bug, or something i did wrong) when I was changing segment.

The problem was that the last state was saved as expected and when it loaded it made a copy of the state on the first value of the new segment. In other words if the segment was 200 samples then sample 200 and sample 201 were identical!

What I did to resolve is it to give an extra dt in the tstop and make my stimulating voltage an extra copy of the last time frame. This was just for the time data and stimulus data to match size. Then I saved all returned results except the last one which I ignored since it reappeared on the next run.
In other words I calculated the values of the segment plus the next time instance of NEURON and let that the one to be saved by SaveState and didnt record it with matlab, so when it reloaded in the next run it was in the correct time instance when the result was recorded later.

It took me some time to realise this was going on.
I hope this makes sense, and possible help someone someday.
Andreas.

ted
Site Admin
Posts: 5566
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: Segmenting a simulation using SaveState

Post by ted » Sat Feb 25, 2012 12:59 am

Thanks for sharing your solution.
One thing that I have noticed on the output when segmenting is that there was a big problem (possibly a bug, or something i did wrong) when I was changing segment.

The problem was that the last state was saved as expected and when it loaded it made a copy of the state on the first value of the new segment. In other words if the segment was 200 samples then sample 200 and sample 201 were identical!
Strange. What heppens if the argument is 2 or 3?

Prokopiou

Re: Segmenting a simulation using SaveState

Post by Prokopiou » Wed Mar 07, 2012 9:35 am

Hello Ted,

apologies for the late reply, I moved on with the code and I needed to finish before I broke it back to what it was before to check this...

so I have set the segment to be of size 2 and I observed this:

Assume the correct signal is 1 2 3 4 5 6 7 8 9 10 with the size 2 block process the output given was 1 2 2 4 4 6 6 8 8 10 so the first copied the last. The resulting plot showed the signal behaving as it should only more jagged looking.

So then I set the size of the block to 1 to see what happens and as expected the neuronal response remained at v_init.

This is easy to overcome though by what I did that was to calculate the next state and save it into the file and just not record its value, and when the next block came in it copied the correct value.

Andreas.

ted
Site Admin
Posts: 5566
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: Segmenting a simulation using SaveState

Post by ted » Wed Mar 07, 2012 2:13 pm

I'm sorry, I just can't visualize what you're describing. I will have to see the phenomenon for myself. Can you zip up the code necessary to reproduce this and email it to me?

Addendum: got it, thanks!

ted
Site Admin
Posts: 5566
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: Segmenting a simulation using SaveState

Post by ted » Wed Apr 25, 2012 11:07 am

There is another strategy avoids SaveSession altogether: at the end of each simulation segment, just
1. write data to output file(s)
2. resize the "recording" Vector(s) to 0
3. continuerun(t + SEGMENT_DURATION)
Examples are provided here http://www.neuron.yale.edu/neuron/node/70

This has the advantage of being a bit easier to implement, but of course if you ever decide "enough for now, tomorrow we'll start up where we left off," you have to do a SaveSession.

Post Reply