Automating PRAXIS: computing error and setting domains?

Using the Multiple Run Fitter, praxis, etc..
Post Reply
vsekulic
Posts: 6
Joined: Sun Dec 26, 2010 7:55 pm

Automating PRAXIS: computing error and setting domains?

Post by vsekulic » Wed May 07, 2014 5:11 pm

Hello!

I'm trying to automate fitting certain parameters in a multicompartment neuronal model by minimizing the error between the model's somatic Vm and experimentally recorded Vm from the same protocol. I previously did this manually using the MultipleRunFitter (MRF) but find the process inefficient and error-prone as I have many variations on the model that I need to fit against several experimental traces. I've since attempted automating my fitting procedure using fit_praxis() and have gotten the basic framework in place. However, I have two issues of which I'm not certain how to address thus necessitating a forum post.

1. How is the "Error Value" computed by the MRF Fitness Generator? I thought it might be least squares error between model and experimental output, so I implemented the following in my function that I pass to fit_praxis() to minimize:

Code: Select all

    // somav, somat are defined elsewhere
    somav = new Vector()
    somav.record(&soma[0].v(0.5))
    somat = new Vector()
    somat.record(&t)

    run()

    err = 0
    for (i=0; i < somat.size(); i=i+1) {

        // Need to do this since have CVode active. Iterate over somatic Vm vector of model, and
        // find corresponding Vm value of experimental trace by referencing the time vector of the model.
        //
        // NB: expv is a 13,000 element Vector corresponding to 1.3s of recording.
        err = err + (expv.x[somat.x[i]*10-1] - somav.x[i])^2
    }
    err = sqrt(err) // cause why not
The error value reported by err is vastly different than the error displayed by the MRF, by an order of magnitude (greater). Thus, I'm curious how the MRF does it.

2. More importantly, how can I set domain bounds for certain parameters in PRAXIS? In the GUI this can be done by clicking on "Parameters" in the MRF, then selecting "Domain Panel". I need this since some of the parameters are coefficients in the time constant equation for a particular ion channel, and cannot take on certain values (such as zero, or negative values). If I don't set the domain, I will eventually get integration errors in NEURON while running PRAXIS in the GUI, and thus want to avoid this situation when fully automating this procedure. I can't see anywhere in the documentation about how to handle this manually. I've seen that they are defined in the corresponding .ft1 session file in a ParmFitness... End ParmFitness block, but don't know what actually uses this or how to do it in an automated fashion without the MRF.

Thanks in advance for any assistance!

ted
Site Admin
Posts: 5376
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: Automating PRAXIS: computing error and setting domains?

Post by ted » Thu May 08, 2014 12:58 pm

vsekulic wrote:1. How is the "Error Value" computed by the MRF Fitness Generator? I thought it might be least squares error between model and experimental output
Not the "least squares error," whatever that means. As per the documentation in the Programmer's Reference http://www.neuron.yale.edu/neuron/stati ... ulfit.html "error value is the square norm between data and dependent variable treated as continuous curves". In other words, it is the sum of the squared errors.

By the way, if your simulations are indeed using adaptive integration, your fits will overemphasize the regions in the waveform where dt is smallest. That's nice if you want the rising phases of spikes to dominate everything else, but not so good if you are concerned about the interspike trajectory when dt tends to be long. There are ways to avoid this problem.
how can I set domain bounds for certain parameters in PRAXIS?
In the GUI this can be done by clicking on "Parameters" in the MRF, then selecting "Domain Panel". I need this since some of the parameters are coefficients in the time constant equation for a particular ion channel, and cannot take on certain values (such as zero, or negative values). If I don't set the domain, I will eventually get integration errors in NEURON while running PRAXIS in the GUI, and thus want to avoid this situation when fully automating this procedure. I can't see anywhere in the documentation about how to handle this manually. I've seen that they are defined in the corresponding .ft1 session file in a ParmFitness... End ParmFitness block, but don't know what actually uses this or how to do it in an automated fashion without the MRF.
Good question. I'm pretty sure how to alter the ft1 file for use with the MRF, but will have to look into what to do about fit_praxis.

hines
Site Admin
Posts: 1528
Joined: Wed May 18, 2005 3:32 pm

Re: Automating PRAXIS: computing error and setting domains?

Post by hines » Thu May 08, 2014 1:49 pm

2)
People generally incorporate constraints into the fitness function itself.eg.
func f() {
if ($1 <= 0) { return 1e6 }
or, if continuity seems useful multiply by a U function
or use a transformed parameter as in ( if arg1 is positive definite)
arg1 = 10^$1
I've found the last to be quite useful as products of weighted powers of parameters are sometimes more meaningful than weighted sums of incommensurable parameters. e.g
R*C is more meaningful than R + C and other reasons as well (convergence is often orders of magnitude faster)

vsekulic
Posts: 6
Joined: Sun Dec 26, 2010 7:55 pm

Re: Automating PRAXIS: computing error and setting domains?

Post by vsekulic » Sat May 10, 2014 12:47 pm

Thank you both for your replies!
ted wrote:Not the "least squares error," whatever that means. As per the documentation in the Programmer's Reference http://www.neuron.yale.edu/neuron/stati ... ulfit.html "error value is the square norm between data and dependent variable treated as continuous curves". In other words, it is the sum of the squared errors.
Ok, great, thanks! Sorry about the terminology; by "least squares error" I meant the sum of squared errors. (The least squares method of course minimizes this quantity.)
ted wrote:By the way, if your simulations are indeed using adaptive integration, your fits will overemphasize the regions in the waveform where dt is smallest. That's nice if you want the rising phases of spikes to dominate everything else, but not so good if you are concerned about the interspike trajectory when dt tends to be long. There are ways to avoid this problem.
Thank you, that's a very good point. Since the parameters we are optimizing have to do with the kinetics of activation for a particular ion channel, it's fine if regions where dt are smallest - being regions of the trace where there is most change (if I understand this correctly) - are overemphasized. In fact, this is exactly what we would prefer in order to make sure that changes in the parameters have noticeable effects on Vm, as captured in the overemphasis of those datapoints ending up in the resulting recorded vector. We have previously been using variable timestep integration with the GUI and have been getting good results. Your point is good to keep in mind when doing further optimizations. Aside from turning off CVode, I'm not sure how we could address the issues you bring up other than a post-hoc "manual" approach - e.g., compressing or removing portions of the recorded Vm that are overemphasized.

ted
Site Admin
Posts: 5376
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: Automating PRAXIS: computing error and setting domains?

Post by ted » Sat May 10, 2014 8:03 pm

Instead of calculating error from the solution after each fadvance, calculate the error from samples of the solution that are taken at regular intervals. Read the Programmer's Reference documentation of the Vector class's record method to see how.

"What if I have experimental data that were sampled at one interval, but for stability or accuracy I'm running simulations at a different interval?"

Then you could resample the original data--see the Vector class's resample method.

Post Reply