Model (hoc file), plotting output in Python

[vladimirov]

Hi,
Could you please give a hint on how to run existing hoc file which includes some GUI (ses-file) by using Python? I have PyNEURON installed and run my hoc file with

Code: Select all

from neuron import *
h('load_file("mosinit.hoc")')

I get the following errors:

Code: Select all

In [6]: h('load_file("mosinit.hoc")')
NEURON: Couldn't find: nrngui.hoc
 in mosinit.hoc near line 1
 load_file("nrngui.hoc")
                        ^
        0
NEURON: lambda_f not declared at the top level
 in CA1template.hoc near line 212
 external lambda_f
                 ^
        xopen("CA1template.hoc")
      execute1("{xopen("CA1template.hoc")}")
    load_file("CA1template.hoc")
  xopen("mosinit.hoc")
and others
        0
NEURON: syntax error
 in CA1template.hoc near line 214
   forsec all { nseg = int((L/(0.1*lambda_f(100))+.9)/2)*2 + 1  }
            ^
        xopen("mosinit.hoc")
      execute1("{xopen("mosinit.hoc")}")
    load_file("mosinit.hoc")
        0
Out[6]: 1

My hoc file is pretty standard, includes

Code: Select all

load_file("nrngui.hoc")
load_file("CA1template.hoc")

and so on, with

Code: Select all

load_file("session.ses")

in the end.
I basically need Python to plot the output of NEURON code, and run/control simulations on the top level. How do I do it right?
Thank you!

[uri.cohen]

PyNEURON (installed from pypi or bitbucket) does not include the NEURON GUI and hence cannot be used to visualize cells of plot simulation result.
If you're dealing with legacy code, I would recommend getting a standard installation of NEURON and using it without Python for the tasks which require GUI.
If you're looking at the long-term, I would recommend replacing the hoc code which deals with GUI with python one. You may look at neuronvisio from http://michelemattioni.me/neuronvisio/ as a GUI for NEURON.

This is not to say that you need to get rid of all hoc code- rather I would leave all the simulation related code and just invoke it after setting the needed parameters from python.

[vladimirov]

Thank you, Uri. I will try your long-term advice and get rid of hoc GUI but retain the hoc computational part.

[ted]

It may be fine for those who want to work entirely in Python to abandon NEURON's Interviews-implemented GUI, but this has its own costs and hazards. The costs are numerous and significant: it means giving up powerful tools and functionality, such as the CellBuilder, Channel Builder, Linear Circuit Builder, and Model View., which users will find very difficult to replicate on their own. And even hoc's lowly Graph class has more useful features than any graphs I have yet seen that were generated by users working entirely with Python. The principal hazard is breaking a model specification that relies on one of the model-specification GUI tools such as CellBuilder, Channel Builder, Linear Circuit Builder. Finally, far more published NEURON models have been implemented with hoc than with Python. For the near and intermediate term, trying to purge GUI-related hoc code is going to divert user time and effort from productive pursuit of research issues to less-productive code wrangling and "reinvention of the wheel."

My Python expertise is still rather limited, but the fact that anything in hoc can be accessed from Python and vice-versa suggests to me that it should be possible for a Python user to be able to load and run existing hoc code, with little or no modification, and take advantage of whatever InterViews-library-dependent GUI elements that code may bring up. I discussed this with Michael Hines, who replied:

I was unaware that the PyNeuron package had left out InterViews and, apparently, even the nrn/lib/hoc standard hoc lilbrary files. It is true that InterViews has not been available under mswin except as a cygwin version and therefore was not available in conjunction with enthought python. Also, as a cygwin program NEURON could not be built as a 64bit program on WIn7. However, InterViews now builds as a mingw library and so is usable with enthought. Also everything builds as a 64bit program. So I don't think it is necessary to abandon the existing GUI just yet.
. . .
Linux and Mac all nicely allow enthought, python, InterViews, NEURON to play together in the 32 and 64 bit worlds. There are just a few loose ends to tie up so that is also the case for mswin.

The specific problems with mswin are:

. . . I am trying to fix a bug with the mingw64 port. Also both 32 and 64 bit versions have a problem with 'import numpy', apparently due to a version conflict between some mswin library used by enthought and my mingw installation.

[uri.cohen]

Ted, I totally agree with you. I'm not blindly pushing Python to users and openly admit it's current limitations- having only a modest toolset (implemented mostly in neuronvisio) compared to the richness of the NEURON tools. Actually, including the NEURON GUI is the only feature-request I got so far (other than adding MAC and Linux support).

[vladimirov]

From my user's perspective, I am going to stay mainly in hoc because, as Ted pointed out, native NEURON GUI is richer and more versatile than python substitutes I found so far. But I don't see any convenient way to generate publishing-quality plots using NEURON GUI, except pick vector->save in txt file ->import it in Matlab. So I want to add up little python to plot and export the figures in a fast and convenient way. I think that will pay off in the long-term, by saving time for plotting routines.

[ted]

Thanks, Uri, for what you have been doing to help make it easier for users to get started using NEURON via Python. The reason for my previous message was that, as moderator of the Forum, I am always reminded of the fact that these threads are read by users who have a wide range of expertise. This means it is sometimes necessary to insert caveats or try to address potential misunderstandings.

All the problems and limitations and peculiarities discussed in this thread and others will eventually go away, of course. In particular, it would be very surprising if, in the next few years, we don't start seeing pure Python GUI tools that are far more powerful than anything that already exists in either hoc or Python. And the more that people use NEURON via Python and bring their own expertise to bear on these problems, the sooner this will happen.

[ted]

I don't see any convenient way to generate publishing-quality plots using NEURON GUI, except pick vector->save in txt file ->import it in Matlab.

Many use this workflow:
Vector record, then write data to text files, then plot with gnuplot or Sigmaplot or whatever.

I prefer to generate plots with NEURON, using the Vector and Graph classes, then use the Print & File Window Manager to save as idraw files. idraw files are basically encapsulated PostScript, but structured in such a way as to be readable by vector graphics programs such as Corel Draw and Adobe Illustrator. Long ago I used Corel Draw, but for the past decade have stuck with idraw, which is a simple/puny vector graphics program that comes with Interviews, i.e. is part of a full UNIX/OS X/Linux installation of NEURON. I haven't found OpenOffice Draw to be useful for editing idraw files. A couple of years ago I tried using ps2ai to convert idraw files to Adobe Illustrator format, then editing the result with Inkscape, but without much success--the conversion seemed to break things, and the resulting file gagged Inkscape. Maybe that was an Inkscape problem and I should try again . . .

For many of the figures in the NEURON Book, I cobbled up a little hoc GUI tool that combined a Graph plus a few buttons that facilitated common tasks such as changing line styles, size and style of symbols etc.. I used "Vector pick" to nab data from ordinary graphs, then pasted the data into the GUI tool's Graph, applied a few customizations, and saved the result to an idraw file for any additional tweaking that might be necessary (e.g. text alignment, arrows).

[mattions]

just want to highlight two points for future reference:

• - the url for Neuronvisio is http://neuronvisio.org
  - Neuronvisio does not exclude the ability to NEURON gui at the same time you use Neuronvisio.

infact it is possible to load Neuronvisio and Neuron gui at the same time with

Code: Select all

from neuron import gui
fron neuronvisio.controls import Controls
controls = Controls()

On top of that, Neuronvisio does not try to re-implement what it has already being written in NEURON, but it extends it and cover some functionality, like 3D plotting, that were not available in NEURON.

[ted]

cover some functionality, like 3D plotting, that were not available in NEURON.

Just to make sure there's no misunderstanding, could you please explain exactly what you mean by "3D plotting" and how this differs from the capabilities of NEURON's shape plots? For example, your recent article made a similar claim but presented only a 2D projection of a neuron--what made that a 3D plot?

[mattions]

Hi Ted,

as I said I wrote Neuronvisio during my PhD because I had to deal with very tiny spines, and I wanted to see exactly where they were placed. The shape plot from NEURON was able to give a good rough idea, but I need the ability to click on a section to inspect what was the name and the conductances and other interesting feature that were loaded http://neuronvisio.org/_images/spines_detailed.png. Therefore I wrote some 3D rendering with those reqs, on top of integrating with matplotlib and having a way to save simulation results which was compact and flexible. I thought it was kind of useful for other people as well, so I just packaged and made it available to download and install. At that point Uri came along when it was writing a program to download and load the model from ModelDB. Neuronvisio was able to load a downloaded model, and his code was able to fetch the model from modelDB and expose interesting information to the user. So we decided to integrate it in Neuronvisio, and add the modelDB ability to it.

To come back on the 3D rendering, Neuronvisio is built on top of Mayavi, where the zooming and panning of the objects are a bit more immediate than the shape plot 3D from NEURON.

Maybe it's easier to appreciate why we claim it's a 3D plot https://www.dropbox.com/s/39o97g918mgel ... nvisio.png, where the depth of the image are better rendered (using shades and lights set-up by the VTK/Mayavi env.)

As I said, Neuronvisio is a NEURON companion, and it helps a little bit to get the work done =)

[ted]

Could you please tell me if this summary is correct.

The similarities between images of cells and nets produced by Neurovisio and NEURON's shape plots are that both can generate images that
--are 2D projections of 3D structures
--are rotated, translated, and zoomed so that even individual spines are visible
--display neurite diameters and lengths in correct relative proportion
--use color to represent the value of membrane potential with a user-specified color scale

The differences are that
--Neurovisio images include nice rendering of surfaces with shading, but shape plots do not
--in a Neurovisio image, one can click on an individual section to discover its properties. This is not a general feature of InterViews shape plots, but is limited to the Section Parameters tool, and the CellBuilder has a similar capability.

[mattions]

I guess it's mostly correct.

One more diff is that in Neuronvisio the zooming is controlled with the wheel of the mouse, in the shape plot the user needs to select it.
Another one is that Shape Plot is a lot faster than Neuronvisio, however the rendering is not as nice as Neuronvisio.

[ted]

Shape Plot is a lot faster than Neuronvisio, however the rendering is not as nice as Neuronvisio.

"Fast and ugly" sounds more like a bug than a feature. Reminds me of the old joke where a soldier tells his general "The bad news is we have nothing left to eat but ________" (fill in with something unappetizing). The good news? "We have plenty of it."

Unless the speed difference is so great that Neuronvisio is impractical to use, nobody really wants ugly. Also, its shaded renderings are clearly very useful for complex images, especially networks with a lot of overlapping cells.

[ted]

I guess it's mostly correct.

So I might have omitted some details of operational or performance differences, like how to zoom or the speed of rendering. But is there any substantive error in my summary comparison, especially anything that is actually false or would mislead a reader regarding Neuronvisio's features? My knowledge about Neuronvisio is entirely from what I saw in your talk in 2010 in Tempe, AZ, and your recent article, so I want to make sure that my understanding is clear.