A ring network with multithreading

General issues of interest both for network and
individual cell parallelization.

Moderator: hines

ted
Site Admin
Posts: 5334
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: A ring network with multithreading

Post by ted » Thu Jul 09, 2015 5:14 pm

shailesh wrote:How could I make use of multi-threading here?
Unless the individual cells are very large (at least about 5000 states/cell), there probably wouldn't be a noticeable speedup, so this is more of a theoretical than a practical question.
on my desktop, which has 4 cores, I get pc.nhost = 1.
But on NSG with 2 nodes and 4 cores/node, I get pc.nhost = 8. Why so?
Sounds like your desktop machine doesn't have MPI installed.

With regard to the number of processors that actually exist on your desktop machine--what happens if you run neurondemo, select the pyramidal cell model, then click on
Tools / NEURON Main Menu / Parallel Computing
and finally click on the ParallelComputeTool's "Refresh" button? Specifically,how many processors does it find?
(reported in line at the top of the ParallelComputeTool).

shailesh
Posts: 104
Joined: Thu Mar 10, 2011 12:11 am

Re: A ring network with multithreading

Post by shailesh » Thu Jul 09, 2015 11:12 pm

I did that and found something interesting:
> On Windows 7 64-bit with NEURON v7.4 64-bit, I got "4 useful processors"
> On Ubuntu 14.04 64-bit with NEURON v7.4 64-bit, I got "8 useful processors"

My desktop has dual OS, and so the above was observed on exactly the same hardware. My processor is Intel® Core™ i7-2600K which has 4 cores. Why do we see the above difference?

shailesh
Posts: 104
Joined: Thu Mar 10, 2011 12:11 am

Re: A ring network with multithreading

Post by shailesh » Fri Jul 10, 2015 7:23 am

I may have some leads (and further issues) on the matter...
My processor, Intel® Core™ i7-2600K, has 4 cores and supports 8 threads with hyper-threading. So the OS considers it as 8 CPUs (as verified in the 'Task Manager' in Windows and 'About This Computer' in Ubuntu.
Further, on running a network model (parallel enabled, tested on NSG), I find that, for the same model:
> In Windows, the task manager shows 4/8 CPUs busy
> In Ubuntu, the system monitor shows only 2/8 CPUs busy

Not sure what to make of it... (and also from the previous post: Why the difference in available number of useful processors?)

ted
Site Admin
Posts: 5334
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: A ring network with multithreading

Post by ted » Fri Jul 10, 2015 2:10 pm

Your CPU really has only 4 cores. Wikipedia has an entry on hyperthreading that you should read. Bottom line is that hyperthreading refers to technology that is useful for task switching in a graphical environment when many application windows are open. That's great if you're juggling Word, Excel, Powerpoint, a web browser etc. at the same time, but irrelevant for numerical simulations with NEURON or any other simulation environment.

NEURON's ParallelComputeTool uses a brute force approach to discover how many "real" cores exist--repeatedly runs a program that eats CPU cycles with nthreads = 1, 2, etc. until it finds the number of threads at which further increases in nthreads fails to shorten run time. The typical user's PC or Mac is doing a lot of other stuff all the time, so the test run times have some jitter from trial to trial. The jitter is manifest by fluctuations in the reported "number of useful processors." A single trial, launched by a single click on the Refresh button, may produce a result that is smaller or even a bit larger than the actual number of cores, but most often it produces the correct result. It may be necessary to

Code: Select all

REPEAT
  click Refresh
  wait for a result
UNTIL you see what "number of useful processors" is most often reported
That said, some OSes on some machines apppear to be prone to reporting an incorrect number of processors; as I recall, some versions of Ubuntu seemed likely to do that.

shailesh
Posts: 104
Joined: Thu Mar 10, 2011 12:11 am

Re: A ring network with multithreading

Post by shailesh » Fri Jul 10, 2015 10:52 pm

I actually had read an earlier post by you on the forum regarding this and so did the 'Refresh' several times and reported the most frequent value. I suppose it is do with the OS as you suggested.

Any suggestion regarding the earlier question:
A part of the confusion stems from the fact that on my desktop, which has 4 cores, I get pc.nhost = 1.
But on NSG with 2 nodes and 4 cores/node, I get pc.nhost = 8. Why so?
This despite NEURON reporting 4 useful processors on my desktop. Does that not suggest that NEURON is able to utilize all 4 cores (and that the system has MPI installed)? I certainly do get a performance increase when moving from 1 thread to 4 threads.

ted
Site Admin
Posts: 5334
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: A ring network with multithreading

Post by ted » Sun Jul 12, 2015 12:04 pm

A part of the confusion stems from the fact that on my desktop, which has 4 cores, I get pc.nhost = 1.
But on NSG with 2 nodes and 4 cores/node, I get pc.nhost = 8. Why so?
What was the command line that you used to launch NEURON on your desktop machine? If it specified one process, then pc.nhost will return 1.

shailesh
Posts: 104
Joined: Thu Mar 10, 2011 12:11 am

Re: A ring network with multithreading

Post by shailesh » Sun Jul 12, 2015 1:15 pm

I generally run NEURON on Windows 7, and so its mostly by just double-clicking the hoc file. While checking on Ubuntu the command was:

Code: Select all

nrngui init.hoc
In both cases I got pc.nhost = 1. I suppose one does not really require the GUI when handling a simulation over multiple hosts, but I am not too familiar on other ways to launch NEURON.
If it specified one process, then pc.nhost will return 1.
Where and how do we specify the number of processes?

ted
Site Admin
Posts: 5334
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: A ring network with multithreading

Post by ted » Sun Jul 12, 2015 2:53 pm

To execute a simulation of a distributed model on parallel hardware it is best to use mpiexec. This is done from the command line with the following syntax:
mpiexec -n N nrniv -mpi yourcode.hoc
or
mpiexec -np N nrniv -mpi yourcode.hoc
where
N is the number of processes ("hosts")
and
yourcode.hoc
is the main file of your program. The Programmers' Reference contains examples of this on the page devoted to ParallelContext. More examples are in
Hines, M.L. and Carnevale, N.T.
Translating network models to parallel hardware in NEURON.
J. Neurosci. Methods 169:425-455, 2008
which is available as a preprint from the "Publications about NEURON" page at NEURON's web site.
If the main file is written in python, the command is
mpiexec -n N nrniv -mpi -python yourcode.py

Note: in case you specify an absurdly large value for N, and your desktop computer executes the code without complaining, it doesn't follow that you actually have N "cores" or "processing units" in your desktop machine. It just means that the model will be distributed over N processes, and that in the course of simulation some of these processes will have to wait for others to reach a "spike exchange time" before they can be executed.

shailesh
Posts: 104
Joined: Thu Mar 10, 2011 12:11 am

Re: A ring network with multithreading

Post by shailesh » Mon Jul 13, 2015 4:50 am

Thanks! That works now (on Ubuntu). pc.nhost gives what is expected and also an improvement in the performance.
How should I go about trying to do this on Windows? I read a few of the other posts here (suggesting commands involving bash, nrniv etc... viewtopic.php?f=5&t=1380) but it didn't do the trick.

ted
Site Admin
Posts: 5334
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: A ring network with multithreading

Post by ted » Mon Jul 13, 2015 11:25 am

In MSWin, use the NEURON Program Group's bash icon to launch a bash terminal. Then proceed as you would have under Linux (cd to wherever your model files are located etc.).

shailesh
Posts: 104
Joined: Thu Mar 10, 2011 12:11 am

Re: A ring network with multithreading

Post by shailesh » Tue Jul 14, 2015 5:58 am

It's giving me:

Code: Select all

bash.exe"-3.1$ mpiexec -n 4 nrniv -mpi init.hoc
CreateProcess failed, error 2
SMPD manager launch request failed with error 0x80070002.
Error: unable to start the local smpd manager.
I tried installing MPICH2 (wondering if that was the issue, as suggested somewhere online) by following the instructions here: http://swash.sourceforge.net/online_doc ... node9.html. But even then I am getting the same error message. Anything else I could try out?

ted
Site Admin
Posts: 5334
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Re: A ring network with multithreading

Post by ted » Tue Jul 14, 2015 12:39 pm

Suggest uninstalling anything that has to do with MPICH, and NEURON too.
Then get the latest MPI installer from Microsoft, as referenced here
http://www.mpich.org/downloads/
and install that.
Finally get the latest 7.4 mingw alpha installer for MSWin from here
http://www.neuron.yale.edu/ftp/neuron/versions/alpha/
and install that.

Post Reply