neuromau wrote:Testing the installation and running job scripts are covered in the reply to this post.
To test the installation of NEURON, we can create the following test program.
1. Move to the WORK file system:
2. Create a directory called test:
3. Move to the test directory:
4. In the test directory, create a file called test0.hoc (for example,
), and include the following text in the file:
Code: Select all
objref pc
pc = new ParallelContext()
{printf("I am %d of %d\n", pc.id, pc.nhost)}
{pc.barrier()}
quit()
Note that, unlike with parallel code on other supercomputers, we are ending our program with different commands. Stampede may hang on the "pc.runworker" call, so we go straight to a "quit()" call instead (which will work if you have installed the latest development version of NEURON as advised in the post above). However, if one processor calls the "quit()" command before the other processors have reached that point in the code, unexpected behavior may result. Therefore, prior to the "quit()" call, we have added a "pc.barrier()" call to ensure all processors have reached the point where they are ready to quit.
5. In the test directory, create a mod file called test.mod, and include the following text in the file:
6. Compile the test mechanism:
7. Create a job script called
mytest.sh that contains the following code:
Code: Select all
#!/bin/bash
#SBATCH -J myMPItest # name of job
#SBATCH -o myMPItest.%j.o # output file (errors can be placed in this file, too). %j expands to the job id number.
#SBATCH -n 16 # number of cores
#SBATCH -p development # queue to use
#SBATCH -t 00:10:00 # time limit
#SBATCH --mail-user=yourname@school.edu # where to send notifications
#SBATCH --mail-type=ALL # types of mail notifications to receive
ibrun tacc_affinity x86_64/special -mpi test0.hoc
Details about the job script format are included in the
Stampede User Guide. Note the inclusion of 'tacc_affinity' in the ibrun command. Without this option, the '-mpi' flag may not work.
8. After creating this job script, we must submit it. To submit it to the queue, we enter:
sbatch mytest.sh
The batch queue software will perform a series of checks before assigning our job a run number, adding our job to the queue, and printing a confirmation of submission:
Submitted batch job 461814
9. We can check on the status of our job using the following command:
Note that the Stampede User Guide also recommends the
showq function, but this function is currently buggy and may return other users' jobs as well. Therefore,
squeue is preferred.
10. Once the job is complete, an output file called
myMPItest.461814.o is generated. Note that we specified the name of the output file in our submission script, including
%j where the job id number generated by the queuing system was inserted. When we open this file, we see the following:
+ ibrun tacc_affinity x86_64/special -mpi test0.hoc
TACC: Starting up job 461814
TACC: Starting parallel tasks...
numprocs=16
NEURON -- VERSION 7.3 (817:7da2b70c0aed) 2013-03-20
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2012
See
http://www.neuron.yale.edu/credits.html
Additional mechanisms from files
test.mod
I am 10 of 16
I am 3 of 16
I am 5 of 16
I am 0 of 16
I am 1 of 16
I am 4 of 16
I am 6 of 16
I am 7 of 16
I am 14 of 16
I am 13 of 16
I am 8 of 16
I am 11 of 16
I am 9 of 16
I am 2 of 16
I am 15 of 16
I am 12 of 16
TACC: Shutdown complete. Exiting.