help with scanf

[blhuynh]

I have a script that seems to be working except for trying to load in a file to use. I tried to resolved the problem by just isolating the code that was giving me an error and is listed below.

Code: Select all

noaxons = 300
total = 1450
pol = -1

strdef volfile, recfile
volfile = "vol_testfile.txt"
recfile = "act_testfile.txt"

objref file
objref ve_data

proc get_extpot () {
		
	file = new File(volfile)
	file.ropen()

	ve_data = new Matrix()
	ve_data.scanf(file,noaxons,total)
	ve_data.muls(pol)
	file.close()

}
get_extpot()

The text file vol_testfile.txt is a matlab generated .txt file that is 300x1450 that is delimited by spaces in each line, and one row per line. It works if I have noaxons = 100 but not if I have noaxons = 300. It also works if I change ve_data.scanf(file,noaxons,total) to ve_data.scanf(file) but then I'm afraid that I'm not loading the data correctly.

Code: Select all

NEURON -- Release 7.1 (359:7f113b76a94b) 2009-10-26
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2008
See http://www.neuron.yale.edu/credits.html

loading membrane mechanisms from nrnmech.dll
Additional mechanisms from files
 AXNODE75.mod PARAK75.mod ampa.mod gabaa.mod gabab.mod ih.mod ik2.mod isikdr.mod isina.mod it.mod leak.mod nmda.mod sw.mod train.mod
nrniv: EOF in fscan
 in C:/Users/Brian Huynh/Desktop/sweeneyaxon_model/LocalNeurons/test_scanf.hoc near line 22
 get_extpot()
             ^
Matrix[0].scanf(File[0]3001450        , , )
get_extpot(      )
oc>

Any insight into how to fix this would be greatly appreciated. Thanks.

[ted]

With NEURON 7.3 your code has no problem reading a 300 row x 1450 column file, which I generated with this code:

Code: Select all

load_file("stdgui.hoc")
NROW = 300
NCOL = 1450
objref m, vec
m = new Matrix(NROW, NCOL)
proc fillm() { local i  localobj vec
  for i=0,NROW-1 {
    vec = new Vector(NCOL)
    vec.indgen.add(1).mul(0.1+i)
    m.setrow(i, vec)
  }
}
fillm()
// m.printf()
objref fil
fil = new File("data.txt")
fil.wopen()
m.fprint(0, fil)
fil.close()

(you'll want to change data.txt to whatever name you prefer). I was frankly surprised at the speed of data generation and file i/o.

Suggest you uninstall 7.1 (if you're using MSWin, first be sure to use the Uninstaller in the NEURON program group, and eradicate any C:\nrn* that may remain), then install the latest 7.3.

[blhuynh]

I've installed NEURON 7.3 and I can get it to work fine using the 300 row x 1450 column file generated from your code. I see that your file is delimited by varying numbers of spaces. My file is just delimited by single spaces. Would that make a difference?

[ted]

Shouldn't make a bit of difference--whitespace is whitespace, as far as scanf is concerned. But just to be sure, try with your own data file.

[blhuynh]

Sorry, I meant to include that my file isn't working still in scanf. Is there something regarding the delimiting that I should look further into? Also, I can load my file with noaxons= 1 to 299 but as soon as I set noaxons=300 and try to load 300x1450, it doesn't work.

[ted]

Very tricky problem indeed.

The problem lies in your vol_testfile.txt. 108 lines of that file contain at least one instance of NaN. That's more than enough to make the attempted read of the 300th row fail with an eof error, but not enough to cause failure at the 299th row. The Matrix class's scanf() method tries to read nrow*ncol numerical values from a file. The actual formatting of the file doesn't matter. To quote from the documentation of the Matrix class's scanf() method:

Strings in the file that cannot be parsed as numbers are ignored.

[blhuynh]

Thank you very much. I didn't notice the NaNs and I didn't realize that they would error out on NEURON. I appreciate your time and help.

[ted]

The NaNs don't "error out". They're not numbers, so scanf skips over them. The entire file is read, right down to the end, and NEURON reports the eof as it should. The advantage of the Matrix class's scanf is that it's fast, but it's fast because it expects to find what it was told would be there. With hoc you can also construct a procedure that reads a file that mixes numerical data and NaN entries and takes whatever action you want it to when it encounters an NaN, but it would execute more slowly.

Other alternatives:
You could pass your data file through an awk or sed filter that replaces every NaN with 0 or some other appropriate numerical value, or do the same via global serch and replace with a text editor.
You could revise the program that generates the data file so that it writes an appropriate numerical value instead of NaNs.

[ted]

Other alternatives:

Actually the first step is to decide whether the NaNs are benign or signs of a serious problem in the data file itself. NaN occurs 750 times in that file, and 108 of the file's 300 lines contain at least one instance of NaN. Whatever generated the file may be afflicted with the worst kind of bug: the kind that allows a program to execute to completion without notifying the user that there is a problem. You might also want to examine other related data files to see if they too contain NaNs and decide what further action may be necessary.