Data Processing

Last week was spent at work collecting data with the ECTDL (External Cavity Tunable Diode Laser). I've collected about 250 or so sets of data, each spanning about 1.1 cm^-1 (A unit proportional to frequency, obtained by dividing Hz [1/s] by the speed of light in cm/s). And now the truly fun part... Data processing! Allow me to illustrate with a sample...

PZT wavelength Signal
3.000 6305.410 -0.090851
2.975 6305.410 -0.090966
2.950 6305.410 -0.090874
2.925 6305.410 -0.090867
2.900 6305.410 -0.090645
2.875 6305.410 -0.090958
2.850 6305.420 -0.091202
2.825 6305.420 -0.090951
2.800 6305.420 -0.090905
2.775 6305.420 -0.090813
2.750 6305.430 -0.090684
2.725 6305.430 -0.090584
2.700 6305.430 -0.090493
2.675 6305.430 -0.090577
2.650 6305.430 -0.090538
2.625 6305.440 -0.090546
2.600 6305.440 -0.090554
2.575 6305.440 -0.090523
2.550 6305.440 -0.090599
2.525 6305.450 -0.090599
2.500 6305.450 -0.090577
2.475 6305.460 -0.090577
2.450 6305.450 -0.090622
2.425 6305.460 -0.090752
2.400 6305.460 -0.090798
2.375 6305.460 -0.090920
2.350 6305.470 -0.090905
2.325 6305.470 -0.090867
2.300 6305.470 -0.090783
2.275 6305.480 -0.090584
2.250 6305.480 -0.090386
2.225 6305.480 -0.090493
2.200 6305.480 -0.090691
2.175 6305.490 -0.090844
2.150 6305.490 -0.090783
2.125 6305.490 -0.090714
2.100 6305.490 -0.090462
2.075 6305.500 -0.090493
2.050 6305.500 -0.090699
2.025 6305.510 -0.090867
2.000 6305.510 -0.091011
1.975 6305.510 -0.090607
1.950 6305.510 -0.090561
1.925 6305.520 -0.090905
1.900 6305.520 -0.091019
1.875 6305.520 -0.090874
1.850 6305.520 -0.090607
1.825 6305.530 -0.090996

This is a single page from a set of data. The actual range of PZT (Peizoelectric Tuning) voltage ranges from 3 volts to -3 volts. The signal is a measure of the light intensity. It is in an arbitrary unit and is mostly only useful when examining relative intensities. The wavelength is ACTUALLY the frequency, in cm^-1 units.

Anyway, the first thing you should notice is that the wavenumber readings are not very precise. The second thing you should notice is that the relationship between the wavenumber and voltage is not linear. And the third thing you should notice is that sometimes the wavenumber reading jumps back and forth. I need to compensate for this, somehow.

So what do we do when we have a whole lot of data that is not exactly precise but should still follow a nice, nonlinear trend? Why, we do a quadratic or a cubic regression of some sort. Unfortunantly, I can't just plug it into my TI-83 calculator and do it on there, especially considering there's about 250 sets of data of 241 datapoints each. Things will get a little bit disgusting really really fast.

I could copy and paste all of it directly into Excel... But a) The files arn't quite in the correct format for that and b) there's still 250 sets of data and that would be way too time consuming.

SO... I need to figure out how to fit data manually, without the luxury of calculators or applets on the internet or Excel. I need to get down to the bare bones of how it's done, and then write my own program to go through all 250 sets and re-save them as files in a nicer format that can hopefully be copied to Excel a lot better.

Do I have any leads? You betcha. I have two leads, to be precise. An easy way, and a hard way.

The easy way involves various summations, partial derivatives (A form of derivatives that will be taught in second year calculus. It is used in equations involving more than one variable. Essentially, you differentiate with respect to one variable and treat the others exactly like they were constants), followed by a nice easy linear system to solve.

The hard way involves matrices, and doing some awkwards things with them that I don't quite understand... But the harder method is apparantly more adaptable to other kinds of fitting lines than the former, and will also force me to learn more about linear algebra, something which will prove extremely useful in the long run. So... I'd better get cracking, cuz I'm going for the hard way.

I'll need to learn how to do matrices in java, and also I'll need to program some methods to deal with all of the useful operations that can be done with matrices.

If anyone has any wisdom to share on this topic, please give me a yell, or leave a comment!

Have a good one.