Speedup parsing lengths by exploiting float32 vs. uint32 structure

[MarcCote]

Hi @emanuele, I believe I found a way to improve the speed for parsing the streamline lengths. It might need more testing, but I thought I would share it with you anyway.

Here are the results I get on my laptop (with SSD). The relevant parts are load_trk_newer.py and load_trk_no_numba.py. I also added a checksum to make sure the loaded streamlines are the same for all methods (Checksum: 28226791454.458008).

Using load_trk.py
Loading data/sub-599469_var-10M_tract.trk
Parsing lenghts of 10000000 streamlines
14.822022438049316 sec.
Extracting 4000000 streamlines with the desired id
78.17423558235168 sec.
Converting all streamlines to the container array
0.13917803764343262 sec.
Total time: 94.22346591949463 sec.

Using load_trk_new.py
Loading data/sub-599469_var-10M_tract.trk
Parsing lenghts of 10000000 streamlines
12.099209785461426 sec.
Extracting 4000000 streamlines with the desired id
22.72183060646057 sec.
Converting all streamlines to the container array
0.1382434368133545 sec.
Total time: 37.516242265701294 sec.

  Using load_trk_newer.py
  Loading data/sub-599469_var-10M_tract.trk
  Parsing lenghts of 10000000 streamlines
  2.5172533988952637 sec.
  Extracting 4000000 streamlines with the desired id
  7.231152772903442 sec.
  Casting ArraySequence to list of ndarray    <------ Working directly with an ArraySequence would avoid this.
  10.564280033111572 sec.
  Converting all streamlines to the container array
  0.29190659523010254 sec.
  Total time: 21.28335165977478 sec. Checksum: 28226791454.458008
  
  Using load_trk_no_numba.py
  Loading data/sub-599469_var-10M_tract.trk
  Loading the whole data blob.
  751859555 values read in 1.3201448917388916 sec.
  Parsing lengths of 10000000 streamlines
  1.38914155960083 sec.
  Extracting 4000000 streamlines
  4.8606531620025635 sec.
  Converting all streamlines to the container array
  0.11880946159362793 sec.
  Total time: 9.421625137329102 sec. Checksum: 28226791454.458008

Using load_trk_numba.py
Loading data/sub-599469_var-10M_tract.trk
Loading the whole data blob.
751859555 values read in 1.5123486518859863 sec.
Parsing lengths of 10000000 streamlines
1.3011393547058105 sec.
Extracting 4000000 streamlines
2.1483805179595947 sec.
Converting all streamlines to the container array
0.1565077304840088 sec.
Total time: 6.776575803756714 sec. Checksum: 28226791454.458008

Saving original streamlines in npz format to data/sub-599469_var-10M_tract_no_resample.npy
Total time: 49.95222306251526 sec.

Loading data/sub-599469_var-10M_tract_no_resample.npy
Total time: 37.35902452468872 sec.

[MarcCote]

A heuristic for extracting streamline lengths from a TRK file.