Handle missing value

[dongli]

Here is the test CSV file:

foo,bar
1.0,2.0
,4.0
5.0,
7.0,

And with following codes:

program test

  use csv_module

  implicit none

  type(csv_file) f
  logical status_ok
  character(10), allocatable :: columns(:)
  integer i
  real(8), allocatable :: x(:)

  call f%read('test.csv', header_row=1, status_ok=status_ok)
  print *, 'status_ok =', status_ok

  call f%get_header(columns, status_ok)
  print *, 'status_ok =', status_ok
  print *, columns

  call f%get(1, x, status_ok)
  print *, 'status_ok =', status_ok
  do i = 1, size(x)
    print *, x(i)
  end do

end program test

The output is:

 status_ok = T
 status_ok = F
 foo       bar
 status_ok = T
   1.00000000000000
  0.000000000000000E+000  ! <-- Assume this should be missing value
   5.00000000000000
   7.00000000000000

As you can see, the second value is expected to be missing value, but I got 0.000000000000000E+000, which may cause difficulty to judge whether the value is valid or not. Can we specify the missing value when get as:

call f%get(1, x, missing_value=99999.0, status_ok=status_ok)

[jvdp1]

I am just running in this issue: I have CSV files with temperature measurements, and some missing values (empty cells in the CSV files).
Missing values are replaced by "0", and these cannot be detected after reading the file (because a temperature could be equal to 0).

I forked this project, and modified the in this branch such that missing values (defined as '1,,3' or '1, ,3') are replaced by a string provided by the user, e.g.,:

call f%read('file.csv', missing = '-9999', status_ok= status_ok)

It does the job for my case. But I may overlook things.

[epagone]

I think that this is an important enhancement. Is there a reason why @jvdp1's fork changes cannot be imported?

[jvdp1]

Hi. I don't know. Probably because I never open a PR? If @jacobwilliams agrees, I can open a PR from my fork.

[jacobwilliams]

Sure! Send a pull request and I'll look at it.