Frequency tables

[Beliavsky]

For a 1D array of integer or character data, one often wants to summarize the frequency of observations. One could have a subroutine returning two allocatable arrays of equal size, one an array of distinct values and the other an integer array of counts, for example

subroutine frequencies(ivec,ival,icount)
integer, intent(in) :: ivec(:)
integer, intent(out), allocatable :: ival(:) ! unique values
integer, intent(out), allocatable :: icount(:) ! counts of unique values
end subroutine frequencies

For integer, character, and logical 1D arrays one could define a MODE function that returns the most common value. The mode of continuous data is often not meaningful, so I exclude REAL.

[Beliavsky]

Here is a subroutine to compute a frequency table and a driver that I posted to Fortran Discourse. Ideally there would be options to sort the unique values or to sort by their frequencies. For large vectors it would be more efficient to sort the input data before finding unique values.

module unique_mod
implicit none
contains
subroutine unique(vec,vec_unique,freq)
! Return the unique values from vec and optionally their frequencies
integer, intent(in)                         :: vec(:)
integer, intent(out), allocatable           :: vec_unique(:) ! (nuniq) unique values
integer, intent(out), allocatable, optional :: freq(:)       ! (nuniq) corresponding frequencies, if PRESENT
integer                                     :: i,num(size(vec)),nuniq
logical                                     :: mask(size(vec))
mask = .false.
do i=1,size(vec)
    num(i) = count(vec(i) == vec) ! count the number of occurrences of this element:
    if (num(i)==1) then ! there is only one, flag it:
        mask(i) = .true.
    else
        if (.not. any(vec(i)==vec .and. mask)) mask(i) = .true. ! flag this value only if it hasn't already been flagged:
    end if
end do
! return only flagged elements:
nuniq = count(mask)
allocate(vec_unique(nuniq))
vec_unique = pack(vec,mask)
if (present(freq)) then
   allocate (freq(nuniq))
   freq = pack(num,mask)
end if
end subroutine unique
end module unique_mod   

program main
use unique_mod, only: unique
implicit none
integer, allocatable :: unq(:),freq(:),freq_chk(:)
integer, parameter   :: n = 100
real                 :: x(n)
integer              :: i,iran(n),nuniq
character (len=*), parameter :: fmt_i = "(100(i0,1x))"
! small test case
call unique([3,1,1,5],unq,freq)
print*,unq
print*,freq
! larger test case
call random_seed()
call random_number(x)
iran = nint(100*x)
call unique(iran,unq,freq)
nuniq = size(unq)
write (*,fmt_i) unq
write (*,fmt_i) freq
allocate (freq_chk(nuniq))
freq_chk = [(count(iran==unq(i)),i=1,nuniq)]
write (*,fmt_i) freq_chk
print*,"should be zero:",sum((freq-freq_chk)**2),sum(freq)-n
end program main

[jvdp1]

This unique subroutine sounds interesting and fits in the scope of stdlib IMO. Would you like to specify the API?

[Beliavsky]

For continuous data, one usually wants to know the frequencies of data in bins. I have a subroutine that with a call

call print_bin_freq(dble([(i,i=-3,3)]),xx,outu=6,fmt_header="(/,'FREQUENCIES')")

gives output such as

FREQUENCIES
     >=min      <max          #
             -3.0000         14
   -3.0000   -2.0000        208
   -2.0000   -1.0000       1418
   -1.0000    0.0000       3402
    0.0000    1.0000       3387
    1.0000    2.0000       1349
    2.0000    3.0000        207
    3.0000                   15

[Beliavsky]

There should be a mode function, which returns the value with the highest frequency in the array. It would be meaningful for integer, character, and logical data. For raw continuous data, one can compute the mode, but it may not be meaningful, since all values may have frequency one. The mode is a byproduct of having a frequency table.

[arjenmarkus]

That would be useful, but keep in mind htat the mode is not necessarily unique. How should one deal with that? Simply return the first (smallest) value? Or offer a version that reports them all? Op zo 25 apr. 2021 om 19:05 schreef Beliavsky ***@***.***>:

…

There should be a mode function, which returns the value with the highest frequency in the array. It would be meaningful for integer, character, and logical data. For raw continuous data, one can compute the mode, but it may not be meaningful, since all values may have frequency one. The mode is a byproduct of having a frequency table. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#380 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN6YR6XA32EOB2Q6L7Q7ZLTKRDWVANCNFSM42TEYNRQ> .