issues with plotting secondary axis

[IndrajeetPatil]

Following my Twitter conversation with @dpseidel (https://twitter.com/patilindrajeets/status/1030247641240756224), I am filing an issue. I am trying to create a histogram with both counts and relative frequency (or proportion) data displayed on the y-axes, the former on the left y-axis and the latter on the right. I have managed to write a custom function that does this and it's going to be part of my package. But the function works only when data = NULL and not otherwise and I am not sure what's going on.

I have created a toy example here because I didn't want to share the whole function script.

library(tidyverse)
library(magrittr)
library(ggplot2)

# function body
toy_bar <- function(data = NULL, x) {
  
================ creating data frame ========================

  # preparing a dataframe out of provided inputs
  if (!is.null(data)) {
    # if dataframe is provided
    data <-
      dplyr::select(.data = data,
                    x = !!rlang::enquo(x))

  } else {
    # if vectors are provided
    data <-
      base::cbind.data.frame(x = x)
  }

  # convert to a tibble dataframe
  data %<>%
    tibble::as_data_frame(x = .)

  # denominator for computing proportions later
  total_obs <- nrow(x = data)

================ creating plot =================================

  # constructing the plot
  plot <- ggplot2::ggplot(data = data,
                          mapping = ggplot2::aes(x = x)) +
    # plot counts on y-axis
    ggplot2::stat_bin(
      col = "black",
      alpha = 0.7,
      na.rm = TRUE,
      mapping = ggplot2::aes(y = ..count..,
                             fill = ..count..)
    ) +
    # no color gradient needed for this plot
    ggplot2::scale_fill_gradient(name = "count",
                                 low = "white",
                                 high = "white") +
    # adding seconary axis with proportions
    ggplot2::scale_y_continuous(
      sec.axis = ggplot2::sec_axis(
        trans = ~ . / total_obs,
        labels = scales::percent,
        name = "proportion (in %)"
      )
    ) +
    ggplot2::ylab("count") +
    ggplot2::guides(fill = FALSE)

  # return the output
  return(plot)

}

# this works
toy_bar(x = ToothGrowth$len)
#> `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

# this doesn't work
toy_bar(data = ToothGrowth, x = len)
#> `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
#> Error in as.list.environment(x, all.names = TRUE): object 'len' not found

# sesion info
devtools::session_info()
#> Session info -------------------------------------------------------------
#>  setting  value                       
#>  version  R version 3.5.1 (2018-07-02)
#>  system   x86_64, mingw32             
#>  ui       RTerm                       
#>  language (EN)                        
#>  collate  English_United States.1252  
#>  tz       America/New_York            
#>  date     2018-08-17
#> Packages -----------------------------------------------------------------
#>  package       * version    date      
#>  assertthat      0.2.0      2017-04-11
#>  backports       1.1.2      2017-12-13
#>  base          * 3.5.1      2018-07-02
#>  bindr           0.1.1      2018-03-13
#>  bindrcpp        0.2.2      2018-03-29
#>  broom           0.5.0.9001 2018-08-09
#>  cellranger      1.1.0      2016-07-27
#>  cli             1.0.0      2017-11-05
#>  colorspace      1.3-2      2016-12-14
#>  compiler        3.5.1      2018-07-02
#>  crayon          1.3.4      2017-09-16
#>  datasets      * 3.5.1      2018-07-02
#>  devtools        1.13.6     2018-06-27
#>  digest          0.6.15     2018-01-28
#>  dplyr         * 0.7.6      2018-06-29
#>  evaluate        0.11       2018-07-17
#>  forcats       * 0.3.0      2018-02-19
#>  ggplot2       * 3.0.0      2018-07-03
#>  glue            1.3.0      2018-07-17
#>  graphics      * 3.5.1      2018-07-02
#>  grDevices     * 3.5.1      2018-07-02
#>  grid            3.5.1      2018-07-02
#>  gtable          0.2.0      2016-02-26
#>  haven           1.1.2      2018-06-27
#>  hms             0.4.2      2018-03-10
#>  htmltools       0.3.6      2017-04-28
#>  httr            1.3.1      2017-08-20
#>  jsonlite        1.5        2017-06-01
#>  knitr           1.20.12    2018-08-13
#>  labeling        0.3        2014-08-23
#>  lazyeval        0.2.1      2017-10-29
#>  lubridate       1.7.4      2018-04-11
#>  magrittr      * 1.5        2014-11-22
#>  memoise         1.1.0      2017-04-21
#>  methods       * 3.5.1      2018-07-02
#>  modelgenerics   0.0.1.9000 2018-08-09
#>  modelr          0.1.2      2018-05-11
#>  munsell         0.5.0      2018-06-12
#>  pillar          1.3.0.9000 2018-07-23
#>  pkgconfig       2.0.2      2018-08-16
#>  plyr            1.8.4      2016-06-08
#>  purrr         * 0.2.5      2018-05-29
#>  R6              2.2.2      2017-06-17
#>  Rcpp            0.12.18    2018-07-23
#>  readr         * 1.1.1      2017-05-16
#>  readxl          1.1.0      2018-04-20
#>  rlang           0.2.2      2018-08-16
#>  rmarkdown       1.10       2018-06-11
#>  rprojroot       1.3-2      2018-01-03
#>  rvest           0.3.2      2016-06-17
#>  scales          1.0.0      2018-08-09
#>  stats         * 3.5.1      2018-07-02
#>  stringi         1.2.4      2018-07-20
#>  stringr       * 1.3.1      2018-05-10
#>  tibble        * 1.4.2      2018-01-22
#>  tidyr         * 0.8.1      2018-05-18
#>  tidyselect      0.2.4      2018-02-26
#>  tidyverse     * 1.2.1      2017-11-14
#>  tools           3.5.1      2018-07-02
#>  utils         * 3.5.1      2018-07-02
#>  withr           2.1.2      2018-03-15
#>  xfun            0.3        2018-07-06
#>  xml2            1.2.0      2018-01-24
#>  yaml            2.2.0      2018-07-25                
#>  Github (tidymodels/broom@93610ae)    

Created on 2018-08-17 by the reprex package (v0.2.0.9000).

[dpseidel]

Just to clarify a little:
The issue at hand is that while PR #2797 fixed one issue -- the one demonstrated above, where sec.axis did not work with tidy eval -- it changed the functionality such that objects created inside the function are no longer recognized when called in the sec.axis transform, a simple reprex:

library(ggplot2) # 3.0.0.9000
library(tibble)
data = tibble(x = runif(10), y = runif(10))

# this fails because n_obs is not included in the evaluation environment of AxisSecondary$transform_range()
# this same function  works with v.3.0.0
test <- function(data){
  n_obs <- nrow(data)
  plot <- ggplot(data = data, 
                 aes(x = x, y = y)) + geom_point() + scale_y_continuous(
                   sec.axis = sec_axis(trans = ~ . / n_obs, 
                                       labels = scales::percent))
  plot
}

test(data)
#> Error in rlang::eval_tidy(rlang::f_rhs(self$trans), data = range): object 'n_obs' not found

# this works because data is in the global environment
test1 <- function(data){
  plot <- ggplot(data = data, 
                 aes(x = x, y = y)) + geom_point() + scale_y_continuous(
                   sec.axis = sec_axis(trans = ~ . / nrow(data), 
                         labels = scales::percent))
  plot
}

test1(data)

Similarly, if you superassign n_obs in the original function (not recommended, just proving my point), it works.

[IndrajeetPatil]

@dpseidel Oh, I see. Thanks for the detailed explanation.

As per your example, I modified the toy function shown above and downloaded the development version of ggplot2, but now neither of the function calls work.

library(tidyverse)
library(magrittr)
library(ggplot2) # 3.0.0.9000

toy_bar <- function(data = NULL, x) {
  # ======= dataframe ==========================
  # preparing a dataframe out of provided inputs
  if (!is.null(data)) {
    # if dataframe is provided
    data <-
      dplyr::select(.data = data,
                    x = !!rlang::enquo(x))

  } else {
    # if vectors are provided
    data <-
      base::cbind.data.frame(x = x)
  }

  # convert to a tibble dataframe
  data %<>%
    tibble::as_data_frame(x = .)

  # ===================== plot =========================

  # constructing plot
  plot <- ggplot2::ggplot(data = data,
                          mapping = ggplot2::aes(x = x)) +
    # plot counts on y-axis
    ggplot2::stat_bin(
      col = "black",
      alpha = 0.7,
      na.rm = TRUE,
      mapping = ggplot2::aes(y = ..count..,
                             fill = ..count..)
    ) +
    # no color gradient needed for this plot
    ggplot2::scale_fill_gradient(name = "count",
                                 low = "white",
                                 high = "white") +
    # adding seconary axis with proportions
    ggplot2::scale_y_continuous(
      sec.axis = ggplot2::sec_axis(
        trans = ~ . / nrow(x = data),
        labels = scales::percent,
        name = "proportion (in %)"
      )
    ) +
    ggplot2::ylab("count") +
    ggplot2::guides(fill = FALSE)

  # return the output
  return(plot)

}

# this works
toy_bar(x = ToothGrowth$len)
#> `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
#> Error in f(..., self = self): transformation for secondary axes must be monotonic

# this doesn't work
toy_bar(data = ToothGrowth, x = len)
#> `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
#> Error in f(..., self = self): transformation for secondary axes must be monotonic

You get the exact same errors even if ggplot2 3.0.0 is used.

Created on 2018-08-17 by the reprex package (v0.2.0.9000).

[dpseidel]

I cannot reproduce these monotonicity errors on 3.0.0 - the first call works, the second does not, just in your original example.

I can reproduce these errors in the dev version, but this stems from the same issue I mentioned above. In this case because data is still an object created entirely inside your function, and thus currently not seen by the evaluating environment -- AxisSecondary$transform_range() is calling a different data than you intend and returning numeric(0) internally which causes the monotonicity error later down the line in AxisSecondary$break_info. You can see this for yourself if you change your argument name from anything but data: for example, if data is changed to foo throughout your function and then passed to trans as expected ~ . / nrow(x = foo) , then you'll see the expected error: Error in nrow(x = foo) : object 'foo' not found.

The issue remains that the transformation of the secondary axis, when called within a function, is evaluated without the context of any objects created within said function.

[clauswilke]

@dpseidel I think this can be fixed by explicitly providing the evaluating environment, as I just did in this commit: clauswilke@d113402

library(ggplot2)
library(tibble)
data = tibble(x = runif(10), y = runif(10))

test <- function(data){
  n_obs <- nrow(data)
  plot <- ggplot(data = data, 
                 aes(x = x, y = y)) + geom_point() + scale_y_continuous(
                   sec.axis = sec_axis(trans = ~ . / n_obs, 
                                       labels = scales::percent))
  plot
}

test(data)

Created on 2018-08-17 by the reprex package (v0.2.0).

[clauswilke]

Also:

library(tidyverse)
library(magrittr)
#> 
#> Attaching package: 'magrittr'
#> The following object is masked from 'package:purrr':
#> 
#>     set_names
#> The following object is masked from 'package:tidyr':
#> 
#>     extract
library(ggplot2) # 3.0.0.9000

toy_bar <- function(data = NULL, x) {
  # ======= dataframe ==========================
  # preparing a dataframe out of provided inputs
  if (!is.null(data)) {
    # if dataframe is provided
    data <-
      dplyr::select(.data = data,
                    x = !!rlang::enquo(x))
    
  } else {
    # if vectors are provided
    data <-
      base::cbind.data.frame(x = x)
  }
  
  # convert to a tibble dataframe
  data %<>%
    tibble::as_data_frame(x = .)
  
  # ===================== plot =========================
  
  # constructing plot
  plot <- ggplot2::ggplot(data = data,
                          mapping = ggplot2::aes(x = x)) +
    # plot counts on y-axis
    ggplot2::stat_bin(
      col = "black",
      alpha = 0.7,
      na.rm = TRUE,
      mapping = ggplot2::aes(y = ..count..,
                             fill = ..count..)
    ) +
    # no color gradient needed for this plot
    ggplot2::scale_fill_gradient(name = "count",
                                 low = "white",
                                 high = "white") +
    # adding seconary axis with proportions
    ggplot2::scale_y_continuous(
      sec.axis = ggplot2::sec_axis(
        trans = ~ . / nrow(x = data),
        labels = scales::percent,
        name = "proportion (in %)"
      )
    ) +
    ggplot2::ylab("count") +
    ggplot2::guides(fill = FALSE)
  
  # return the output
  return(plot)
  
}

# this works
toy_bar(x = ToothGrowth$len)
#> `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

# this works
toy_bar(data = ToothGrowth, x = len)
#> `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

Created on 2018-08-17 by the reprex package (v0.2.0).

[dpseidel]

Great! Thanks @clauswilke! Do you want to open a PR? I can but it seems like you've got it fixed!

[clauswilke]

I made a PR. Are there any other tests we need to run? This seems to work but I'm feeling somewhat queasy that there may be something else we're missing.

[dpseidel]

Yeah I agree, I've been expanding the sec.axis test suite quite a bit lately but clearly this was still missed. We should at least explicitly add tests for appropriate behavior inside functions, beyond that perhaps @hadley or @thomasp85 have ideas for additional testing?

[lock]

This old issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with reprex) and link to this issue. https://reprex.tidyverse.org/