Size and Color inside marker not working properly in R

[renanxcortes]

Hello, When I was running a code that was supposed to draw a scatterplot and color and resize a single dot of it, I realized that it wasn't working properly. The only way to fix this problem was to add an "arrange" function for the State Initial in the input dataset of plotly. Otherwise, this code was coloring and resizing the shape of Arkansas instead of the US.

It seems like the code was coloring and resizing it according to the alphabetical order of the states initial and not according to the 'ifelse' function inside the marker list.

Here's the code and the file "national_county.txt" is attached:

# Code to extractdata from FRED
library(alfred)
library(tidyverse)
library(lubridate)
library(plotly)
library(zoo)
library(data.table)
library(BBmisc)

calculate_acum_growth_S_months    <- function(x, s) { aux <- rollapply(x, width = s, FUN = sum); return(c(rep(NA, s - 1),(aux / lag(aux, s) - 1) * 100)) }
calculate_acelleration_S_months   <- function(x, s, k) { aux <- diff(calculate_acum_growth_S_months(x, s), k); return(c(NA, aux)) }


# List of US States initials
county_corresp <- tbl_df(fread("national_county.txt"))
names(county_corresp) <- c("STATE", "STATEFP", "COUNTYFP", "COUNTYNAME", "CLASSFP")
states_initials <- unique(county_corresp$STATE)

# United States first (all join will be after the first grab)
string_grab = c("UNRATENSA", paste0(states_initials, "URN"))

# Inspired in https://stackoverflow.com/questions/11254524/omit-rows-containing-specific-column-of-na
# Just modified a little bit in order to use the column name "dplyr's wise"
completeFun <- function(data, desiredCols) {
  completeVec <- complete.cases(data[, desiredCols])
  return(data[completeVec, ])
}


get_fred_series_paese <- function(series_id, series_name = NULL,
                                  observation_start = NULL, observation_end = NULL) {
  
  
  length_series_id <- nchar(series_id)
  
  if (is.character(series_id) == FALSE) {
    stop("series_id is always in characters")
  }
  
  if (is.null(series_name) == TRUE ) {
    series_name <- series_id
  }
  
  if (is.null(observation_start) == TRUE) {
    observation_start <- "1776-07-04"
  }
  
  if (is.null(observation_end) == TRUE) {
    observation_end <- "9999-12-31"
  }
  
  df_series <-
    try({
      fromJSON(
        paste0("https://api.stlouisfed.org/fred/series/observations?series_id=",
               series_id,
               "&observation_start=",
               observation_start,
               "&observation_end=",
               observation_end,
               "&output_type=2",
               "&api_key=98f9f5cad7212e246dc5955e9b744b24&file_type=json")
      )$observations %>%
        mutate_(date = ~as_date(date))
    }, silent = TRUE)
  
  print(class(df_series))
  
  if (!class(df_series) == "try-error") {
    colnames(df_series)[!(colnames(df_series) %in% "date")] <- series_name
    df_series[, 2] <- as.numeric(unlist(df_series[, 2]))
    df_series
    
  } else {
    df_series <- tibble(date = as.Date(NA))
  }
  
}

create_df <- function(i) {
x <- get_fred_series_paese(string_grab[i])
if (!is.error(x)) # Even if FRED doesn't the series, it won't stop
{x} else 
{tibble(date = as.Date(NA))}}

data = 1:length(string_grab) %>% 
       map(create_df) %>% 
       reduce(full_join, by = "date") %>%
       completeFun("date")


data_clean <- data %>%
  rename(USURN = UNRATENSA) %>%
  gather(Raw_Type, UR, -date) %>%
  separate(Raw_Type, into = c("State", "Seasonal_Type"), sep = 2) %>%
  mutate(year = year(date),
         month = month(date)) %>%
  group_by(State, Seasonal_Type) %>%
  mutate(Var = calculate_acum_growth_S_months(UR, 12),
         Ace = calculate_acelleration_S_months(Var, s = 12, k = 1))


base_aux = data_clean %>%
              filter(Seasonal_Type == "URN",
                     year == 2016,
                     month == 10) #%>%
              #arrange(State)


# Spatial Unemployment
base_aux %>%
  plot_ly(x = ~Ace, 
          y = ~Var, 
          type = 'scatter', 
          mode = 'markers', 
          marker = list(size = ifelse(base_aux$State == "US", 20, 10),
                        #sizeref = .10, 
                        color = ifelse(base_aux$State == "US", "red","#004B82")#,
                        #symbol = 1:length(base_aux$State)
          ),  
          hoverinfo = "text",
          text = paste("", base_aux$State, "<br>",
                       "Growth: ", round(base_aux$Var, 3), "<br>",
                       "Acelleration: ", round(base_aux$Ace, 3)),
          showlegend = TRUE)

national_county.txt

[cpsievert]

I see what looks to be a dplyr error during creation of data_clean

Error in .f(.x[[i]], ...) : object 'UNRATENSA' not found

[cpsievert]

Please do try to reduce the complexity of your example

[renanxcortes]

Ow, sure, sorry that my example was not working! Here's a much simpler example of what I'm referring to. When you arrange the dataset, the markers are colored appropriately, but if you don't, plotly colors alphabetically (perhaps, it does some arrange internally... I don't know...). Am I missing something or my question is justified?

ps.: I attached a zip file with the data. Please let know if you could run the example.

Thank you!

library(tidyverse)
library(plotly)

data <- readRDS("data_issue_1351.rds")


base_aux_states = data %>%
                  filter(Seasonal_Type == "URN",
                         year == 2018,
                         month == 1) #%>%
                  #arrange(ST)


# Scatter Plot
base_aux_states %>%
  plot_ly(x = ~Ace, 
          y = ~Var, 
          type = 'scatter', 
          mode = 'markers', 
          marker = list(size = ifelse(base_aux_states$ST == "US", 20, 10),
                        color = ifelse(base_aux_states$ST == "US", "red","#004B82")
          ),  
          hoverinfo = "text",
          text = paste("", base_aux_states$ST, "<br>",
                       "Growth: ", round(base_aux_states$Var, 3), "<br>",
                       "Acelleration: ", round(base_aux_states$Ace, 3)),
          showlegend = TRUE) %>%
  layout(title = paste0("Example Graph"))

data_issue_1351.zip

[cpsievert]

That zip file uncompresses to a unix executable?

[renanxcortes]

Hm... I don't know what might be happening... Please try using this .csv file (I created a repo).

https://raw.githubusercontent.com/renanxcortes/repo_for_supplementary_files/master/data_issue_1351.csv

Let me know if it works for you.

Thanks,
Renan

[cpsievert]

Ok, this is what I get, were you expecting something different?

library(plotly)

f <- "https://raw.githubusercontent.com/renanxcortes/repo_for_supplementary_files/master/data_issue_1351.csv"

base_aux_states <- readr::read_csv(f) %>%
  filter(Seasonal_Type == "URN", year == 2018, month == 1) 

# Scatter Plot
base_aux_states %>%
  plot_ly(x = ~Ace, 
          y = ~Var, 
          type = 'scatter', 
          mode = 'markers', 
          marker = list(size = ifelse(base_aux_states$ST == "US", 20, 10),
                        color = ifelse(base_aux_states$ST == "US", "red","#004B82")
          ),  
          hoverinfo = "text",
          text = paste("", base_aux_states$ST, "<br>",
                       "Growth: ", round(base_aux_states$Var, 3), "<br>",
                       "Acelleration: ", round(base_aux_states$Ace, 3)),
          showlegend = TRUE) %>%
  layout(title = paste0("Example Graph"))

[renanxcortes]

Hm... oh my God... that's really strange... When I run your code I get exactly what you get... However, when I run reading from the .rds (which basically the same data, without the new X1 column generated with your code). I get this:

I'm running exactly this:

library(tidyverse)
library(plotly)

data <- readRDS("data_issue_1351.rds")


base_aux_states = data %>%
                  filter(Seasonal_Type == "URN",
                         year == 2018,
                         month == 1) #%>%
                  #arrange(ST)

# Scatter Plot
base_aux_states %>%
  plot_ly(x = ~Ace, 
          y = ~Var, 
          type = 'scatter', 
          mode = 'markers', 
          marker = list(size = ifelse(base_aux_states$ST == "US", 20, 10),
                        color = ifelse(base_aux_states$ST == "US", "red","#004B82")
          ),  
          hoverinfo = "text",
          text = paste("", base_aux_states$ST, "<br>",
                       "Growth: ", round(base_aux_states$Var, 3), "<br>",
                       "Acelleration: ", round(base_aux_states$Ace, 3)),
          showlegend = TRUE) %>%
  layout(title = paste0("Example Graph"))

I'm using plotly version 4.8.0.

Is there a way for you to test your code with reading from the .rds and see if you get the same as me?

[cpsievert]

Is there a way for you to test your code with reading from the .rds and see if you get the same as me?

If you can get me a link to the rds? I have a strong feeling this is not a plotly issue though

[renanxcortes]

Is there a way for you to test your code with reading from the .rds and see if you get the same as me?

If you can get me a link to the rds? I have a strong feeling this is not a plotly issue though

Ok, thanks. I sent through e-mail.

[cpsievert]

Ahh, it appears the dplyr grouping is messing something up in plotly. You can fix it for now by ungroup() the dataset.

base_aux_states <- readRDS("data_issue_1351.rds") %>%
  ungroup() %>%
  filter(Seasonal_Type == "URN", year == 2018, month == 1) 

# Scatter Plot
base_aux_states %>%
  plot_ly(x = ~Ace, 
          y = ~Var, 
          type = 'scatter', 
          mode = 'markers', 
          marker = list(size = if_else(base_aux_states$ST == "US", 20, 10),
                        color = if_else(base_aux_states$ST == "US", "red", "#004B82")
          ),  
          hoverinfo = "text",
          text = paste("", base_aux_states$ST, "<br>",
                       "Growth: ", round(base_aux_states$Var, 3), "<br>",
                       "Acelleration: ", round(base_aux_states$Ace, 3)),
          showlegend = TRUE) %>%
  layout(title = paste0("Example Graph"))

I'd consider this a plotly bug though

[renanxcortes]

Oh! Sure, the data was grouped, I didn't remember that! Thank you! Hopefully you'll fix this!