Skip to content

[Prototype] Client-side caching option for Epidata.covidcast Python #541

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 13 commits into
base: dev
Choose a base branch
from
Open

[Prototype] Client-side caching option for Epidata.covidcast Python #541

wants to merge 13 commits into from

Conversation

dshemetov
Copy link
Contributor

@dshemetov dshemetov commented May 14, 2021

Description

Epidata.covidcast is idempotent on the hour-scale (I think). This PR adds client-side caching to Epidata.covidcast in Python, to save user effort in writing their own file saving mechanisms.

Example:

# works as normal
> df = Epidata.covidcast('source', 'signal', 'day', 'county', 20200414, '01234')
# a regular call to the API that caches locally
> df = Epidata.covidcast('source', 'signal', 'day', 'county', 20200414, '01234', cache_timeout=20)
# performs a local lookup (if run within 20s)
> df = Epidata.covidcast('source', 'signal', 'day', 'county', 20200414, '01234', cache_timeout=20)

The goal is to expose cache_timeout to the covidcast Python library in another PR. We can keep the default behavior cacheless to remove hidden state issues.

Overlapping, but not identical time windows will look to the cache and the web gracefully (because the covidcast Python library splits date range requests into daily Epidata.covidcast calls). This is not true for geos (but could be if we can provide metadata of available geos per indicator).

Changes

Switch to requests_cache when the cache_timeout argument is sent.

@krivard krivard requested a review from chinandrew May 17, 2021 17:08
Epidata.covidcast('src', 'sig', 'day', 'county', 20200414, '01234')
get.assert_called_once()
post.assert_not_called()
assert call('get') in Session.request.call_args_list()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: check that the length of the call args (number of calls) is correct. otherwise lgtm

@krivard
Copy link
Contributor

krivard commented May 21, 2021

Are any of these changes:

  • breaking
  • announce-able
  • needing public documentation

@sgratzl sgratzl added python client changes the Python client enhancement labels Jun 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement python client changes the Python client
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants