Skip to content

Add support for an externally provided HyperProcess #117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 9, 2021

Conversation

vogelsgesang
Copy link
Contributor

@vogelsgesang vogelsgesang commented Feb 7, 2021

So far, pantab always internally created an adhoc HyperProcess. This
commit allows users to instead spawn their own HyperProcess and pass
it to pantab's functions through a hyper keyword argument.

By providing their own HyperProcess, users

  • have full control over the start-up flags of HyperProcess and can,
    e.g., disable the creation of the hyperd.log log file.
  • reuse the same HyperProcess across multiple function calls and
    thereby save a few milliseconds per function invocation.

closes #51, closes #39

@vogelsgesang vogelsgesang force-pushed the external-hyperd-connection branch from bc82f9e to 2bc1325 Compare February 7, 2021 23:08
@vogelsgesang
Copy link
Contributor Author

No clue which issue the linux build has with the formatting... On my local machine black runs through without any issues.
Could it be that black is somehow dependent on the environment it's running on?

@WillAyd
Copy link
Collaborator

WillAyd commented Feb 8, 2021

Can you check #118 as well? I'm guessing some of the changes from this PR made their way in there

@WillAyd
Copy link
Collaborator

WillAyd commented Feb 8, 2021

No clue which issue the linux build has with the formatting... On my local machine black runs through without any issues.
Could it be that black is somehow dependent on the environment it's running on?

It's possible a versioning thing. I would suggest trying black-20.8b1 locally to match what CI is getting

@vogelsgesang vogelsgesang force-pushed the external-hyperd-connection branch from b613f2d to 14b37c8 Compare February 8, 2021 19:53
@vogelsgesang
Copy link
Contributor Author

Ok - the black issues were indeed resolved by installing the right version. Now, I am running into errors from mypy:

pantab/_tester.py:8: error: Cannot find implementation or library stub for module named 'pytest'
pantab/_hyper_util.py:4: error: Cannot find implementation or library stub for module named 'tableauhyperapi'
pantab/_compat.py:3: error: Cannot find implementation or library stub for module named 'pandas'
pantab/_types.py:4: error: Cannot find implementation or library stub for module named 'tableauhyperapi'
pantab/_writer.py:8: error: Cannot find implementation or library stub for module named 'numpy'
pantab/_writer.py:9: error: Cannot find implementation or library stub for module named 'pandas'
pantab/_writer.py:10: error: Cannot find implementation or library stub for module named 'tableauhyperapi'
pantab/_reader.py:6: error: Cannot find implementation or library stub for module named 'numpy'
pantab/_reader.py:7: error: Cannot find implementation or library stub for module named 'pandas'
pantab/_reader.py:8: error: Cannot find implementation or library stub for module named 'tableauhyperapi'
pantab/tests/test_writer.py:3: error: Cannot find implementation or library stub for module named 'numpy'
pantab/tests/test_writer.py:4: error: Cannot find implementation or library stub for module named 'pandas'
pantab/tests/test_writer.py:5: error: Cannot find implementation or library stub for module named 'pytest'
pantab/tests/test_writer.py:6: error: Cannot find implementation or library stub for module named 'tableauhyperapi'
pantab/tests/test_roundtrip.py:1: error: Cannot find implementation or library stub for module named 'numpy'
pantab/tests/test_roundtrip.py:2: error: Cannot find implementation or library stub for module named 'pandas'
pantab/tests/test_roundtrip.py:3: error: Cannot find implementation or library stub for module named 'pandas.testing'
pantab/tests/test_roundtrip.py:4: error: Cannot find implementation or library stub for module named 'tableauhyperapi'
pantab/tests/test_reader.py:1: error: Cannot find implementation or library stub for module named 'pandas'
pantab/tests/test_reader.py:2: error: Cannot find implementation or library stub for module named 'pandas.testing'
pantab/tests/test_reader.py:3: error: Cannot find implementation or library stub for module named 'pytest'
pantab/tests/test_reader.py:4: error: Cannot find implementation or library stub for module named 'tableauhyperapi'
pantab/tests/conftest.py:3: error: Cannot find implementation or library stub for module named 'numpy'
pantab/tests/conftest.py:3: note: See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports
pantab/tests/conftest.py:4: error: Cannot find implementation or library stub for module named 'pandas'
pantab/tests/conftest.py:5: error: Cannot find implementation or library stub for module named 'pytest'
pantab/tests/conftest.py:6: error: Cannot find implementation or library stub for module named 'tableauhyperapi'

I don't understand how my code changes could have broken this. Are the CI jobs currently expected to pass?

@WillAyd
Copy link
Collaborator

WillAyd commented Feb 8, 2021 via email

@WillAyd
Copy link
Collaborator

WillAyd commented Feb 8, 2021

Looks like the Mypy issue might just be a bug with the latest version:

python/mypy#9940

I'll push up a patch separately

Copy link
Collaborator

@WillAyd WillAyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking very nice. mypy issue is fixed on master so feel free to rebase or merge master whenever convenient

@WillAyd WillAyd added the enhancement New feature or request label Feb 8, 2021
@vogelsgesang vogelsgesang force-pushed the external-hyperd-connection branch from 1c5702e to 0037b38 Compare February 8, 2021 23:43
@pep8speaks
Copy link

pep8speaks commented Feb 8, 2021

Hello @vogelsgesang! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-02-09 00:02:05 UTC

@vogelsgesang vogelsgesang force-pushed the external-hyperd-connection branch from 0037b38 to d19bf81 Compare February 8, 2021 23:44
So far, pantab always internally created an adhoc `HyperProcess`. This
commit allows users to instead spawn their own `HyperProcess` and pass
it to pantab's functions through a `hyper` keyword argument.

By providing their own `HyperProcess`, users
* have full control over the start-up flags of `HyperProcess` and can,
  e.g., disable the creation of the `hyperd.log` log file.
* reuse the same `HyperProcess` across multiple function calls and
  thereby save a few milliseconds per function invocation.
@vogelsgesang vogelsgesang force-pushed the external-hyperd-connection branch from d19bf81 to 7347cc7 Compare February 9, 2021 00:02
@vogelsgesang
Copy link
Contributor Author

should be ready to be merged now

@WillAyd WillAyd merged commit f2473dc into innobi:master Feb 9, 2021
@WillAyd
Copy link
Collaborator

WillAyd commented Feb 9, 2021

Thanks @vogelsgesang

@vogelsgesang vogelsgesang deleted the external-hyperd-connection branch February 9, 2021 18:07
@farhangithub27
Copy link

Any timeline when this will be released to pypi.

@vogelsgesang
Copy link
Contributor Author

There are still two pull requests in the pipeline (#123 and #128) which we probably want to merge before cutting a new release.

Neither of those changes is a strict requirement before cutting a new release. But we would prefer to have them in the 2.0 release, since at least #123 is also an interface-breaking change. We prefer to bundle those interface-breaking changes into one release instead of having two subsequent interface-breaking changes, so we reduce the burden of upgrading the pantab dependency for down-stream users of pantab.

Originally, we hoped to move #123 and #128 ahead much faster. Unfortunately, it took longer because I first had to get approval by my company to upstream those changes and this whole legal process took a while. But that's out of the way now, so I hope we will get those two changes merged within the next week or so.

The final decision on when to release a new version is up to @WillAyd, though...

@vogelsgesang
Copy link
Contributor Author

@farhangithub27 pantab 2.0 was just released to pypi and this new version also comes with support for the externally provided HyperProcess

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow Telemetry Allow user to pass connection / hyper instance
4 participants