Fix decode json error at trailing empty line

[clouds56]

No description provided.

[apetresc]

Thanks for the contribution, @clouds56 :)

Is this related to #471? Because I have been unable to reproduce that issue with any of my configured versions of Livy. Do you have a failing query and Livy/Spark version that triggers this issue? I would at least like to add a failing test that this resolves, to make sure it's quashed.

Thanks again!

[clouds56]

Yes, I think it should be the common case, since the encoding side (I'm using scala kernel) goes like

sparkmagic/sparkmagic/sparkmagic/livyclientlib/sparkstorecommand.py

Line 92 in 6b46996

return Command(u'{}.foreach(println)'.format(command))

There would be a empty newline at the end. I wonder how it works before?

In my case, server (arch linux) spark/livy version
https://aur.archlinux.org/packages/apache-spark/ 2.3.1
https://livy.incubator.apache.org/download/ 0.5.0-incubating
client (macOS)
python3 3.7.0
sparkmagic 0.12.6 (install via pip)

[liampetti]

Same problem here using Livy 5.0

%%sql
SELECT 1

An internal error was encountered.
Please file an issue at https://github.com/jupyter-incubator/sparkmagic
Error: Cannot parse object as JSON: '['{"1":1}', '']'

[Tagar]

We can't run %%sql

not

%%spark -c sql

Would it be possible to merge this PR? Thanks.

[jcesart20]

Yes, this bug is blocking the %%sql usage, could you please merge this fix? thanks!

[apetresc]

Just pushed this to release 0.12.7, which is going out momentarily.

[Tagar]

that's great news @apetresc we have this as part of 0.12.7 release - thank you

Do you know if this release also addresses Python 3 compatibility issues described in
#507

cc @gm-spacagna

[Tagar]

separately, FYI - I don't see 0.12.7 available on conda yet

https://anaconda.org/search?q=sparkmagic

[apetresc]

Do you know if this release also addresses Python 3 compatibility issues described in
#507

No, I'm still working on merging that one. Basically I'm having trouble validating the fix due to Hadoop-related issues in my test environment that I'm still working through. It should be ready soon, though, I hope. (My read-through of the code is good, I just don't want to push a release until my test bench is happy under every possible permutation of conditions, which is starting to add up to quite a lot)

separately, FYI - I don't see 0.12.7 available on conda yet

Hmm actually I have no idea how releases get propagated to Conda; the TravisCI job that pushes to PyPI doesn't include any references to it. However, 0.12.6 is there (which I also did the release process for), which means it must be getting pulled on their end. I guess it's a scheduled job that just hasn't had a chance to propagate yet. I'll check back in a few days to see if it's appeared on Conda yet, and if it hasn't I'll try to ping the original maintainer to figure out what I need to do.

[Tagar]

Do you know if this release also addresses Python 3 compatibility issues described in
#507

No, I'm still working on merging that one. Basically I'm having trouble validating the fix due to Hadoop-related issues in my test environment that I'm still working through. It should be ready soon, though, I hope. (My read-through of the code is good, I just don't want to push a release until my test bench is happy under every possible permutation of conditions, which is starting to add up to quite a lot)

Thanks a lot @apetresc for working on this..
Just out of curiosity - do we have some automated tests environment that includes hadoop?
I think this should have helped with speed of adopting new PRs.

Let me know if there is something that we can help?

Thanks again.

ps. I opened a separate issue on conda/conda for question on syncing up conda repo.