make_iap_request.py update seems to have broken the Launching Dataproc Jobs with Cloud Composer Qwiklab

[0yster]

In which file did you encounter the issue?

make_iap_request.py

The Qwiklab Launching Dataproc Jobs with Cloud Composer uses the make_iap_request.py file, which now no longer works. As a result the lab is currently offline.

Did you change the file? If so, how?

When a change was made by @arithmetic1728, sijunliu@ to this file, a Qwiklab that uses the file no longer works.
5ef84ba

Describe the issue

error:
(venv) student_00_da8d23ee74ab@cloudshell:~/professional-services/examples/cloud-composer-examples/composer_http_post_example (qwiklabs-gcp-00-68bd727eb07f)$ python dag_trigger.py \

--url="${AIRFLOW_URI}/api/experimental/dags/average-speed/dag_runs"
--iapClientId=$CLIENT_ID
--raw_path=gs://$PROJECT/cloud-composer-lab/raw-$EXPORT_TS/
Traceback (most recent call last):
File "/home/student_00_da8d23ee74ab/professional-services/examples/cloud-composer-examples/composer_http_post_example/venv/lib/python3.7/site-packages/google/auth/compute_engine/credentials.py", line 289, in _call_meta
data_identity_endpoint
self._target_audience
File "/home/student_00_da8d23ee74ab/professional-services/examples/cloud-composer-examples/composer_http_post_example/venv/lib/python3.7/site-packages/google/auth/compute_engine/_metadata.py", line 172, in get
response,
google.auth.exceptions.TransportError: ("Failed to retrieve http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/identity?audience=1087176421767-er001vugel6q9ottl7ihs8rp6963iv6o.apps.googl
eusercontent.com&format=full from the Google Compute Enginemetadata service. Status: 404 Response:\nb''", <google.auth.transport.requests._Response object at 0x7fe83917c048>)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "dag_trigger.py", line 100, in
main()
File "dag_trigger.py", line 96, in main
data=json.dumps(payload))
File "/home/student_00_da8d23ee74ab/professional-services/examples/cloud-composer-examples/composer_http_post_example/make_iap_request.py", line 44, in make_iap_request
google_open_id_connect_token = id_token.fetch_id_token(Request(), client_id)
File "/home/student_00_da8d23ee74ab/professional-services/examples/cloud-composer-examples/composer_http_post_example/venv/lib/python3.7/site-packages/google/oauth2/id_token.py", line 213, in fetch_id_token
credentials.refresh(request)
File "/home/student_00_da8d23ee74ab/professional-services/examples/cloud-composer-examples/composer_http_post_example/venv/lib/python3.7/site-packages/google/auth/compute_engine/credentials.py", line 312, in refresh
self.token, self.expiry = self._call_metadata_identity_endpoint(request)
File "/home/student_00_da8d23ee74ab/professional-services/examples/cloud-composer-examples/composer_http_post_example/venv/lib/python3.7/site-packages/google/auth/compute_engine/credentials.py", line 294, in _call_meta
data_identity_endpoint
six.raise_from(new_exc, caught_exc)
File "", line 3, in raise_from
google.auth.exceptions.RefreshError: ("Failed to retrieve http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/identity?audience=1087176421767-er001vugel6q9ottl7ihs8rp6963iv6o.apps.googleu
sercontent.com&format=full from the Google Compute Enginemetadata service. Status: 404 Response:\nb''", <google.auth.transport.requests._Response object at 0x7fe83917c048>)

[arithmetic1728]

@0yster I think I figured out the problem. Could you help me verify if the fix works for you?

(1) uninstall google-auth

python -m pip uninstall google-auth

install google-auth with the proposed fix

python -m pip install git+https://github.com/googleapis/google-auth-library-python.git@66fa9436bd73bac3afca345a70fbf4a99e3cfcb2#egg=google-auth

(2) run python dag_trigger.py again and see if that fixes the problem?

[0yster]

That looks like the solution! The lab seems to be working correctly now! Can you please let me know when the google-auth library is updated? The lab always pulls from the library each time a student runs it, so once it's ready, the lab is fixed. Thank you so much for your quick response. JoEllen Yannis Technical Curriculum Developer, Self-paced labs

…

On Fri, May 8, 2020 at 4:41 PM arithmetic1728 ***@***.***> wrote: @0yster <https://github.com/0yster> I think I figured out the problem. Could you help me verify the fix? (1) open the file /home/student_00_da8d23ee74ab/professional-services/examples/cloud-composer-examples/composer_http_post_example/venv/lib/python3.7/site-packages/google/oauth2/id_token.py (2) change line 215 to except (ImportError, exceptions.TransportError, exceptions.RefreshError): (3) run python dag_trigger.py again and see if that fixes the problem? If it works I will fix it in google-auth library and do a new release, then you can update your google-auth library. Thanks. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3712 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIT6OXJ4Z2WCC5HFWJCUFM3RQRU6HANCNFSM4M4N2HFQ> .

[arithmetic1728]

@0yster I will release google-auth version 1.14.3 asap (today or tomorrow) and let you know once it is released. Thank you for confirming the solution works!

[leahecole]

@arithmetic1728 thanks for the fix! Does this script need to be updated or is the google-auth update sufficient?

[arithmetic1728]

@leahecole I also need to update the google-auth version in requirements.txt. Could you take a look at #3726?