Skip to content

Cannot reload BigQuery job status or results in sync queries #1551

@ernestoalejo

Description

@ernestoalejo
  • The documentation here is not valid as it uses an undeclared variable (job) and a function that doesn't exist in sync queries (query.reload()).
  • When I tried to use query.job I found each time it returns a different job instance, which is confusing.
  • After caching job = query.job in a variable I was able to use job.state != 'DONE' and job.reload() to keep track of the job status instead of query.completed like documented (literally job.completed); but I had to call query.fetch_data() to manually extract the rows from the query. This makes the Query class a simple factory method to build new jobs.

I'm not sure if this is only a documentation issue or if the QueryJob class lacks some important methods to be more than a factory for new jobs.

Original code in the documentation:

    query = """
      SELECT count(*) AS age_count FROM dataset_name.person_ages
    """
    query = client.run_sync_query(query)
    query.timeout_ms = 1000
    query.run()  # API request
    retry_count = 100
    while retry_count > 0 and not job.complete:
      retry_count -= 1
      time.sleep(10)
      query.reload()  # API request
    print query.rows

Modified code:

    q = """
      SELECT count(*) AS age_count FROM dataset_name.person_ages
    """
    query = client.run_sync_query(q)
    query.timeout_ms = 1000
    query.run()  # API request

    job = query.job

    retry_count = 100
    while retry_count > 0 and not job.state != 'DONE':
      retry_count -= 1
      time.sleep(10)
      job.reload()  # API request

    (rows, _, _) = query.fetch_data()
    print rows

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the BigQuery API.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions