-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Adding Bigtable docs. #1547
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding Bigtable docs. #1547
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| Base for Everything | ||
| =================== | ||
|
|
||
| To use the API, the :class:`Client <gcloud.bigtable.client.Client>` | ||
| class defines a high-level interface which handles authorization | ||
| and creating other objects: | ||
|
|
||
| .. code:: python | ||
|
|
||
| from gcloud.bigtable.client import Client | ||
| client = Client() | ||
|
|
||
| Long-lived Defaults | ||
| ------------------- | ||
|
|
||
| When creating a :class:`Client <gcloud.bigtable.client.Client>`, the | ||
| ``user_agent`` and ``timeout_seconds`` arguments have sensible | ||
| defaults | ||
| (:data:`DEFAULT_USER_AGENT <gcloud.bigtable.client.DEFAULT_USER_AGENT>` and | ||
| :data:`DEFAULT_TIMEOUT_SECONDS <gcloud.bigtable.client.DEFAULT_TIMEOUT_SECONDS>`). | ||
| However, you may over-ride them and these will be used throughout all API | ||
| requests made with the ``client`` you create. | ||
|
|
||
| Authorization | ||
| ------------- | ||
|
|
||
| - For an overview of authentication in ``gcloud-python``, | ||
| see :doc:`gcloud-auth`. | ||
|
|
||
| - In addition to any authentication configuration, you can also set the | ||
| :envvar:`GCLOUD_PROJECT` environment variable for the project you'd like | ||
| to interact with. If you are Google App Engine or Google Compute Engine | ||
| this will be detected automatically. (Setting this environment | ||
| variable is not required, you may instead pass the ``project`` explicitly | ||
| when constructing a :class:`Client <gcloud.storage.client.Client>`). | ||
|
|
||
| - After configuring your environment, create a | ||
| :class:`Client <gcloud.storage.client.Client>` | ||
|
|
||
| .. code:: | ||
|
|
||
| >>> from gcloud import bigtable | ||
| >>> client = bigtable.Client() | ||
|
|
||
| or pass in ``credentials`` and ``project`` explicitly | ||
|
|
||
| .. code:: | ||
|
|
||
| >>> from gcloud import bigtable | ||
| >>> client = bigtable.Client(project='my-project', credentials=creds) | ||
|
|
||
| .. tip:: | ||
|
|
||
| Be sure to use the **Project ID**, not the **Project Number**. | ||
|
|
||
| Admin API Access | ||
| ---------------- | ||
|
|
||
| If you'll be using your client to make `Cluster Admin`_ and `Table Admin`_ | ||
| API requests, you'll need to pass the ``admin`` argument: | ||
|
|
||
| .. code:: python | ||
|
|
||
| client = bigtable.Client(admin=True) | ||
|
|
||
| Read-Only Mode | ||
| -------------- | ||
|
|
||
| If on the other hand, you only have (or want) read access to the data, | ||
| you can pass the ``read_only`` argument: | ||
|
|
||
| .. code:: python | ||
|
|
||
| client = bigtable.Client(read_only=True) | ||
|
|
||
| This will ensure that the | ||
| :data:`READ_ONLY_SCOPE <gcloud.bigtable.client.READ_ONLY_SCOPE>` is used | ||
| for API requests (so any accidental requests that would modify data will | ||
| fail). | ||
|
|
||
| Next Step | ||
| --------- | ||
|
|
||
| After a :class:`Client <gcloud.bigtable.client.Client>`, the next highest-level | ||
| object is a :class:`Cluster <gcloud.bigtable.cluster.Cluster>`. You'll need | ||
| one before you can interact with tables or data. | ||
|
|
||
| Head next to learn about the :doc:`bigtable-cluster-api`. | ||
|
|
||
| .. _Cluster Admin: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/tree/master/bigtable-protos/src/main/proto/google/bigtable/admin/cluster/v1 | ||
| .. _Table Admin: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/tree/master/bigtable-protos/src/main/proto/google/bigtable/admin/table/v1 | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| Client | ||
| ~~~~~~ | ||
|
|
||
| .. automodule:: gcloud.bigtable.client | ||
| :members: | ||
| :undoc-members: | ||
| :show-inheritance: |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,179 @@ | ||
| Cluster Admin API | ||
| ================= | ||
|
|
||
| After creating a :class:`Client <gcloud.bigtable.client.Client>`, you can | ||
| interact with individual clusters, groups of clusters or available | ||
| zones for a project. | ||
|
|
||
| List Clusters | ||
| ------------- | ||
|
|
||
| If you want a comprehensive list of all existing clusters, make a | ||
| `ListClusters`_ API request with | ||
| :meth:`Client.list_clusters() <gcloud.bigtable.client.Client.list_clusters>`: | ||
|
|
||
| .. code:: python | ||
|
|
||
| clusters = client.list_clusters() | ||
|
|
||
| List Zones | ||
| ---------- | ||
|
|
||
| If you aren't sure which ``zone`` to create a cluster in, find out | ||
| which zones your project has access to with a `ListZones`_ API request | ||
| with :meth:`Client.list_zones() <gcloud.bigtable.client.Client.list_zones>`: | ||
|
|
||
| .. code:: python | ||
|
|
||
| zones = client.list_zones() | ||
|
|
||
| You can choose a :class:`string <str>` from among the result to pass to | ||
| the :class:`Cluster <gcloud.bigtable.cluster.Cluster>` constructor. | ||
|
|
||
| The available zones (as of February 2016) are | ||
|
|
||
| .. code:: python | ||
|
|
||
| >>> zones | ||
| [u'asia-east1-b', u'europe-west1-c', u'us-central1-c', u'us-central1-b'] | ||
|
|
||
| Cluster Factory | ||
| --------------- | ||
|
|
||
| To create a :class:`Cluster <gcloud.bigtable.cluster.Cluster>` object: | ||
|
|
||
| .. code:: python | ||
|
|
||
| cluster = client.cluster(zone, cluster_id, | ||
| display_name=display_name, | ||
| serve_nodes=3) | ||
|
|
||
| Both ``display_name`` and ``serve_nodes`` are optional. When not provided, | ||
| ``display_name`` defaults to the ``cluster_id`` value and ``serve_nodes`` | ||
| defaults to the minimum allowed: | ||
| :data:`DEFAULT_SERVE_NODES <gcloud.bigtable.cluster.DEFAULT_SERVE_NODES>`. | ||
|
|
||
| Even if this :class:`Cluster <gcloud.bigtable.cluster.Cluster>` already | ||
| has been created with the API, you'll want this object to use as a | ||
| parent of a :class:`Table <gcloud.bigtable.table.Table>` just as the | ||
| :class:`Client <gcloud.bigtable.client.Client>` is used as the parent of | ||
| a :class:`Cluster <gcloud.bigtable.cluster.Cluster>`. | ||
|
|
||
| Create a new Cluster | ||
| -------------------- | ||
|
|
||
| After creating the cluster object, make a `CreateCluster`_ API request | ||
| with :meth:`create() <gcloud.bigtable.cluster.Cluster.create>`: | ||
|
|
||
| .. code:: python | ||
|
|
||
| cluster.display_name = 'My very own cluster' | ||
| cluster.create() | ||
|
|
||
| If you would like more than the minimum number of nodes | ||
| (:data:`DEFAULT_SERVE_NODES <gcloud.bigtable.cluster.DEFAULT_SERVE_NODES>`) | ||
| in your cluster: | ||
|
|
||
| .. code:: python | ||
|
|
||
| cluster.serve_nodes = 10 | ||
| cluster.create() | ||
|
|
||
| Check on Current Operation | ||
| -------------------------- | ||
|
|
||
| .. note:: | ||
|
|
||
| When modifying a cluster (via a `CreateCluster`_, `UpdateCluster`_ or | ||
| `UndeleteCluster`_ request), the Bigtable API will return a long-running | ||
| `Operation`_. This will be stored on the object after each of | ||
| :meth:`create() <gcloud.bigtable.cluster.Cluster.create>`, | ||
| :meth:`update() <gcloud.bigtable.cluster.Cluster.update>` and | ||
| :meth:`undelete() <gcloud.bigtable.cluster.Cluster.undelete>` are called. | ||
|
|
||
| You can check if a long-running operation (for a | ||
| :meth:`create() <gcloud.bigtable.cluster.Cluster.create>`, | ||
| :meth:`update() <gcloud.bigtable.cluster.Cluster.update>` or | ||
| :meth:`undelete() <gcloud.bigtable.cluster.Cluster.undelete>`) has finished | ||
| by making a `GetOperation`_ request with | ||
| :meth:`Operation.finished() <gcloud.bigtable.cluster.Operation.finished>`: | ||
|
|
||
| .. code:: python | ||
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
Sorry, something went wrong. |
||
|
|
||
| >>> operation = cluster.create() | ||
| >>> operation.finished() | ||
| True | ||
|
|
||
| .. note:: | ||
|
|
||
| Once an :class:`Operation <gcloud.bigtable.cluster.Operation>` object | ||
| has returned :data:`True` from | ||
| :meth:`finished() <gcloud.bigtable.cluster.Operation.finished>`, the | ||
| object should not be re-used. Subsequent calls to | ||
| :meth:`finished() <gcloud.bigtable.cluster.Operation.finished>` | ||
| will result in a :class:`ValueError <exceptions.ValueError>`. | ||
|
|
||
| Get metadata for an existing Cluster | ||
| ------------------------------------ | ||
|
|
||
| After creating the cluster object, make a `GetCluster`_ API request | ||
| with :meth:`reload() <gcloud.bigtable.cluster.Cluster.reload>`: | ||
|
|
||
| .. code:: python | ||
|
|
||
| cluster.reload() | ||
|
|
||
| This will load ``serve_nodes`` and ``display_name`` for the existing | ||
| ``cluster`` in addition to the ``cluster_id``, ``zone`` and ``project`` | ||
| already set on the :class:`Cluster <gcloud.bigtable.cluster.Cluster>` object. | ||
|
|
||
| Update an existing Cluster | ||
| -------------------------- | ||
|
|
||
| After creating the cluster object, make an `UpdateCluster`_ API request | ||
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
Sorry, something went wrong. |
||
| with :meth:`update() <gcloud.bigtable.cluster.Cluster.update>`: | ||
|
|
||
| .. code:: python | ||
|
|
||
| client.display_name = 'New display_name' | ||
| cluster.update() | ||
|
|
||
| Delete an existing Cluster | ||
| -------------------------- | ||
|
|
||
| Make a `DeleteCluster`_ API request with | ||
| :meth:`delete() <gcloud.bigtable.cluster.Cluster.delete>`: | ||
|
|
||
| .. code:: python | ||
|
|
||
| cluster.delete() | ||
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
Sorry, something went wrong. |
||
|
|
||
| Undelete a deleted Cluster | ||
| -------------------------- | ||
|
|
||
| Make an `UndeleteCluster`_ API request with | ||
| :meth:`undelete() <gcloud.bigtable.cluster.Cluster.undelete>`: | ||
|
|
||
| .. code:: python | ||
|
|
||
| cluster.undelete() | ||
|
|
||
| Next Step | ||
| --------- | ||
|
|
||
| Now we go down the hierarchy from | ||
| :class:`Cluster <gcloud.bigtable.cluster.Cluster>` to a | ||
| :class:`Table <gcloud.bigtable.table.Table>`. | ||
|
|
||
| Head next to learn about the :doc:`bigtable-table-api`. | ||
|
|
||
| .. _Cluster Admin API: https://cloud.google.com/bigtable/docs/creating-cluster | ||
| .. _CreateCluster: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/blob/2aae624081f652427052fb652d3ae43d8ac5bf5a/bigtable-protos/src/main/proto/google/bigtable/admin/cluster/v1/bigtable_cluster_service.proto#L66-L68 | ||
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
Sorry, something went wrong. |
||
| .. _GetCluster: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/blob/2aae624081f652427052fb652d3ae43d8ac5bf5a/bigtable-protos/src/main/proto/google/bigtable/admin/cluster/v1/bigtable_cluster_service.proto#L38-L40 | ||
| .. _UpdateCluster: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/blob/2aae624081f652427052fb652d3ae43d8ac5bf5a/bigtable-protos/src/main/proto/google/bigtable/admin/cluster/v1/bigtable_cluster_service.proto#L93-L95 | ||
| .. _DeleteCluster: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/blob/2aae624081f652427052fb652d3ae43d8ac5bf5a/bigtable-protos/src/main/proto/google/bigtable/admin/cluster/v1/bigtable_cluster_service.proto#L109-L111 | ||
| .. _ListZones: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/blob/2aae624081f652427052fb652d3ae43d8ac5bf5a/bigtable-protos/src/main/proto/google/bigtable/admin/cluster/v1/bigtable_cluster_service.proto#L33-L35 | ||
| .. _ListClusters: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/blob/2aae624081f652427052fb652d3ae43d8ac5bf5a/bigtable-protos/src/main/proto/google/bigtable/admin/cluster/v1/bigtable_cluster_service.proto#L44-L46 | ||
| .. _GetOperation: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/blob/2aae624081f652427052fb652d3ae43d8ac5bf5a/bigtable-protos/src/main/proto/google/longrunning/operations.proto#L43-L45 | ||
| .. _UndeleteCluster: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/blob/2aae624081f652427052fb652d3ae43d8ac5bf5a/bigtable-protos/src/main/proto/google/bigtable/admin/cluster/v1/bigtable_cluster_service.proto#L126-L128 | ||
| .. _Operation: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/blob/2aae624081f652427052fb652d3ae43d8ac5bf5a/bigtable-protos/src/main/proto/google/longrunning/operations.proto#L73-L102 | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| Cluster | ||
| ~~~~~~~ | ||
|
|
||
| .. automodule:: gcloud.bigtable.cluster | ||
| :members: | ||
| :undoc-members: | ||
| :show-inheritance: |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| Column Families | ||
| =============== | ||
|
|
||
| When creating a | ||
| :class:`ColumnFamily <gcloud.bigtable.column_family.ColumnFamily>`, it is | ||
| possible to set garbage collection rules for expired data. | ||
|
|
||
| By setting a rule, cells in the table matching the rule will be deleted | ||
| during periodic garbage collection (which executes opportunistically in the | ||
| background). | ||
|
|
||
| The types | ||
| :class:`MaxAgeGCRule <gcloud.bigtable.column_family.MaxAgeGCRule>`, | ||
| :class:`MaxVersionsGCRule <gcloud.bigtable.column_family.MaxVersionsGCRule>`, | ||
| :class:`GarbageCollectionRuleUnion <gcloud.bigtable.column_family.GarbageCollectionRuleUnion>` and | ||
| :class:`GarbageCollectionRuleIntersection <gcloud.bigtable.column_family.GarbageCollectionRuleIntersection>` | ||
| can all be used as the optional ``gc_rule`` argument in the | ||
| :class:`ColumnFamily <gcloud.bigtable.column_family.ColumnFamily>` | ||
| constructor. This value is then used in the | ||
| :meth:`create() <gcloud.bigtable.column_family.ColumnFamily.create>` and | ||
| :meth:`update() <gcloud.bigtable.column_family.ColumnFamily.update>` methods. | ||
|
|
||
| These rules can be nested arbitrarily, with a | ||
| :class:`MaxAgeGCRule <gcloud.bigtable.column_family.MaxAgeGCRule>` or | ||
| :class:`MaxVersionsGCRule <gcloud.bigtable.column_family.MaxVersionsGCRule>` | ||
| at the lowest level of the nesting: | ||
|
|
||
| .. code:: python | ||
|
|
||
| import datetime | ||
|
|
||
| max_age = datetime.timedelta(days=3) | ||
| rule1 = MaxAgeGCRule(max_age) | ||
| rule2 = MaxVersionsGCRule(1) | ||
|
|
||
| # Make a composite that matches anything older than 3 days **AND** | ||
| # with more than 1 version. | ||
| rule3 = GarbageCollectionIntersection(rules=[rule1, rule2]) | ||
|
|
||
| # Make another composite that matches our previous intersection | ||
| # **OR** anything that has more than 3 versions. | ||
| rule4 = GarbageCollectionRule(max_num_versions=3) | ||
| rule5 = GarbageCollectionUnion(rules=[rule3, rule4]) | ||
|
|
||
| ---- | ||
|
|
||
| .. automodule:: gcloud.bigtable.column_family | ||
| :members: | ||
| :undoc-members: | ||
| :show-inheritance: |
This comment was marked as spam.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
This comment was marked as spam.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.