Skip to content

Commit 47baea9

Browse files
waprinwaprin
authored
Add Bigtable Autoscaler sample (#943)
* Add Bigtable Autoscaler sample * update readme * add return type to docstring * remove trailing whitespace * Fix main call * fix instance flag name * style and jonwayne review * lint and jonwayne * fix tests * fix tests * fix tests * fix reqs
1 parent a6df5b2 commit 47baea9

File tree

7 files changed

+475
-0
lines changed

7 files changed

+475
-0
lines changed

‎bigtable/autoscaler/README.rst‎

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
.. This file is automatically generated. Do not edit this file directly.
2+
3+
Google Cloud Bigtable Python Samples
4+
===============================================================================
5+
6+
This directory contains samples for Google Cloud Bigtable. `Google Cloud Bigtable`_ is Google's NoSQL Big Data database service. It's the same database that powers many core Google services, including Search, Analytics, Maps, and Gmail.
7+
8+
9+
This sample demonstrates using `Stackdriver monitoring`_,
10+
to scale Cloud Bigtable based on CPU usage.
11+
12+
.. Stackdriver Monitoring: http://cloud.google.com/monitoring/docs
13+
14+
15+
.. _Google Cloud Bigtable: https://cloud.google.com/bigtable/docs
16+
17+
Setup
18+
-------------------------------------------------------------------------------
19+
20+
21+
Authentication
22+
++++++++++++++
23+
24+
Authentication is typically done through `Application Default Credentials`_,
25+
which means you do not have to change the code to authenticate as long as
26+
your environment has credentials. You have a few options for setting up
27+
authentication:
28+
29+
#. When running locally, use the `Google Cloud SDK`_
30+
31+
.. code-block:: bash
32+
33+
gcloud auth application-default login
34+
35+
36+
#. When running on App Engine or Compute Engine, credentials are already
37+
set-up. However, you may need to configure your Compute Engine instance
38+
with `additional scopes`_.
39+
40+
#. You can create a `Service Account key file`_. This file can be used to
41+
authenticate to Google Cloud Platform services from any environment. To use
42+
the file, set the ``GOOGLE_APPLICATION_CREDENTIALS`` environment variable to
43+
the path to the key file, for example:
44+
45+
.. code-block:: bash
46+
47+
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service_account.json
48+
49+
.. _Application Default Credentials: https://cloud.google.com/docs/authentication#getting_credentials_for_server-centric_flow
50+
.. _additional scopes: https://cloud.google.com/compute/docs/authentication#using
51+
.. _Service Account key file: https://developers.google.com/identity/protocols/OAuth2ServiceAccount#creatinganaccount
52+
53+
Install Dependencies
54+
++++++++++++++++++++
55+
56+
#. Install `pip`_ and `virtualenv`_ if you do not already have them.
57+
58+
#. Create a virtualenv. Samples are compatible with Python 2.7 and 3.4+.
59+
60+
.. code-block:: bash
61+
62+
$ virtualenv env
63+
$ source env/bin/activate
64+
65+
#. Install the dependencies needed to run the samples.
66+
67+
.. code-block:: bash
68+
69+
$ pip install -r requirements.txt
70+
71+
.. _pip: https://pip.pypa.io/
72+
.. _virtualenv: https://virtualenv.pypa.io/
73+
74+
Samples
75+
-------------------------------------------------------------------------------
76+
77+
Autoscaling example
78+
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
79+
80+
81+
82+
To run this sample:
83+
84+
.. code-block:: bash
85+
86+
$ python autoscaler.py
87+
88+
usage: autoscaler.py [-h] [--high_cpu_threshold HIGH_CPU_THRESHOLD]
89+
[--low_cpu_threshold LOW_CPU_THRESHOLD]
90+
[--short_sleep SHORT_SLEEP] [--long_sleep LONG_SLEEP]
91+
instance_id
92+
93+
Scales Bigtable clusters based on CPU usage.
94+
95+
positional arguments:
96+
instance_id ID of the Cloud Bigtable instance to connect to.
97+
98+
optional arguments:
99+
-h, --help show this help message and exit
100+
--high_cpu_threshold HIGH_CPU_THRESHOLD
101+
If Bigtable CPU usages is above this threshold, scale
102+
up
103+
--low_cpu_threshold LOW_CPU_THRESHOLD
104+
If Bigtable CPU usages is above this threshold, scale
105+
up
106+
--short_sleep SHORT_SLEEP
107+
How long to sleep in seconds between checking metrics
108+
after no scale operation
109+
--long_sleep LONG_SLEEP
110+
How long to sleep in seconds between checking metrics
111+
after a scaling operation
112+
113+
114+
115+
116+
The client library
117+
-------------------------------------------------------------------------------
118+
119+
This sample uses the `Google Cloud Client Library for Python`_.
120+
You can read the documentation for more details on API usage and use GitHub
121+
to `browse the source`_ and `report issues`_.
122+
123+
.. Google Cloud Client Library for Python:
124+
https://googlecloudplatform.github.io/google-cloud-python/
125+
.. browse the source:
126+
https://github.com/GoogleCloudPlatform/google-cloud-python
127+
.. report issues:
128+
https://github.com/GoogleCloudPlatform/google-cloud-python/issues
129+
130+
131+
.. _Google Cloud SDK: https://cloud.google.com/sdk/

‎bigtable/autoscaler/README.rst.in‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# This file is used to generate README.rst
2+
3+
product:
4+
name: Google Cloud Bigtable
5+
short_name: Cloud Bigtable
6+
url: https://cloud.google.com/bigtable/docs
7+
description: >
8+
`Google Cloud Bigtable`_ is Google's NoSQL Big Data database service. It's
9+
the same database that powers many core Google services, including Search,
10+
Analytics, Maps, and Gmail.
11+
12+
description: |
13+
This sample demonstrates using `Stackdriver monitoring`_,
14+
to scale Cloud Bigtable based on CPU usage.
15+
16+
.. Stackdriver Monitoring: http://cloud.google.com/monitoring/docs
17+
18+
setup:
19+
- auth
20+
- install_deps
21+
22+
samples:
23+
- name: Autoscaling example
24+
file: autoscaler.py
25+
show_help: true
26+
27+
cloud_client_library: true

‎bigtable/autoscaler/autoscaler.py‎

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
# Copyright 2017 Google Inc.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""Sample that demonstrates how to use Bigtable Stackdriver metrics to
16+
autoscale Google Cloud Bigtable."""
17+
18+
import argparse
19+
import time
20+
21+
from google.cloud import bigtable
22+
from google.cloud import monitoring
23+
24+
import strategies
25+
26+
27+
def get_cpu_load():
28+
"""Returns the most recent Bigtable CPU load measurement.
29+
30+
Returns:
31+
float: The most recent Bigtable CPU usage metric
32+
"""
33+
client = monitoring.Client()
34+
query = client.query('bigtable.googleapis.com/cluster/cpu_load', minutes=5)
35+
time_series = list(query)
36+
recent_time_series = time_series[0]
37+
return recent_time_series.points[0].value
38+
39+
40+
def scale_bigtable(bigtable_instance, bigtable_cluster, scale_up):
41+
"""Scales the number of Bigtable nodes up or down.
42+
43+
Edits the number of nodes in the Bigtable cluster to be increased
44+
or decreased, depending on the `scale_up` boolean argument. Currently
45+
the `incremental` strategy from `strategies.py` is used.
46+
47+
48+
Args:
49+
bigtable_instance (str): Cloud Bigtable instance id to scale
50+
bigtable_cluster (str): Cloud Bigtable cluster id to scale
51+
scale_up (bool): If true, scale up, otherwise scale down
52+
"""
53+
bigtable_client = bigtable.Client(admin=True)
54+
instance = bigtable_client.instance(bigtable_instance)
55+
instance.reload()
56+
57+
cluster = instance.cluster(bigtable_cluster)
58+
cluster.reload()
59+
60+
current_node_count = cluster.serve_nodes
61+
62+
if current_node_count <= 3 and not scale_up:
63+
# Can't downscale lower than 3 nodes
64+
return
65+
66+
if scale_up:
67+
strategies_dict = strategies.UPSCALE_STRATEGIES
68+
else:
69+
strategies_dict = strategies.DOWNSCALE_STRATEGIES
70+
71+
strategy = strategies_dict['incremental']
72+
new_node_count = strategy(cluster.serve_nodes)
73+
cluster.serve_nodes = new_node_count
74+
cluster.update()
75+
print('Scaled from {} up to {} nodes.'.format(
76+
current_node_count, new_node_count))
77+
78+
79+
def main(
80+
bigtable_instance,
81+
bigtable_cluster,
82+
high_cpu_threshold,
83+
low_cpu_threshold,
84+
short_sleep,
85+
long_sleep):
86+
"""Main loop runner that autoscales Bigtable.
87+
88+
Args:
89+
bigtable_instance (str): Cloud Bigtable instance id to autoscale
90+
high_cpu_threshold (float): If CPU is higher than this, scale up.
91+
low_cpu_threshold (float): If CPU is higher than this, scale down.
92+
short_sleep (int): How long to sleep after no operation
93+
long_sleep (int): How long to sleep after the cluster nodes are
94+
changed
95+
"""
96+
cluster_cpu = get_cpu_load()
97+
print('Detected cpu of {}'.format(cluster_cpu))
98+
if cluster_cpu > high_cpu_threshold:
99+
scale_bigtable(bigtable_instance, bigtable_cluster, True)
100+
time.sleep(long_sleep)
101+
elif cluster_cpu < low_cpu_threshold:
102+
scale_bigtable(bigtable_instance, bigtable_cluster, False)
103+
time.sleep(short_sleep)
104+
else:
105+
print('CPU within threshold, sleeping.')
106+
time.sleep(short_sleep)
107+
108+
109+
if __name__ == '__main__':
110+
parser = argparse.ArgumentParser(
111+
description='Scales Bigtable clusters based on CPU usage.')
112+
parser.add_argument(
113+
'bigtable_instance',
114+
help='ID of the Cloud Bigtable instance to connect to.')
115+
parser.add_argument(
116+
'bigtable_cluster',
117+
help='ID of the Cloud Bigtable cluster to connect to.')
118+
parser.add_argument(
119+
'--high_cpu_threshold',
120+
help='If Bigtable CPU usages is above this threshold, scale up',
121+
default=0.6)
122+
parser.add_argument(
123+
'--low_cpu_threshold',
124+
help='If Bigtable CPU usages is above this threshold, scale up',
125+
default=0.2)
126+
parser.add_argument(
127+
'--short_sleep',
128+
help='How long to sleep in seconds between checking metrics after no '
129+
'scale operation',
130+
default=60)
131+
parser.add_argument(
132+
'--long_sleep',
133+
help='How long to sleep in seconds between checking metrics after a '
134+
'scaling operation',
135+
default=60 * 10)
136+
args = parser.parse_args()
137+
138+
while True:
139+
main(
140+
args.bigtable_instance,
141+
args.bigtable_cluster,
142+
float(args.high_cpu_threshold),
143+
float(args.low_cpu_threshold),
144+
int(args.short_sleep),
145+
int(args.long_sleep))

‎bigtable/autoscaler/autoscaler_test.py‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# Copyright 2017 Google Inc.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""Unit and system tests for autoscaler.py"""
16+
17+
import os
18+
import time
19+
20+
from google.cloud import bigtable
21+
from mock import patch
22+
23+
from autoscaler import get_cpu_load
24+
from autoscaler import main
25+
from autoscaler import scale_bigtable
26+
27+
# tests assume instance and cluster have the same ID
28+
BIGTABLE_INSTANCE = os.environ['BIGTABLE_CLUSTER']
29+
30+
# System tests to verify API calls succeed
31+
32+
33+
def test_get_cpu_load():
34+
assert get_cpu_load() > 0.0
35+
36+
37+
def test_scale_bigtable():
38+
bigtable_client = bigtable.Client(admin=True)
39+
instance = bigtable_client.instance(BIGTABLE_INSTANCE)
40+
instance.reload()
41+
42+
cluster = instance.cluster(BIGTABLE_INSTANCE)
43+
cluster.reload()
44+
original_node_count = cluster.serve_nodes
45+
46+
scale_bigtable(BIGTABLE_INSTANCE, BIGTABLE_INSTANCE, True)
47+
48+
time.sleep(3)
49+
cluster.reload()
50+
51+
new_node_count = cluster.serve_nodes
52+
assert (new_node_count == (original_node_count + 2))
53+
54+
scale_bigtable(BIGTABLE_INSTANCE, BIGTABLE_INSTANCE, False)
55+
time.sleep(3)
56+
cluster.reload()
57+
final_node_count = cluster.serve_nodes
58+
assert final_node_count == original_node_count
59+
60+
61+
# Unit test for logic
62+
63+
@patch('time.sleep')
64+
@patch('autoscaler.get_cpu_load')
65+
@patch('autoscaler.scale_bigtable')
66+
def test_main(scale_bigtable, get_cpu_load, sleep):
67+
SHORT_SLEEP = 5
68+
LONG_SLEEP = 10
69+
get_cpu_load.return_value = 0.5
70+
71+
main(BIGTABLE_INSTANCE, BIGTABLE_INSTANCE, 0.6, 0.3, SHORT_SLEEP,
72+
LONG_SLEEP)
73+
scale_bigtable.assert_not_called()
74+
scale_bigtable.reset_mock()
75+
76+
get_cpu_load.return_value = 0.7
77+
main(BIGTABLE_INSTANCE, BIGTABLE_INSTANCE, 0.6, 0.3, SHORT_SLEEP,
78+
LONG_SLEEP)
79+
scale_bigtable.assert_called_once_with(BIGTABLE_INSTANCE,
80+
BIGTABLE_INSTANCE, True)
81+
scale_bigtable.reset_mock()
82+
83+
get_cpu_load.return_value = 0.2
84+
main(BIGTABLE_INSTANCE, BIGTABLE_INSTANCE, 0.6, 0.3, SHORT_SLEEP,
85+
LONG_SLEEP)
86+
scale_bigtable.assert_called_once_with(BIGTABLE_INSTANCE,
87+
BIGTABLE_INSTANCE, False)
88+
89+
scale_bigtable.reset_mock()

‎bigtable/autoscaler/requirements.txt‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
google-cloud-bigtable==0.24.0
2+
google-cloud-monitoring==0.24.0

0 commit comments

Comments
 (0)