Skip to content

Dashboards API v2 #85

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 59 commits into from
Apr 14, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
d35fd0a
Make example more generic
davideschiera Mar 28, 2019
49467dc
Fix dashboard/panel scope definition
davideschiera Mar 28, 2019
0fed9d9
Add note and refactor name
davideschiera Mar 28, 2019
a7d34cd
Make API endpoints customizable
davideschiera Mar 28, 2019
b5f0691
Move v1 to separate module
davideschiera Mar 28, 2019
394622c
Support dashboards API v2
davideschiera Mar 28, 2019
b0f237a
Update examples to work with dashboards API v2
davideschiera Mar 28, 2019
7450251
Fix import
davideschiera Mar 28, 2019
56fc518
Add versioning to dashboard backups
davideschiera Mar 29, 2019
b11f777
Fix URL
davideschiera Mar 29, 2019
37be901
Revert temporary local change
davideschiera Mar 29, 2019
2604a24
Apply reviews
davideschiera Mar 29, 2019
62e11b9
Add show cases/tests
davideschiera Mar 29, 2019
7211dfb
Improve scope handling
davideschiera Mar 29, 2019
b47139c
Missing new line
davideschiera Mar 29, 2019
3ff5af8
Fix scope
davideschiera Mar 29, 2019
1ca4656
Revert "Fix scope"
davideschiera Mar 29, 2019
b2f4600
Accept some invalid scopes for now
davideschiera Mar 29, 2019
312ad79
Fix dashboard save
davideschiera Mar 29, 2019
b16bb17
Merge branch 'fix-dashboards-lib' into dashboards-api-v2
davideschiera Apr 1, 2019
3bc5205
Scaffolding dashboard conversions
davideschiera Mar 29, 2019
0c1f1d2
Initial implementation of v1 => v2 migration
davideschiera Apr 1, 2019
efe6a32
Accept panels without scope
davideschiera Apr 1, 2019
e2831be
Merge branch 'fix-dashboards-lib' into dashboards-api-v2
davideschiera Apr 1, 2019
ce26a83
Scaffolding dashboard conversions
davideschiera Mar 29, 2019
88dc25a
Initial implementation of v1 => v2 migration
davideschiera Apr 1, 2019
387e011
Fix migration
davideschiera Apr 1, 2019
1b50dbe
Add migration example
davideschiera Apr 1, 2019
1b85df8
Merge remote-tracking branch 'origin/dashboard-convert-v1-to-v2' into…
davideschiera Apr 1, 2019
92d4f05
Fixes and support for scope migration
davideschiera Apr 2, 2019
49c4bee
Temporary change
davideschiera Apr 2, 2019
aa6aeac
More fixes
davideschiera Apr 2, 2019
22f1bae
Comment
davideschiera Apr 2, 2019
900ca41
Support "not x = y" and "not x != y"
davideschiera Apr 8, 2019
49c30e0
Merge branch 'fix-dashboards-lib' into dashboards-api-v2
davideschiera Apr 8, 2019
49a7274
Drop support for annotations in dashboards
davideschiera Apr 8, 2019
1e39476
Remove panel ID
davideschiera Apr 8, 2019
cd05735
Merge branch 'dashboards-api-v2' into dashboard-convert-v1-to-v2
davideschiera Apr 8, 2019
fbaf5d9
Remove panel ID
davideschiera Apr 8, 2019
f75d491
Convert scope conversion function to public/static
davideschiera Apr 10, 2019
333a172
Merge branch 'fix-dashboards-lib' into dashboards-api-v2
davideschiera Apr 10, 2019
d7585cf
Merge branch 'dashboards-api-v2' into dashboard-convert-v1-to-v2
davideschiera Apr 10, 2019
4ffa6a0
Fix fn name
davideschiera Apr 10, 2019
2f15a11
Merge branch 'dashboards-api-v2' into dashboard-convert-v1-to-v2
davideschiera Apr 10, 2019
dc89e1d
Fix fn name
davideschiera Apr 10, 2019
8512fa6
Improve script output
davideschiera Apr 10, 2019
8292983
Improve script
davideschiera Apr 10, 2019
5c29924
Keep as is the GridConfiguration for panels
papajulio Apr 11, 2019
fb57423
Merge branch 'master' into dashboards-api-v2
davideschiera Apr 11, 2019
436b005
Merge changes from #84 to API v1
davideschiera Apr 11, 2019
df8aabb
Handle old files
davideschiera Apr 11, 2019
f895357
Merge branch 'dashboards-api-v2' into dashboard-convert-v1-to-v2
davideschiera Apr 11, 2019
9dd861b
Drop layout property
davideschiera Apr 11, 2019
43c4dc8
Print all errors, not just the first one
davideschiera Apr 11, 2019
f98d049
Fix conversion of metrics and topology panels
davideschiera Apr 11, 2019
ce6999d
Allow dashboards created with API v1 to be uploaded to API v2
davideschiera Apr 11, 2019
4daa2fd
Fix import of topology panels
davideschiera Apr 12, 2019
c1b0a4f
Fix import of default dashboards
davideschiera Apr 12, 2019
87d1205
Sort metrics
davideschiera Apr 12, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions examples/create_dashboard.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])), '..'))
from sdcclient import SdcClient
from sdcclient import SdMonitorClient


#
Expand Down Expand Up @@ -42,7 +42,7 @@ def usage():
#
# Instantiate the SDC client
#
sdclient = SdcClient(sdc_token)
sdclient = SdMonitorClient(sdc_token)

#
# Create the new dashboard, applying to cassandra in production
Expand All @@ -55,7 +55,7 @@ def usage():
# in Sysdig Cloud Explore page.
# You can also refer to AWS tags by using "cloudProvider.tag.*" metadata or
# agent tags by using "agent.tag.*" metadata
dashboardFilter = "kubernetes.namespace.name = prod"
dashboardFilter = 'proc.name = "cassandra"'
print('Creating dashboard from view')
ok, res = sdclient.create_dashboard_from_view(dashboardName, viewName, dashboardFilter)
#
Expand All @@ -75,7 +75,7 @@ def usage():
# Name of the dashboard to copy
dashboardCopy = "Copy of {}".format(dashboardName)
# Filter to apply to the new dashboard. Same as above.
dashboardFilter = "kubernetes.namespace.name != prod"
dashboardFilter = 'proc.name != "cassandra"'

print('Creating dashboard from dashboard')
ok, res = sdclient.create_dashboard_from_dashboard(dashboardCopy, dashboardName, dashboardFilter)
Expand Down
14 changes: 7 additions & 7 deletions examples/dashboard.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])), '..'))
from sdcclient import SdcClient
from sdcclient import SdMonitorClient


#
Expand Down Expand Up @@ -39,7 +39,7 @@ def usage():
#
# Instantiate the SDC client
#
sdclient = SdcClient(sdc_token)
sdclient = SdMonitorClient(sdc_token)


#
Expand Down Expand Up @@ -77,10 +77,10 @@ def usage():
panel_name = 'CPU Over Time'
panel_type = 'timeSeries'
metrics = [
{'id': 'kubernetes.pod.name'},
{'id': 'proc.name'},
{'id': 'cpu.used.percent', 'aggregations': {'time': 'avg', 'group': 'avg'}}
]
scope = 'kubernetes.namespace.name = "dev" and kubernetes.replicationController.name = "cassandra"'
scope = 'proc.name = "cassandra"'
ok, res = sdclient.add_dashboard_panel(dashboard_configuration, panel_name, panel_type, metrics, scope=scope)

# Check the result
Expand All @@ -101,9 +101,9 @@ def usage():
{'id': 'host.hostName'},
{'id': 'cpu.used.percent', 'aggregations': {'time': 'avg', 'group': 'avg'}}
]
sort_by = {'metric': 'cpu.used.percent', 'mode': 'desc'}
sort_direction = 'desc'
limit = 10
ok, res = sdclient.add_dashboard_panel(dashboard_configuration, panel_name, panel_type, metrics, sort_by=sort_by, limit=limit)
ok, res = sdclient.add_dashboard_panel(dashboard_configuration, panel_name, panel_type, metrics, sort_direction=sort_direction, limit=limit)

# Check the result
if ok:
Expand Down Expand Up @@ -137,7 +137,7 @@ def usage():
#
# Remove a panel
#
ok, res = sdclient.remove_dashboard_panel(dashboard_configuration, 'CPU')
ok, res = sdclient.remove_dashboard_panel(dashboard_configuration, 'CPU Over Time')

# Check the result
if ok:
Expand Down
67 changes: 67 additions & 0 deletions examples/dashboard_backup_v1_restore_v2.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
#!/usr/bin/env python
#
# Save the first user dashboard to file and then use create_dashboard_from_file()
# to apply the stored dasboard again with a different filter.
#
import os
import sys
import json
sys.path.insert(
0, os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])), '..'))
from sdcclient import SdMonitorClient
from sdcclient import SdMonitorClientV1

#
# Parse arguments
#
if len(sys.argv) != 5:
print(
'usage: %s <sysdig-v1-url> <sysdig-v1-token> <sysdig-v2-url> <sysdig-v2-token>'
% sys.argv[0])
print(
'You can find your token at https://app.sysdigcloud.com/#/settings/user'
)
sys.exit(1)

sdc_v1_url = sys.argv[1]
sdc_v1_token = sys.argv[2]
sdc_v2_url = sys.argv[3]
sdc_v2_token = sys.argv[4]

#
# Instantiate the SDC client
#
sdclient_v2 = SdMonitorClient(sdc_v2_token, sdc_url=sdc_v2_url)
sdclient_v1 = SdMonitorClientV1(sdc_v1_token, sdc_url=sdc_v1_url)

#
# Serialize the first user dashboard to disk
#
ok, res = sdclient_v1.get_dashboards()

if not ok:
print(res)
sys.exit(1)

for dashboard in res['dashboards']:
file_name = '{}.json'.format(dashboard['id'])
print('Saving v1 dashboard {} to file {}...'.format(
dashboard['name'], file_name))
sdclient_v1.save_dashboard_to_file(dashboard, file_name)

print('Importing dashboard to v2...')
ok, res = sdclient_v2.create_dashboard_from_file(
u'import of {}'.format(dashboard['name']),
file_name,
None,
shared=dashboard['isShared'],
public=dashboard['isPublic'])

if ok:
print('Dashboard {} imported!'.format(dashboard['name']))
sdclient_v2.delete_dashboard(res['dashboard'])
else:
print('Dashboard {} import failed:'.format(dashboard['name']))
print(res)

print('\n')
9 changes: 4 additions & 5 deletions examples/dashboard_save_load.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
import sys
import json
sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])), '..'))
from sdcclient import SdcClient
from sdcclient import SdMonitorClient

#
# Parse arguments
Expand All @@ -22,7 +22,7 @@
#
# Instantiate the SDC client
#
sdclient = SdcClient(sdc_token)
sdclient = SdMonitorClient(sdc_token)

#
# Serialize the first user dashboard to disk
Expand All @@ -34,8 +34,7 @@
sys.exit(1)

if len(res[u'dashboards']) > 0:
with open('dashboard.json', 'w') as outf:
json.dump(res[u'dashboards'][0], outf)
sdclient.save_dashboard_to_file(res[u'dashboards'][0], 'dashboard.json')
else:
print('the user has no dashboards. Exiting.')
sys.exit(0)
Expand All @@ -44,7 +43,7 @@
# Now create the dashboard from the file. We use a filter for the Cassandra process
# as an example.
#
dashboardFilter = "proc.name = cassandra"
dashboardFilter = 'proc.name = "cassandra"'

ok, res = sdclient.create_dashboard_from_file('test dasboard from file', 'dashboard.json', dashboardFilter)

Expand Down
4 changes: 2 additions & 2 deletions examples/delete_dashboard.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])), '..'))
from sdcclient import SdcClient
from sdcclient import SdMonitorClient


#
Expand Down Expand Up @@ -38,7 +38,7 @@ def usage():
#
# Instantiate the SDC client
#
sdclient = SdcClient(sdc_token)
sdclient = SdMonitorClient(sdc_token)

#
# List the dashboards
Expand Down
12 changes: 5 additions & 7 deletions examples/download_dashboards.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
import zipfile
import json
sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])), '..'))
from sdcclient import SdcClient
from sdcclient import SdMonitorClient


def zipdir(path, ziph):
Expand Down Expand Up @@ -52,7 +52,7 @@ def cleanup_dir(path):
#
# Instantiate the SDC client
#
sdclient = SdcClient(sdc_token, sdc_url='https://app.sysdigcloud.com')
sdclient = SdMonitorClient(sdc_token)

#
# Fire the request.
Expand All @@ -76,11 +76,9 @@ def cleanup_dir(path):


for db in res['dashboards']:
file_path = os.path.join(sysdig_dashboard_dir, str(db['id']))
f = open(file_path, 'w')
f.write(json.dumps(db))
print("Name: %s, # Charts: %d" % (db['name'], len(db['items'])))
f.close()
sdclient.save_dashboard_to_file(db, os.path.join(sysdig_dashboard_dir, str(db['id'])))

print("Name: %s, # Charts: %d" % (db['name'], len(db['widgets'])))

zipf = zipfile.ZipFile(dashboard_state_file, 'w', zipfile.ZIP_DEFLATED)
zipdir(sysdig_dashboard_dir, zipf)
Expand Down
6 changes: 3 additions & 3 deletions examples/list_dashboards.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])), '..'))
from sdcclient import SdcClient
from sdcclient import SdMonitorClient

#
# Parse arguments
Expand All @@ -21,7 +21,7 @@
#
# Instantiate the SDC client
#
sdclient = SdcClient(sdc_token)
sdclient = SdMonitorClient(sdc_token)

#
# Fire the request.
Expand All @@ -36,4 +36,4 @@
sys.exit(1)

for db in res['dashboards']:
print("Name: %s, # Charts: %d" % (db['name'], len(db['items'] if 'items' in db else [])))
print("Name: %s, # Charts: %d" % (db['name'], len(db['widgets'] if 'widgets' in db else [])))
27 changes: 10 additions & 17 deletions examples/restore_dashboards.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
import zipfile
import json
sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])), '..'))
from sdcclient import SdcClient
from sdcclient import SdMonitorClient

#
# Parse arguments
Expand All @@ -24,32 +24,25 @@
#
# Instantiate the SDC client
#
sdclient = SdcClient(sdc_token)
sdclient = SdMonitorClient(sdc_token)

zipf = zipfile.ZipFile(dashboard_state_file, 'r')


dashboard_conf_items = ['showAsType', 'filterRoot', 'linkMetrics',
'singleTimeNavigation', 'gridConfiguration', 'responsive',
'nodesNoiseFilter', 'compareWith', 'format', 'linksNoiseFilter',
'filterProcesses', 'isLegendExpanded', 'inhertitTimeNavigation',
'schema', 'sortAscending', 'mapDataLimit', 'metrics', 'filterExtNodes',
'sorting', 'name', 'sourceExploreView', 'items', 'showAs', 'eventsFilter',
'timeMode', 'isShared', 'sourceDrilldownView', 'filterExpression']

for info in zipf.infolist():
data = zipf.read(info.filename)
try:
j = json.loads(data)
except ValueError:
print('Non-JSON item found in ZIP: ' + info.filename + ' (skipping)')
print('Invalid JSON file found in ZIP file ' + info.filename + ': skipping')
continue
k = {}
for item in j.keys():
if item in dashboard_conf_items:
k[item] = j[item]

ok, res = sdclient.create_dashboard_with_configuration(k)
#
# Handle old files
#
if 'dashboard' in j:
j = j['dashboard']

ok, res = sdclient.create_dashboard_with_configuration(j)
if ok:
print('Restored Dashboard named: ' + j['name'])
else:
Expand Down
1 change: 1 addition & 0 deletions sdcclient/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from sdcclient._monitor import SdcClient
from sdcclient._monitor import SdMonitorClient
from sdcclient._monitor_v1 import SdMonitorClientV1
from sdcclient._secure import SdSecureClient
from sdcclient._scanning import SdScanningClient
18 changes: 10 additions & 8 deletions sdcclient/_common.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,16 +38,18 @@ def _checkResponse(self, res):
return False

if 'errors' in j:
if 'message' in j['errors'][0]:
self.lasterr = j['errors'][0]['message']
error_msgs = []
for error in j['errors']:
error_msg = []
if 'message' in error:
error_msg.append(error['message'])

if 'reason' in j['errors'][0]:
if self.lasterr is not None:
self.lasterr += ' '
else:
self.lasrerr = ''
if 'reason' in error:
error_msg.append(error['reason'])

self.lasterr += j['errors'][0]['reason']
error_msgs.append(': '.join(error_msg))

self.lasterr = '\n'.join(error_msgs)
elif 'message' in j:
self.lasterr = j['message']
else:
Expand Down
Loading