-
Notifications
You must be signed in to change notification settings - Fork 6
DOC-730 | GenAI Suite #696
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Changes from all commits
Commits
Show all changes
56 commits
Select commit
Hold shift + click to select a range
0ad0a03
WIP
Simran-B 3ef992f
Fix image
Simran-B 0f5743a
adding natural language service documentation
mdawas b862dc2
remove dev guide of NL service; was added by mistake
mdawas 430efc0
add user-facing-documentation for ragloader
sachinsharma9780 5685e23
add user-facing-documentation for ragretriever
sachinsharma9780 f098711
WIP: Add platform overview content from #671
Simran-B 5b12ac5
add user-facing-documentation for genai
maxkernbach 7077922
add user-facing-documentation for arangodb-mlflow
maxkernbach 7dc6c94
Re-arrange content
Simran-B 354875f
Merge branch 'main' of https://github.com/arangodb/docs-hugo into dss
Simran-B d11e700
Fix links
Simran-B dc90cbb
Merge branch 'main' into dss
nerpaula e82b2f8
fix conflicts
nerpaula 66d5571
rework graph rag main page
nerpaula 1cbb171
Merge branch 'main' into dss
nerpaula 1fdc37e
formatting
nerpaula 4e233f1
Merge branch 'main' into dss
nerpaula e0a2f62
more formatting
nerpaula f440210
add initial docs for triton
nerpaula af94dce
mark txt2aql as draft (available only in q3)
nerpaula ef4e2f5
adjust triton, headlines, add links
nerpaula dbd2d0d
add info about private and public llms
nerpaula 362acb6
add initial graphrag notebook tutorial draft
nerpaula a3fbb95
Merge branch 'main' of https://github.com/arangodb/docs-hugo into dss
nerpaula 394b507
add links and examples to notebook servers page
nerpaula 835be85
update triton page
nerpaula 79dabb1
Merge branch 'main' of https://github.com/arangodb/docs-hugo into dss
Simran-B 1e6486a
Fix merge issue with AG notebooks
Simran-B 4e4eaea
Add Platform feature list
Simran-B 18ce876
Formatting
Simran-B 61bcc34
add platform tag
nerpaula 8739293
remove password field from tutorial, fix typo
nerpaula e8166e9
Single feature list for prerelease
Simran-B a9b9238
terminology: use importer and retriever
nerpaula a4b95db
add pre-release and get in touch note
nerpaula 0bb7c82
add more cross references
nerpaula c109016
add pre-release note to remaining pages
nerpaula 8d7ad8b
update GraphML screenshot
nerpaula 53c92c9
Platform requirements, installation, and interfaces
Simran-B 0e0b768
Merge branch 'dss' of https://github.com/arangodb/docs-hugo into dss
Simran-B e6e5f4b
API reference links
Simran-B f101a88
Update Data Science card, add Platform card
Simran-B b172447
Move GraphML under GenAI Suite in Platform overview
Simran-B 171fc23
Consistently capitalize it as GenAI Suite (uppercase S)
Simran-B c50eb46
Add GraphRAG limitations
Simran-B 687586c
Merge branch 'main' into dss
nerpaula 71cf734
add GraphRAG web interface docs
nerpaula f6a47a6
Apply suggestions from code review
nerpaula 20b0d03
Add and use kbd shortcode for keyboard shortcuts
Simran-B cd5e569
Partially address feedback
Simran-B 0cf6c78
move general KG definition
nerpaula ec4cf61
incorporate feedback
nerpaula 94ea0e8
Review
Simran-B 8fe99ce
Review
Simran-B 42735dc
Formatting
Simran-B File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,57 @@ | ||
--- | ||
title: Feature list of the ArangoDB Platform | ||
menuTitle: ArangoDB Platform | ||
weight: 10 | ||
description: >- | ||
The ArangoDB Platform is a scalable architecture that gets you all features | ||
of ArangoDB including graph-powered machine learning and GenAI as a single | ||
solution with a unified interface | ||
--- | ||
For in-depth information about the ArangoDB Platform as a whole and how to | ||
deploy and use it, see [The ArangoDB Platform](../../components/platform.md). | ||
|
||
## Architecture | ||
|
||
- **Core Database**: The ArangoDB database system forms the solid core | ||
of the ArangoDB Platform. | ||
|
||
- **Kubernetes**: An open-source container orchestration system for automating | ||
software deployment, scaling, and management designed by Google. It is the | ||
autopilot for operating ArangoDB clusters and the additional Platform services. | ||
|
||
- **Helm**: A package manager for Kubernetes that enables consistent, repeatable | ||
installations and version control. | ||
|
||
- **Envoy**: A high-performance service proxy that acts as the gateway for the | ||
ArangoDB Platform for centralizing authentication and routing. | ||
|
||
- **Web interface**: The Platform includes a unified, browser-based UI that lets | ||
you access its features in an intuitive way. Optional products like the | ||
GenAI Suite seamlessly integrate into the UI if installed. | ||
|
||
## Features | ||
|
||
- [**ArangoDB Core**](core.md): The ArangoDB database system with support for | ||
graphs, documents, key-value, full-text search, and vector search. | ||
|
||
- [**Graph Visualizer**](../../graphs/graph-visualizer.md): | ||
A web-based tool for exploring your graph data with an intuitive interface and | ||
sophisticated querying capabilities. | ||
|
||
- [**Graph Analytics**](../../graphs/graph-analytics.md): | ||
A service that can efficiently load graph data from the core database system | ||
and run graph algorithms such as PageRank and many more. | ||
|
||
- [**GenAI Suite**](../../data-science/_index.md): | ||
ArangoDB's graph-powered machine learning (GraphML) as well as GraphRAG for | ||
automatically building knowledge graphs from text and taking advantage of both | ||
excerpts and higher-level summaries as context for turbocharging GenAI | ||
applications. | ||
|
||
- [**Notebook servers**](../../data-science/notebook-servers.md): | ||
Run Jupyter kernels in the Platform for hosting interactive, Python-based | ||
notebooks to experiment and develop applications. | ||
|
||
- [**MLflow integration**](../../data-science/graphrag/services/mlflow.md): | ||
Use the popular MLflow for machine learning practitioners as part of the | ||
ArangoDB Platform. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,219 @@ | ||
--- | ||
title: The ArangoDB Platform | ||
menuTitle: Platform | ||
weight: 169 | ||
description: >- | ||
The ArangoDB Platform brings everything ArangoDB offers together to a single | ||
solution that you can deploy on-prem or use as a managed service | ||
--- | ||
{{< tip >}} | ||
The ArangoDB Platform & GenAI Suite is available as a pre-release. To get | ||
exclusive early access, [get in touch](https://arangodb.com/contact/) with | ||
the ArangoDB team. | ||
{{< /tip >}} | ||
|
||
The ArangoDB Platform is a technical infrastructure that acts as the umbrella | ||
for hosting the entire ArangoDB offering of products. The Platform makes it easy | ||
to deploy and operate the core ArangoDB database system along with any additional | ||
ArangoDB products for machine learning, data explorations, and more. You can | ||
run it on-premises or in the cloud yourself on top of Kubernetes to access all | ||
of the platform features. | ||
|
||
## Features of the ArangoDB Platform | ||
|
||
- **Core database system**: The ArangoDB graph database system for storing | ||
interconnected data.{{< comment >}} You can use the free Community Edition or the commercial | ||
Enterprise Edition.{{< /comment >}} | ||
- **Graph Visualizer**: A web-based tool for exploring your graph data with an | ||
intuitive interface and sophisticated querying capabilities. | ||
- **Graph Analytics**: A suite of graph algorithms including PageRank, | ||
community detection, and centrality measures with support for GPU | ||
acceleration thanks to Nvidia cuGraph. | ||
- **GenAI Suite**: A set of machine learning services, APIs, and | ||
user interfaces that are available as a package as well as individual products. | ||
- **GraphML**: A turnkey solution for graph machine learning for prediction | ||
use cases such as fraud detection, supply chain, healthcare, retail, and | ||
cyber security. | ||
- **GraphRAG**: Leverage ArangoDB's graph, document, key-value, | ||
full-text search, and vector search features to streamline knowledge | ||
extraction and retrieval. | ||
{{< comment >}}TODO: Not available in prerelease version | ||
- **Txt2AQL**: Unlock natural language querying with a service that converts | ||
user input into ArangoDB Query Language (AQL), powered by fine-tuned | ||
private or public LLMs. | ||
{{< /comment >}} | ||
- **GraphRAG Importer**: Extract entities and relationships from large | ||
text-based files, converting unstructured data into a knowledge graph | ||
stored in ArangoDB. | ||
- **GraphRAG Retriever**: Perform semantic similarity searches or aggregate | ||
insights from graph communities with global and local queries. | ||
- **MLflow integration**: Use the popular MLflow for machine learning | ||
practitioners as part of the ArangoDB Platform. | ||
- **Jupyter notebooks**: Run a Jupyter kernel in the platform for hosting | ||
interactive notebooks for experimentation and development of applications | ||
that use ArangoDB as their backend. | ||
{{< comment >}}TODO: Mostly unrelated to Platform, vector index in core, | ||
- **Vector embeddings**: You can train machine learning models for later use | ||
nerpaula marked this conversation as resolved.
Show resolved
Hide resolved
|
||
in vector search in conjunction with the core database system's `vector` | ||
index type. It allows you to find similar items in your dataset. | ||
{{< /comment >}} | ||
|
||
## Get started with the ArangoDB Platform | ||
|
||
### Use the ArangoDB Platform as a managed service | ||
|
||
The ArangoDB Platform is not available as a managed service yet, but it will | ||
become available for the [ArangoGraph Insights Platform](../arangograph/_index.md) | ||
in the future. Until then, you can request early access to the self-hosted | ||
ArangoDB Platform for testing. | ||
|
||
### Self-host the ArangoDB Platform | ||
|
||
You can set up and run the ArangoDB Platform on-premises or in the cloud and | ||
manage this deployment yourself. | ||
|
||
#### Requirements for self-hosting | ||
|
||
- **Early access to the ArangoDB Platform**: | ||
[Get in touch](https://arangodb.com/contact/) with the ArangoDB team to get | ||
exclusive early access to the pre-release of the ArangoDB Platform & GenAI Suite. | ||
|
||
- **Kubernetes**: Orchestrates the selected services that comprise the | ||
ArangoDB Platform, running them in containers for safety and scalability. | ||
|
||
Set up a [Kubernetes](https://kubernetes.io/) cluster if you don't have one | ||
available yet. | ||
|
||
- **kubectl**: A command line tool for communicating with a Kubernetes cluster's | ||
control plane. | ||
|
||
Install [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl) for applying | ||
specifications such as for creating the ArangoDB Core deployment, as well as | ||
for checking pods, logs, etc. | ||
|
||
- **Helm**: A package manager for Kubernetes. | ||
|
||
You need to have [helm](https://helm.sh/docs/intro/install/) installed in order | ||
to install the required certificate manager and the ArangoDB Kubernetes Operator | ||
as part of the Platform setup. | ||
|
||
- **Container registry**: A repository for storing and accessing container images. | ||
|
||
You need to have a container registry for installing the images of the Platform | ||
services. It can be a local registry. | ||
Simran-B marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
{{< comment >}} | ||
- **Licenses**: If you want to use any paid features, you need to purchase the | ||
respective packages. | ||
{{< /comment >}} | ||
|
||
#### Setup | ||
|
||
1. Obtain a zip package of the ArangoDB Platform for the offline installation. | ||
It includes helm charts, manifests, and blobs of the container image layers. | ||
You also receive a package configuration file from the ArangoDB team. | ||
Simran-B marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
2. Install the certificate manager. You can check <https://github.com/cert-manager/cert-manager> | ||
for the available releases. | ||
|
||
```sh | ||
VERSION_CERT='1.18.2' # Use a newer version if available | ||
helm repo add jetstack https://charts.jetstack.io | ||
helm repo update | ||
|
||
helm upgrade --install cert-manager \ | ||
--namespace cert-manager --create-namespace \ | ||
--version "v${VERSION_CERT}" \ | ||
jetstack/cert-manager \ | ||
--set crds.enabled=true | ||
``` | ||
|
||
3. Install the ArangoDB operator for Kubernetes `kube-arangodb` with helm, | ||
with options to enable webhooks, certificates, and the gateway feature. | ||
|
||
```sh | ||
VERSION_OPERATOR='1.2.50' # Use a newer version if available | ||
|
||
helm upgrade --install operator \ | ||
--namespace arangodb --create-namespace \ | ||
"https://github.com/arangodb/kube-arangodb/releases/download/${VERSION_OPERATOR}/kube-arangodb-${VERSION_OPERATOR}.tgz" \ | ||
--set "webhooks.enabled=true" \ | ||
--set "certificate.enabled=true" \ | ||
--set "operator.args[0]=--deployment.feature.gateway=true" \ | ||
--set "operator.architectures={amd64}" # or {arm64} for ARM-based CPUs | ||
``` | ||
|
||
4. Create an `ArangoDeployment` specification for the ArangoDB Core. See the | ||
[ArangoDeployment Custom Resource Overview](https://arangodb.github.io/kube-arangodb/docs/deployment-resource-reference.html) | ||
and the linked reference. | ||
|
||
You need to enable the gateway feature by setting `spec.gateway.enabled` and | ||
`spec.gateway.dynamic` to `true` in the specification: | ||
|
||
```yaml | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe make it easier for first-time users by providing a standard cluster config here? Single server would also be possible, but something more production-like makes more sense to me |
||
apiVersion: "database.arangodb.com/v1" | ||
kind: "ArangoDeployment" | ||
metadata: | ||
name: "platform-example" | ||
spec: | ||
gateway: | ||
enabled: true | ||
dynamic: true | ||
# ... | ||
``` | ||
|
||
5. Download the ArangoDB Platform CLI tool `arangodb_operator_platform` from | ||
<https://github.com/arangodb/kube-arangodb/releases>. | ||
It is available for Linux and macOS, for the x86-64 as well as 64-bit ARM | ||
architecture (e.g. `arangodb_operator_platform_linux_amd64`). | ||
|
||
It is recommended to rename the downloaded executable to | ||
nerpaula marked this conversation as resolved.
Show resolved
Hide resolved
|
||
`arangodb_operator_platform` and add it to the `PATH` environment variable | ||
to make it available as a command in the system. | ||
|
||
The Platform CLI tool simplifies the further setup and later management of | ||
the Platform's Kubernetes services. | ||
|
||
6. Import the zip package of the ArangoDB Platform into the container registry. | ||
Replace `platform.zip` with the file path of the offline installation package. | ||
Replace `gcr.io/my-reg` with the address of your registry. | ||
|
||
```sh | ||
arangodb_operator_platform package import \ | ||
--registry-docker-credentials \ | ||
gcr.io/my-reg \ | ||
./platform.zip \ | ||
platform.imported.yaml | ||
``` | ||
|
||
7. Install the package using the package configuration you received from the | ||
ArangoDB team (`platform.yaml`) and the configuration generated by the | ||
previous command (`platform.imported.yaml`). These configurations are merged, | ||
allowing for targeted upgrades and user-defined overrides. | ||
|
||
The package installation creates and enables various services, including | ||
the unified web interface of the Platform. | ||
|
||
```sh | ||
arangodb_operator_platform --context arangodb package install \ | ||
--platform.name platform-example \ | ||
./platform.yaml \ | ||
./platform.imported.yaml | ||
``` | ||
|
||
## Interfaces | ||
|
||
The ArangoDB Platform uses a gateway to make all its services available via a | ||
single port at the external address of the deployment. For a local deployment, | ||
the base URL is `https://127.0.0.1:8529`. | ||
|
||
### Unified web interface | ||
|
||
You can access the ArangoDB Platform web interface with a browser by appending | ||
`/ui/` to the base URL, e.g. `https://127.0.0.1:8529/ui/`. | ||
|
||
### ArangoDB Core | ||
|
||
The HTTP API of the ArangoDB Core database system is available at the base URL. | ||
For example, the URL of the Cursor API for submitting AQL queries (against the `_system` database) is | ||
`https://127.0.0.1:8529/_db/_system/_api/cursor`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.