Skip to content

update connect API docs #13145

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dannyroosevelt merged 4 commits into from
Aug 7, 2024
Merged

update connect API docs #13145

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f0243b8
Update index.mdx
dannyroosevelt Aug 6, 2024
7034a2f
Merge branch 'master' into danny/connect-docs-external-user-api
dannyroosevelt Aug 7, 2024
a5d4c9c
updating connect docs
dannyroosevelt Aug 7, 2024
435081b
Merge branch 'master' into danny/connect-docs-external-user-api
dannyroosevelt Aug 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs-v2/next.config.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,8 @@ module.exports = withNextra({
SERVICE_DB_SIZE_LIMIT: "60KB",
TMP_SIZE_LIMIT: "2GB",
DELAY_MIN_MAX_TIME:
"You can pause your workflow for as little as one millisecond, or as long as one year",
PUBLIC_APPS: "2,000",
"You can pause your workflow for as little as one millisecond, or as long as one year",
PUBLIC_APPS: "2,200",
FREE_INSPECTOR_EVENT_LIMIT: "7 days of events",
WARM_WORKERS_INTERVAL: "10 minutes",
WARM_WORKERS_CREDITS_PER_INTERVAL: "5",
Expand Down
226 changes: 200 additions & 26 deletions docs-v2/pages/connect/index.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,55 +3,70 @@ import { Steps } from 'nextra/components'

# Pipedream Connect

## Overview

Pipedream Connect is the easiest way for your users to connect to any API, **right in your product**.

Connect enables you to easily integrate more than {process.env.PUBLIC_APPS}+ APIs directly into your application, letting you make authenticated requests on behalf of your end users, while not having to handle authorization grants or refresh tokens yourself, and it includes these features:
1. [Client SDK](https://github.com/PipedreamHQ/pipedream/tree/master/packages/sdk) to initiate authorization for any one of the [{process.env.PUBLIC_APPS}+ APIs](https://pipedream.com/apps) available on Pipedream
2. [REST API](#rest-api-overview) to retrieve credentials (access token, API key, etc.) for your end users
3. The Pipedream platform, which includes [a serverless runtime](/), [thousands of pre-built source-available triggers and actions](https://github.com/PipedreamHQ/pipedream/tree/master/components), and a first-class developer experience to build, test, and deploy custom workflows


<Callout type="info">
Pipedream Connect is currently in **preview** and we're looking for feedback!
**Pipedream Connect is currently in preview and we're looking for feedback!**

**While in the preview phase, the API may change without notice. We'll do our best to communicate any changes, but please be aware that breaking changes may occur.**

Please [let us know](https://pipedream.com/support) how you're using it, what's not working, and what else you'd like to see!
Please [let us know](mailto:connect@pipedream.com) how you're using it, what's not working, and what else you'd like to see!
</Callout>

## Overview

Connect enables you to easily integrate more than {process.env.PUBLIC_APPS}+ APIs directly into your application, letting you make authenticated requests on behalf of your end users, while not having handle authorization grants or refresh tokens yourself, and it includes these features:
1. [Client SDK](https://github.com/PipedreamHQ/pipedream/tree/master/packages/sdk) to initiate authorization for any one of the [{process.env.PUBLIC_APPS}+ APIs](https://pipedream.com/apps) available on Pipedream
2. [REST API](/connected-accounts/api) to retrieve credentials (access token, API key, etc.) for your end users
3. The rest of Pipedream, which includes [a serverless runtime platform](/), [thousands of pre-built source-available triggers and actions](https://github.com/PipedreamHQ/pipedream/tree/master/components), and a first-class developer experience to build, test, and deploy custom workflows

## Getting started

<Steps>

### Enable the feature flag
Enable the feature flag for **Connect** in your Pipedream account: [https://pipedream.com/settings/alpha](https://pipedream.com/settings/alpha)
- Enable the feature flag for **Pipedream Connect** in your [account settings here](https://pipedream.com/settings/alpha).

<Callout type="info">
Please reach out to our team at [[email protected]]([email protected]) to enable your account for Connect, once you've enabled the feature flag.
Please reach out to our team at [[email protected]](mailto:[email protected]) to activate your account for Connect once you've enabled the feature flag and send us your [workspace ID](https://pipedream.com/settings/account).
</Callout>

### Verify your application's domain
To integrate the Pipedream SDK on your site, you'll need to verify ownership of your web domain. [Check out the docs](workspaces/domain-verification#pipedream-connect) to get started.
- To integrate the Pipedream SDK on your site, you'll need to verify ownership of your web domain. [Check out the docs](workspaces/domain-verification#pipedream-connect) to get started.

### Configure an OAuth client in Pipedream
To use Pipedream Connect, you'll need to create an OAuth client in Pipedream, for whatever app you want to integrate. [Check out the docs](/connected-accounts/oauth-clients#configuring-custom-oauth-clients) to get started.
- To use Pipedream Connect, you'll need to create an OAuth client in Pipedream for whatever app you want to integrate. [Check out the docs](/connected-accounts/oauth-clients#configuring-custom-oauth-clients) to get started.
- Make sure to check the box to **Allow external user connections** in the OAuth client configuration in Pipedream

### Open your Pipedream project
Open an existing Pipedream project or create a new one: [https://pipedream.com/projects](https://pipedream.com/projects)
- Note your **Project Public Key** and **Project Secret Key** from the **Connect** tab within your project, you'll need these when configuring the SDK and making API requests
- Open an existing Pipedream project or create a new one: [https://pipedream.com/projects](https://pipedream.com/projects)
- Note your **Project Public Key** and **Project Secret Key** from the **Connect** tab within your project — you'll need these when configuring the SDK and making API requests

### Run the example app
The code for the Pipedream SDK can be found in our [public `pipedream` repo](https://github.com/PipedreamHQ/pipedream), in `/packages/sdk`. Check out the `README` there to get started.
- Install the [SDK](https://www.npmjs.com/package/@pipedream/sdk) via npm: `npm install @pipedream/sdk`
- The code for the Pipedream SDK lives in the [public pipedream repo](https://github.com/PipedreamHQ/pipedream/tree/master/packages/sdk)
- Check out the example app and README there to get started

### Retrieve your user credentials from Pipedream's API
Check out to the API reference below to retrieve your users' credentials from Pipedream's API after they've connected their account.
- Check out to the API reference below to retrieve your users' credentials from Pipedream's API after they've connected their account.

</Steps>

## Retrieving account details and user credentials
After your end users have connected their account using Pipedream Connect, you can retrieve their credentials using Pipedream's REST API, which enables you to make authenticated requests on their behalf.

### REST API Overview
The base URL for all Connect requests is `https://api.pipedream.com/v1/connect`.
<Callout type="warning">
Note that both the Base URL and authentication method for the Connect REST API are different from the rest of Pipedream's REST API.
</Callout>

**Base URL for all Connect requests**
```
https://api.pipedream.com/v1/connect
```

**Authentication**

You authenticate to the Connect REST API using **Basic Auth**. Send your Pipedream [Project Public Key]() as the username and the [Project Secret Key]() as the password. When you make API requests, pass an
`Authorization` header of the following format:
Expand All @@ -69,8 +84,12 @@ curl 'https://api.pipedream.com/v1/connect/accounts' \

### API Reference

#### List accounts
List all connected accounts for end users within your project
#### List all accounts
List all connected accounts for all end users within your project

<Callout type="warning">
This endpoint is not currently paginated, so we'll attempt to return all connected accounts for all users within your project. We intend to add pagination soon.
</Callout>

```
GET /accounts/
Expand All @@ -85,6 +104,7 @@ GET /accounts/
{
"id": "apn_XehyZPr",
"name": null,
"external_id": "user-abc-123",
"healthy": true,
"dead": false,
"app": {
Expand All @@ -97,6 +117,8 @@ GET /accounts/
{
"id": "apn_b6h9QDK",
"name": null,
"external_id": "user-abc-123",
"external_id": "user-abc-123",
"healthy": true,
"dead": false,
"app": {
Expand All @@ -109,6 +131,7 @@ GET /accounts/
{
"id": "apn_0WhJYxv",
"name": null,
"external_id": "user-abc-123",
"healthy": true,
"dead": false,
"app": {
Expand All @@ -121,6 +144,7 @@ GET /accounts/
{
"id": "apn_kVh9PJx",
"name": null,
"external_id": "user-abc-123",
"healthy": true,
"dead": false,
"app": {
Expand All @@ -133,6 +157,7 @@ GET /accounts/
{
"id": "apn_WYhMlrz",
"name": null,
"external_id": "user-abc-123",
"healthy": true,
"dead": false,
"app": {
Expand Down Expand Up @@ -162,6 +187,7 @@ GET /apps/:app_id/accounts
{
"id": "apn_WYhMlrz",
"name": null,
"external_id": "user-abc-123",
"healthy": true,
"dead": false,
"app": {
Expand All @@ -180,8 +206,9 @@ GET /apps/:app_id/accounts
]
```

#### Retrieve account details
Retrieve the account details for a specific account
#### Retrieve account details for a specific auth provision

Retrieve the account details for a specific account based on the auth provision

```
GET /accounts/:apn_id
Expand All @@ -196,6 +223,7 @@ GET /accounts/:apn_id
"data": {
"id": "apn_WYhMlrz",
"name": null,
"external_id": "user-abc-123",
"healthy": true,
"dead": false,
"app": {
Expand All @@ -221,6 +249,7 @@ GET /accounts/:apn_id
"data": {
"id": "apn_WYhMlrz",
"name": null,
"external_id": "user-abc-123",
"healthy": true,
"dead": false,
"app": {
Expand All @@ -245,25 +274,165 @@ GET /accounts/:apn_id
}
```

#### Retrieve account details for a specific external user

Retrieve the account details for a specific account based on the external user ID

```
GET /users/:external_id/accounts
```
- `:external_id` is the end user's ID in the Connect Partner's system
- Optionally include `?include_credentials=1` as a query-string parameter to include the account credentials in the response

##### Example response (without account credentials)

```json
[
{
"id": "apn_WYhM5ov",
"name": null,
"external_id": "user-abc-123",
"healthy": true,
"dead": false,
"app": {
"id": "oa_aw4ib2",
"name_slug": "airtable_oauth",
"name": "Airtable",
"auth_type": "oauth",
"description": "Airtable is a low-code platform to build next-gen apps. Move beyond rigid tools, operationalize your critical data, and reimagine workflows with AI.",
"img_src": "https://assets.pipedream.net/s.v0/app_XBxhAl/logo/orig",
"custom_fields_json": "[]",
"categories": [
"Productivity"
]
},
"created_at": "2024-08-06T21:51:30.000Z",
"updated_at": "2024-08-06T21:51:30.000Z",
"expires_at": "2024-08-06T22:51:30.000Z",
"error": null,
"last_refreshed_at": null,
"next_refresh_at": "2024-08-06T22:04:41.000Z"
},
{
"id": "apn_KAh7JwW",
"name": null,
"external_id": "user-abc-123",
"healthy": true,
"dead": false,
"app": {
"id": "oa_aPXiQd",
"name_slug": "github",
"name": "GitHub",
"auth_type": "oauth",
"description": "Where the world builds software. Millions of developers and companies build, ship, and maintain their software on GitHub—the largest and most advanced development platform in the world.",
"img_src": "https://assets.pipedream.net/s.v0/app_OrZhaO/logo/orig",
"custom_fields_json": "[]",
"categories": [
"Developer Tools"
]
},
"created_at": "2024-08-06T21:53:05.000Z",
"updated_at": "2024-08-06T21:53:05.000Z",
"expires_at": null,
"error": null,
"last_refreshed_at": null,
"next_refresh_at": "2024-08-06T22:50:01.000Z"
}
]
```

##### Example Response (with `include_credentials=1`)

```json
[
{
"id": "apn_WYhM5ov",
"name": null,
"external_id": "user-abc-123",
"healthy": true,
"dead": false,
"app": {
"id": "oa_aw4ib2",
"name_slug": "airtable_oauth",
"name": "Airtable",
"auth_type": "oauth",
"description": "Airtable is a low-code platform to build next-gen apps. Move beyond rigid tools, operationalize your critical data, and reimagine workflows with AI.",
"img_src": "https://assets.pipedream.net/s.v0/app_XBxhAl/logo/orig",
"custom_fields_json": "[]",
"categories": [
"Productivity"
]
},
"created_at": "2024-08-06T21:51:30.000Z",
"updated_at": "2024-08-06T21:51:30.000Z",
"credentials": {
"oauth_client_id": "dd7a26ca-ba11-4f80-8667-508364f97c58",
"oauth_access_token": "oaarBssnxxxxxxxxxx.v1.xxxxxxxxxx.xxxxxxxxxx",
"oauth_refresh_token": "oaarBssnxxxxxxxxxx.v1.refresh.xxxxxxxxxx.xxxxxxxxxx",
"oauth_uid": "usrnbIhrdiaOwPf7q"
Comment on lines +369 to +372
Copy link
Contributor

@coderabbitai coderabbitai bot Aug 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Detected generic API key.

The API key should not be exposed in the documentation. Consider using a placeholder or removing it.

-      "oauth_access_token": "oaaLa2Ob1umiregWa.v1.xxxxxxxx.xxxxxxxx",
-      "oauth_refresh_token": "oaaLa2Ob1umiregWa.v1.refresh.xxxxxxxx",

Committable suggestion was skipped due to low confidence.

Tools
Gitleaks

369-369: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

372-372: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

},
"expires_at": "2024-08-06T22:51:30.000Z",
"error": null,
"last_refreshed_at": null,
"next_refresh_at": "2024-08-06T22:04:41.000Z"
},
{
"id": "apn_KAh7JwW",
"name": null,
"external_id": "user-abc-123",
"healthy": true,
"dead": false,
"app": {
"id": "oa_aPXiQd",
"name_slug": "github",
"name": "GitHub",
"auth_type": "oauth",
"description": "Where the world builds software. Millions of developers and companies build, ship, and maintain their software on GitHub—the largest and most advanced development platform in the world.",
"img_src": "https://assets.pipedream.net/s.v0/app_OrZhaO/logo/orig",
"custom_fields_json": "[]",
"categories": [
"Developer Tools"
]
},
"created_at": "2024-08-06T21:53:05.000Z",
"updated_at": "2024-08-06T21:53:05.000Z",
"credentials": {
"oauth_client_id": "57dc28a15xxxxxxxxxx",
"oauth_access_token": "gho_zasPWxxxxxxxxxxxxx",
"oauth_uid": "104484339"
},
"expires_at": null,
"error": null,
"last_refreshed_at": null,
"next_refresh_at": "2024-08-06T22:50:01.000Z"
}
]
```

#### Delete individual connected account
Delete a specific connected account for an end user

```
DELETE /v1/connect/accounts/:apn_id
```

- `:apn_id` is the auth provision of the account you want to retrieve
- We'll return a `204 No Content` response if the account was successfully deleted

#### Delete all connected accounts for a specific app

`/DELETE /apps/:app_id/accounts`
```
/DELETE /apps/:app_id/accounts
```

- `:app_id` can be `oauth_app_id` or `app_id`
- We'll return a `204 No Content` response if the accounts were successfully deleted

#### Delete an end user and all their connected accounts

`DELETE /users/:external_id`
```
DELETE /users/:external_id
```

- `:external_id` corresponds to the the end user's ID in your system
- We'll return a `204 No Content` response if the user was successfully deleted
Expand All @@ -273,5 +442,10 @@ DELETE /v1/connect/accounts/:apn_id

- **App**: GitHub, Notion, Slack, Google Sheets, etc. — the API you want to make requests to on behalf of your end user. See our full list here: [https://pipedream.com/apps](https://pipedream.com/apps).
- **Connect Partner**: This is probably you — the Pipedream customer who is developing an application and wants to use Pipedream Connect to make authenticated requests on behalf of their end users.
- **Connected Account**: Read more about connected accounts [here](/connected-accounts) — this is the account connection for your end user.
- **End User**: The customer of the Connect Partner and whose data you want to access on their behalf.
- **Connected Account**: Read more about connected accounts [here](/connected-accounts) — this is the account connection for your end user, and is also referred to as **auth provision**.
- **End User**: The customer of the Connect Partner and whose data you want to access on their behalf (`external_id` in the API).
- **OAuth Client**: The OAuth client you create in Pipedream to connect to an app. Read more about OAuth clients [here](/connected-accounts/oauth-clients).

## Known limitations
- The only apps that work with Pipedream Connect today are apps that use OAuth 2.0 to authenticate, and that do not require any user-input fields during the OAuth flow. We're working on extending support to all integrated apps on Pipedream.
- There is very limited error handling today, but this is top of mind for us and is on the near-term roadmap. If you or your end users encounter any issues, please reach out to us at [[email protected]](mailto:[email protected])