Skip to content

Commit a873881

Browse files
committed
Add notes about remote files and adaptive scaling.
1 parent 926fc6b commit a873881

File tree

2 files changed

+16
-2
lines changed

2 files changed

+16
-2
lines changed

docs/source/coiled.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,14 @@ configure the hardware and software environment.
6666
```{literalinclude} ../../docs_src/coiled/coiled_functions_task.py
6767
```
6868

69+
By default, {func}`@coiled.function <coiled.function>`
70+
[scales adaptively](https://docs.coiled.io/user_guide/usage/functions/index.html#adaptive-scaling)
71+
to the workload. It means that coiled infers from the number of submitted tasks and
72+
previous runtimes, how many additional remote workers it should deploy to handle the
73+
workload. It provides a convenient mechanism to scale without intervention. Also,
74+
workers launched by {func}`@coiled.function <coiled.function>` will shutdown quicker
75+
than a cluster.
76+
6977
```{seealso}
7078
Serverless functions are more thoroughly explained in
7179
[coiled's guide](https://docs.coiled.io/user_guide/usage/functions/index.html).

docs/source/remote_backends.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
There are a couple of things you need to know when using backends that launch workers
44
remotely, meaning not on your machine.
55

6-
## Cross-Platform
6+
## Cross-platform support
77

88
Issue: {issue}`102`.
99

@@ -19,7 +19,13 @@ Windows users on the other hand should use the
1919
[WSL (Windows Subsystem for Linux)](https://learn.microsoft.com/en-us/windows/wsl/about)
2020
to run their projects.
2121

22-
## Local Paths
22+
## Local files
23+
24+
Avoid using local files with remote backends and use storages like S3 for dependencies
25+
and products. The reason is that every local file needs to be send to the remote workers
26+
and when your internet connection is slow you will face a hefty penalty on runtime.
27+
28+
## Local paths
2329

2430
In most projects you are using local paths to refer to dependencies and products of your
2531
tasks. This becomes an interesting problem with remote workers since your local files

0 commit comments

Comments
 (0)