Painless docs overhaul (explore & analyze) #3580

kilfoyle · 2025-10-21T18:25:24Z

This refreshes the Scripting section of the Explore and Analyze docs, based on the Painless doc project.

The content has had both technical and writer reviews, so this is essentially a port from the gdoc project folder into the V3 docs system.

All changes are in this Introduction to Painless section.

Rel: https://github.com/elastic/docs-content-internal/issues/299

github-actions · 2025-10-21T19:38:06Z

🔍 Preview links for changed docs

More links …

benironside

As requested, I didn't do a thorough review here since others already did. But from doing a spot check, this looks awesome. Great work, tons of improvements!

explore-analyze/toc.yml

karenzone

✨

shainaraskas

did only a very brief sweep due to limited time. couple of things:

the newly introduced tutorials aren't structured as tutorials - the steps are all one big block of prose. they really need to be broken into sequential steps.
language in the tutorials is very flowery. could pull it back a little.
Lots of booboo typos and formatting errors here and there (commented on the first handful I found). Could use a finer toothed comb. Would also suggest just requesting a review from copilot in github to do an editing pass

explore-analyze/scripting/common-script-uses.md

explore-analyze/scripting/modules-scripting-painless.md

explore-analyze/scripting/modules-scripting-security.md

kilfoyle · 2025-10-29T21:44:05Z

Thanks for the suggestions @shainaraskas! I've resolved all of your comments.

I think the "How to write scripts" area could do with some more style and structure improvements, apart from what you've suggested, but since that content wasn't touched as part of the Kibernum team's review (apart from just splitting the original file into separate pages), I've left that alone for now. I can definitely come back to it another day, though.

kilfoyle · 2025-11-03T19:39:15Z

@jdconrad Would you mind giving this a final technical review? I've changed some of the phrasing and layout, but the code and examples are untouched from what the Kibernum team provided. Same thing for the Troubleshooting docs section, and then there will be a final PR to come for the Reference docs. Thank you!

jdconrad · 2025-11-03T20:38:27Z

I will take a look soon.

jdconrad

The examples are great, and something that has been missing for a while. I added a few comments around the correctness of the security layers.

jdconrad · 2025-11-03T23:57:37Z

explore-analyze/scripting/modules-scripting-painless.md

+
+Painless provides three core benefits across all scripting contexts:
+
+* **Security:** Fine-grained allowlists that prevent access to restricted Java APIs and enforce multiple security layers.  


The allowlists don't enforce multiple security layers.

I've changed this to just * **Security:** Fine-grained allowlists that prevent access to restricted Java APIs.

jdconrad · 2025-11-03T23:59:56Z

explore-analyze/scripting/modules-scripting-security.md

+:::

-Finally, scripts used in [scripted metrics aggregations](elasticsearch://reference/aggregations/search-aggregations-metrics-scripted-metric-aggregation.md) can be restricted to a defined list of scripts, or forbidden altogether. This can prevent users from running particularly slow or resource intensive aggregation queries.
+The fine-grained allowlist operates as the **first security layer**. Anything that is not part of the allowlist will result in a compilation error.   


This depends. It could also be a runtime error if the type isn't known at compile-time.

I think we can just say error then, without specifying the type. I've changed it to: The fine-grained allowlist operates as the first security layer. Anything that is not part of the allowlist will result in an error.

jdconrad · 2025-11-04T00:01:06Z

explore-analyze/scripting/modules-scripting-security.md


-{{es}} uses [seccomp](https://en.wikipedia.org/wiki/Seccomp) in Linux, [Seatbelt](https://www.chromium.org/developers/design-documents/sandbox/osx-sandboxing-design) in macOS, and [ActiveProcessLimit](https://msdn.microsoft.com/en-us/library/windows/desktop/ms684147) on Windows as additional security layers to prevent {{es}} from forking or running other processes.
+:::{image} /explore-analyze/images/elasticsearch-painless-security-architecture-overview.png
+:alt: Flow chart showing four layers of security between script request and safe script execution


I'm not sure a flow chart is a good way to present this. Each piece of security is sort of separate from the others in many cases.

Good point! I've removed the flow chart graphic, and rephrased the text a bit to de-emphasize the "layers", so as not to suggest that the security features are applied sequentially. I've also removed the bold format from part of the text.

jdconrad · 2025-11-04T00:01:30Z

explore-analyze/scripting/modules-scripting-security.md

-Finally, scripts used in [scripted metrics aggregations](elasticsearch://reference/aggregations/search-aggregations-metrics-scripted-metric-aggregation.md) can be restricted to a defined list of scripts, or forbidden altogether. This can prevent users from running particularly slow or resource intensive aggregation queries.
+The fine-grained allowlist operates as the **first security layer**. Anything that is not part of the allowlist will result in a compilation error.   
+
+The **second layer of security** is [Java Security Manager (JSM)](https://www.oracle.com/java/technologies/javase/seccodeguide.html). As part of its startup sequence, {{es}} enables JSM to limit the actions that portions of the code can take. Painless uses this additional layer of defense to prevent scripts from doing things such as writing files and listening to sockets.


This should probably be updated to entitlements.

I've changed:
from: enables JSM to limit the actions that portions of the code can take.
to: enables JSM to limit the entitlements that portions of the code have.

Please let me know if that's not correct.

As suggested, I've removed the JSM part completely. So this is how that section looks now:

jdconrad · 2025-11-04T00:03:49Z

explore-analyze/scripting/modules-scripting-shorten-script.md

+  "script_fields": {
+    "my_doubled_field": {
+      "script": {
+        "source": "field('my_field').get(null) * params['multiplier']",


The field access API isn't a direct replacement for doc. The behavior includes a default value (the null) and may access both doc values and _source as underlying structures depending on the field type.

I've changed the last paragraph to:
You can use this abbreviated syntax anywhere that {{es}} supports scripts, such as when you’re creating [runtime fields](../../manage-data/data-store/mapping/map-runtime-field.md). Be mindful, however, that the fieldaccess API is not a direct replacement fordoc. This shortened version of the original script includes a default value (the null) so the script may access both docvalues and_source as underlying structures, depending on the field type.

Again, let me know if I've misunderstood.

This is close.

Depending on the field type the script may access either doc values or _source. Some fields will use _source as a fallback if doc values aren't available for a specific field.

Perfect. Thanks. I've changed this as suggested.

This shortened version of the original script includes a default value (the null), so depending on the field type the script may access either doc values or _source. Some fields will use _source as a fallback if doc values aren't available for a specific field.

kilfoyle · 2025-11-04T17:02:44Z

Thanks a lot for the review @jdconrad! I've applied fixes but please let me know if anything's not right.

jdconrad · 2025-11-05T16:09:44Z

@kilfoyle Thanks for the revisions. I added a few more comments. Please let me know if they make sense or need any further clarification. I think not referencing the Java Security Manager is really important.

kilfoyle · 2025-11-05T16:50:43Z

Thanks again @jdconrad! I've fixed those last couple of remaining things. Please see comments above.

jdconrad

LGTM! Thanks for all the work done here.

kilfoyle added 3 commits October 21, 2025 14:24

[DRAFT] Painless docs overhaul, Part Uno

8b75443

add some tutorials

0ed6b5e

link fix

05b98f8

more stuff

42e67f9

kilfoyle mentioned this pull request Oct 22, 2025

Update link in dsl_how_to_guides.md elastic/elasticsearch-py#3122

Merged

kilfoyle added 2 commits October 23, 2025 13:46

context bridge

13d397e

fix broken links

2ede829

kilfoyle changed the title ~~[DRAFT] Painless docs overhaul, Part Uno~~ Painless docs overhaul, part un Oct 23, 2025

kilfoyle changed the title ~~Painless docs overhaul, part un~~ Painless docs overhaul (explore & analyze) Oct 23, 2025

kilfoyle marked this pull request as ready for review October 24, 2025 13:14

kilfoyle requested review from a team as code owners October 24, 2025 13:14

yetanothertw approved these changes Oct 24, 2025

View reviewed changes

kilfoyle marked this pull request as draft October 24, 2025 14:51

Merge branch 'main' into painless001

12137d5

kilfoyle marked this pull request as ready for review October 27, 2025 16:57

benironside approved these changes Oct 27, 2025

View reviewed changes

shainaraskas reviewed Oct 27, 2025

View reviewed changes

explore-analyze/toc.yml Show resolved Hide resolved

karenzone approved these changes Oct 27, 2025

View reviewed changes

shainaraskas reviewed Oct 27, 2025

View reviewed changes

This comment was marked as duplicate.

Sign in to view

kilfoyle requested a review from shainaraskas October 29, 2025 21:39

Address Shaina's feedback

42241ef

kilfoyle requested a review from jdconrad November 3, 2025 19:39

jdconrad reviewed Nov 4, 2025

View reviewed changes

Apply Jack's suggestions

8c2d54c

A few more updates

b77d336

jdconrad approved these changes Nov 5, 2025

View reviewed changes

Merge branch 'main' into painless001

80b53e2

kilfoyle merged commit a0c782f into elastic:main Nov 5, 2025
7 checks passed


		Painless provides three core benefits across all scripting contexts:

		* Security: Fine-grained allowlists that prevent access to restricted Java APIs and enforce multiple security layers.

Painless docs overhaul (explore & analyze) #3580

Painless docs overhaul (explore & analyze) #3580

Conversation

kilfoyle commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔍 Preview links for changed docs

Uh oh!

benironside left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

karenzone left a comment

Choose a reason for hiding this comment

Uh oh!

shainaraskas left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as duplicate.

Uh oh!

kilfoyle commented Oct 29, 2025

Uh oh!

kilfoyle commented Nov 3, 2025

Uh oh!

jdconrad commented Nov 3, 2025

Uh oh!

jdconrad left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kilfoyle Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kilfoyle Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kilfoyle Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kilfoyle commented Nov 4, 2025

Uh oh!

jdconrad commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kilfoyle commented Nov 5, 2025

Uh oh!

kilfoyle commented Oct 21, 2025 •

edited

Loading

github-actions bot commented Oct 21, 2025 •

edited

Loading

kilfoyle Nov 4, 2025 •

edited

Loading

kilfoyle Nov 5, 2025 •

edited

Loading

kilfoyle Nov 4, 2025 •

edited

Loading

jdconrad commented Nov 5, 2025 •

edited

Loading