Arbitrary code execution in LLMMathChain

[jan-kubena]

System Info

Langchain version: 0.0.244
Numexpr version: 2.8.4
Python version: 3.10.11

Who can help?

@hwchase17 @vowelparrot

Information

• The official example notebooks/scripts
• My own modified scripts

Related Components

• LLMs/Chat Models
• Embedding Models
• Prompts / Prompt Templates / Prompt Selectors
• Output Parsers
• Document Loaders
• Vector Stores / Retrievers
• Memory
• Agents / Agent Executors
• Tools / Toolkits
• Chains
• Callbacks/Tracing
• Async

Reproduction

Numexpr's evaluate function that Langchain uses here in the LLMMathChain is susceptible to arbitrary code execution with eval in the latest released version. See this issue where PoC for numexpr's evaluate is also provided.

This vulnerability allows an arbitrary code execution, that is to run code and commands on target machine, via LLMMathChain's run method with the right prompt. I'd like to ask the Langchain's maintainers to confirm if they want a full PoC with Langchain posted here publicly.

Expected behavior

Numerical expressions should be evaluated securely so as to not allow code execution.

[hinthornw]

Thanks for raising! We are working to move the LLMMathChain out of the langchain core library, for reasons such as this

[dosubot]

Answer generated by a 🤖

Answer

I understand your concern about the potential security vulnerability in the LLMMathChain class of LangChain. You've correctly identified that the numexpr.evaluate function, which is used in the _evaluate_expression method, could potentially execute arbitrary code if a malicious expression is passed to it. This is due to the fact that numexpr.evaluate uses Python's eval function, which can execute arbitrary Python code.

Here's the code snippet in question:

def _evaluate_expression(self, expression: str) -> str:
    try:
        local_dict = {"pi": math.pi, "e": math.e}
        output = str(
            numexpr.evaluate(
                expression.strip(),
                global_dict={},  # restrict access to globals
                local_dict=local_dict,  # add common mathematical functions
            )
        )
    except Exception as e:
        raise ValueError(
            f'LLMMathChain._evaluate("{expression}") raised error: {e}.'
            " Please try again with a valid numerical expression"
        )

    # Remove any leading and trailing brackets from the output
    return re.sub(r"^\[|\]$", "", output)

In this code, numexpr.evaluate is called with the expression argument, which could potentially contain malicious code. Although the global_dict argument is set to an empty dictionary to restrict access to global variables, and the local_dict argument is set to a dictionary containing common mathematical functions, this does not prevent the execution of arbitrary code.

This means that a malicious user could potentially execute arbitrary code on the server where LangChain is running by passing a specially crafted expression to the LLMMathChain's run method. This is indeed a serious security issue that needs to be addressed.

However, please note that this is just one potential interpretation of the issue. If you believe this interpretation is correct and that this is a bug that could impact other users, I encourage you to make a pull request to address this issue. If your experience contradicts this interpretation, I would suggest upgrading to the latest version of LangChain to see if the issue persists.

Sources

• libs/langchain/langchain/chains/llm_math/base.py

---

This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

[zoeqevans]

@hinthornw As an alternative, we could use sympy.sympify? It's a common alternative to eval / numexpr (see this thread on vulnerabilities in numexpr). As well as avoiding that vulnerability, sympy also supports a much larger range of mathematical operations (i.e. rounding, factorials, etc.) and is about 10x as common as numexpr (as measured by github stars), so I'd guess its easier for an LLM to generate syntactically correct input (in fact, I spent about 10mins trying to get GPT-3 to generate a mathematical input that sympy couldn't process, and failed).

I was testing some multi-step QA workflows inspired by a Langchain multi-step agent QA workflow, and was using the llm-math tool in an Agent chain, with prompts like: "Who is Zoe Saldana's partner? What is their current age raised to the 0.43 power, rounded to 3sf?". This would cause numexpr to throw an error, since the chain would try and fail to pass it some kind of rounding instruction. This also happens with the factorial function in #3071. Both of these use cases are fixed by backing LLMMathChain with a sympify call.

Happy to attach some examples / create a PR to scope out the replacement if people agree. It's what I've been using locally and it works great!

[hinthornw]

That makes sense to me. I'd be happy to review a PR! Thank you for being proactive about this!

[jan-kubena]

I'd just like to point out that there still needs to be some kind of validation/protection because sympy.sympify uses eval and shouldn't be used on unsanitized input as per official docs. I think it could also be worth looking at this langchain PR that implemented sympy in RestrictedPython as an inspiration.

[zoeqevans]

Ah damn, yeah I missed that: was foolishly going off the StackOverflow thread and a memory of a still-unresolved effort within Sympify to remove eval / add a safe-mode.

sympify still offers a lot more functionality than numexpr, so I'd still be in favour of swapping anyway, but agree that there aren't any extra security benefits vs. numexpr on the eval front.

[zoeqevans]

@hinthornw PR here

[jan-kubena]

Hi all, input sanitization has been added in numexpr version 2.8.6 for evaluate by default (see this issue I mentioned previously). As far as I can tell, it looks to be pretty secure. It'd be nice for the expression execution to be done in a secure container/environment, but I think that for now, the sanitization that numexpr does is surely better than nothing.

Since that is the case, I'd like to ask @hinthornw to advise on the preferable next course of action, that is whether to just bump the numexpr version, replace numexpr completely with Sympy and a secure container/environment (since it'd be vulnerable by itself) or other solutions to this. Thank you!

[tabdunabi]

Hi @hinthornw, Any ETA on resolving this issue?
Thanks

[elmegatan26]

This vulnerability is getting flagged by InfoSec teams. Any idea on when the update is being released?

[eyurtsev]

@elmegatan26 , @tabdunabi , @jan-kubena , numexpr is now (on master) an optional dependency, we've also added a constraint to specify that code only works with >=2.8.6 (which has input sanitization).

• Is this enough to address concerns flagged by InfoSec teams?
• Also do you mind sharing if you're using LLMathChain yourself and if so which mathematical operations do you rely on?

[tabdunabi]

@eyurtsev, for our use case, we do not use LLMathChain, but our security scans detect the numexpr vulnerability. This is a critical issue for us and blocker to publish our solutions, used by our customers. If making numexpr optional (not installed by default with LangChian), then this is enough for us.