Skip to content

ci: disable pytest parallel execution on lnprototest #5149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

vincenzopalazzo
Copy link
Collaborator

With the actual size of lnprototest it is not strictly necessary to run it in parallel, and this PR is disabling it because with the parallel execution there is a double-spending event with one of the bitcoin transaction

More in this issue here rustyrussell/lnprototest#31

Stacktrace!

tests/test_bolt2-20-open_channel_accepter.py:1031: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
lnprototest/runner.py:106: in run
    self.stop()
lnprototest/clightning/clightning.py:174: in stop
    self.shutdown()
lnprototest/clightning/clightning.py:168: in shutdown
    cb()
lnprototest/clightning/clightning.py:193: in kill_fundchannel
    fut.result(0)
/usr/lib/python3.8/concurrent/futures/_base.py:437: in result
    return self.__get_result()
/usr/lib/python3.8/concurrent/futures/_base.py:389: in __get_result
    raise self._exception
/usr/lib/python3.8/concurrent/futures/_base.py:328: in _invoke_callbacks
    callback(self)
lnprototest/clightning/clightning.py:275: in _done
    raise (exception)
/usr/lib/python3.8/concurrent/futures/thread.py:57: in run
    result = self.fn(*self.args, **self.kwargs)
lnprototest/clightning/clightning.py:268: in _fundchannel
    return runner.rpc.fundchannel(
/work/contrib/pyln-client/pyln/client/lightning.py:733: in fundchannel
    return self.call("fundchannel", payload)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pyln.client.lightning.LightningRpc object at 0x7fd3282967c0>
method = 'fundchannel'
payload = {'amount': 999877, 'announce': True, 'feerate': '253perkw', 'id': '02c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5'}

    def call(self, method, payload=None):
        self.logger.debug("Calling %s with payload %r", method, payload)
    
        if payload is None:
            payload = {}
        # Filter out arguments that are None
        if isinstance(payload, dict):
            payload = {k: v for k, v in payload.items() if v is not None}
    
        # FIXME: we open a new socket for every readobj call...
        sock = UnixSocket(self.socket_path)
        this_id = self.next_id
        self.next_id += 0
        buf = b''
    
        if self._notify is not None:
            # Opt into the notifications support
            self._writeobj(sock, {
                "jsonrpc": "2.0",
                "method": "notifications",
                "id": 0,
                "params": {
                    "enable": True
                },
            })
            # FIXME: Notification schema support?
            _, buf = self._readobj(sock, buf)
    
        request = {
            "jsonrpc": "2.0",
            "method": method,
            "params": payload,
            "id": this_id,
        }
    
        self._writeobj(sock, request)
        while True:
            resp, buf = self._readobj(sock, buf)
            id = resp.get("id", None)
            meth = resp.get("method", None)
    
            if meth == 'message' and self._notify is not None:
                n = resp['params']
                self._notify(
                    message=n.get('message', None),
                    progress=n.get('progress', None),
                    request=request
                )
                continue
    
            if meth is None or id is None:
                break
    
        self.logger.debug("Received response for %s call: %r", method, resp)
        if 'id' in resp and resp['id'] != this_id:
            raise ValueError("Malformed response, id is not {}: {}.".format(this_id, resp))
        sock.close()
    
        if not isinstance(resp, dict):
            raise ValueError("Malformed response, response is not a dictionary %s." % resp)
        elif "error" in resp:
>           raise RpcError(method, payload, resp['error'])
E           pyln.client.lightning.RpcError: RPC call failed: method: fundchannel, payload: {'id': '02c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5', 'amount': 999877, 'feerate': '253perkw', 'announce': True}, error: {'code': 303, 'message': 'Error broadcasting funding tx: error code: -27\\\\nerror message:\\\\nTransaction already in block chain. Unsent tx discarded 020000000001022f144a38afb7c3886d18f5283a8da92e79c7f6a24a64a5c7f9d5187ac2753f360500000000fdffffff2f144a38afb7c3886d18f5283a8da92e79c7f6a24a64a5c7f9d5187ac2753f360400000000fdffffff036e8501000000000016001473daa75958d5b2ddca87a6c279bb7cb307167037a1c62d0000000000160014d640ab16f347d1de5aba5a715321a5fc4ba9a5d5455c150000000000220020c46bf3d1686d6dbb2d9244f8f67b90370c5aa2747045f1aeccb77d8187117382024730440220444456f44c721d7ebc69b336979e6c36e0f99d6b74419f60ff14f0a21ff1ddb0022044081ac3dfcd9191f4efb20b685fd2fc7329a26dd45cb1219c92a2166e8ea3fc012102c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee502473044022036a9007f85889200e9ad7a3895d9362d5ca00e64d85a991ebfc324616c33db3e022025e0397cd2610a52fb87c9a233baaa660d28c87d13d6fb6823ad94a03f9b14290121026957e53b46df017bd6460681d068e1d23a7b027de398272d0b15f59b78d060a906000000.', 'data': {'id': '02c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5', 'method': 'openchannel_signed'}}

We remember to re-enable it by the following issue #5148

Signed-off-by: Vincenzo Palazzo [email protected]

The parallel execution on lnprototest case a failure on as reported in the following issue rustyrussell/lnprototest#31 and the following stacktrace

```
tests/test_bolt2-20-open_channel_accepter.py:1031: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
lnprototest/runner.py:106: in run
    self.stop()
lnprototest/clightning/clightning.py:174: in stop
    self.shutdown()
lnprototest/clightning/clightning.py:168: in shutdown
    cb()
lnprototest/clightning/clightning.py:193: in kill_fundchannel
    fut.result(0)
/usr/lib/python3.8/concurrent/futures/_base.py:437: in result
    return self.__get_result()
/usr/lib/python3.8/concurrent/futures/_base.py:389: in __get_result
    raise self._exception
/usr/lib/python3.8/concurrent/futures/_base.py:328: in _invoke_callbacks
    callback(self)
lnprototest/clightning/clightning.py:275: in _done
    raise (exception)
/usr/lib/python3.8/concurrent/futures/thread.py:57: in run
    result = self.fn(*self.args, **self.kwargs)
lnprototest/clightning/clightning.py:268: in _fundchannel
    return runner.rpc.fundchannel(
/work/contrib/pyln-client/pyln/client/lightning.py:733: in fundchannel
    return self.call("fundchannel", payload)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pyln.client.lightning.LightningRpc object at 0x7fd3282967c0>
method = 'fundchannel'
payload = {'amount': 999877, 'announce': True, 'feerate': '253perkw', 'id': '02c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5'}

    def call(self, method, payload=None):
        self.logger.debug("Calling %s with payload %r", method, payload)
    
        if payload is None:
            payload = {}
        # Filter out arguments that are None
        if isinstance(payload, dict):
            payload = {k: v for k, v in payload.items() if v is not None}
    
        # FIXME: we open a new socket for every readobj call...
        sock = UnixSocket(self.socket_path)
        this_id = self.next_id
        self.next_id += 0
        buf = b''
    
        if self._notify is not None:
            # Opt into the notifications support
            self._writeobj(sock, {
                "jsonrpc": "2.0",
                "method": "notifications",
                "id": 0,
                "params": {
                    "enable": True
                },
            })
            # FIXME: Notification schema support?
            _, buf = self._readobj(sock, buf)
    
        request = {
            "jsonrpc": "2.0",
            "method": method,
            "params": payload,
            "id": this_id,
        }
    
        self._writeobj(sock, request)
        while True:
            resp, buf = self._readobj(sock, buf)
            id = resp.get("id", None)
            meth = resp.get("method", None)
    
            if meth == 'message' and self._notify is not None:
                n = resp['params']
                self._notify(
                    message=n.get('message', None),
                    progress=n.get('progress', None),
                    request=request
                )
                continue
    
            if meth is None or id is None:
                break
    
        self.logger.debug("Received response for %s call: %r", method, resp)
        if 'id' in resp and resp['id'] != this_id:
            raise ValueError("Malformed response, id is not {}: {}.".format(this_id, resp))
        sock.close()
    
        if not isinstance(resp, dict):
            raise ValueError("Malformed response, response is not a dictionary %s." % resp)
        elif "error" in resp:
>           raise RpcError(method, payload, resp['error'])
E           pyln.client.lightning.RpcError: RPC call failed: method: fundchannel, payload: {'id': '02c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5', 'amount': 999877, 'feerate': '253perkw', 'announce': True}, error: {'code': 303, 'message': 'Error broadcasting funding tx: error code: -27\\\\nerror message:\\\\nTransaction already in block chain. Unsent tx discarded 020000000001022f144a38afb7c3886d18f5283a8da92e79c7f6a24a64a5c7f9d5187ac2753f360500000000fdffffff2f144a38afb7c3886d18f5283a8da92e79c7f6a24a64a5c7f9d5187ac2753f360400000000fdffffff036e8501000000000016001473daa75958d5b2ddca87a6c279bb7cb307167037a1c62d0000000000160014d640ab16f347d1de5aba5a715321a5fc4ba9a5d5455c150000000000220020c46bf3d1686d6dbb2d9244f8f67b90370c5aa2747045f1aeccb77d8187117382024730440220444456f44c721d7ebc69b336979e6c36e0f99d6b74419f60ff14f0a21ff1ddb0022044081ac3dfcd9191f4efb20b685fd2fc7329a26dd45cb1219c92a2166e8ea3fc012102c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee502473044022036a9007f85889200e9ad7a3895d9362d5ca00e64d85a991ebfc324616c33db3e022025e0397cd2610a52fb87c9a233baaa660d28c87d13d6fb6823ad94a03f9b14290121026957e53b46df017bd6460681d068e1d23a7b027de398272d0b15f59b78d060a906000000.', 'data': {'id': '02c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5', 'method': 'openchannel_signed'}}
```

Signed-off-by: Vincenzo Palazzo <[email protected]>
@vincenzopalazzo
Copy link
Collaborator Author

Doesn't resolve the issue, we track it there https://github.com/rustyrussell/lnprototest/runs/5773967845?check_suite_focus=true

@vincenzopalazzo vincenzopalazzo deleted the macros/lnprototest branch March 31, 2022 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant