Recompile test contracts with latest Solidity version + large refactor

docs/contributing.rst

@@ -195,18 +195,16 @@ Within the ``pytest`` scope, :file:`conftest.py` files are used for common code
 shared between modules that exist within the same directory as that particular
 :file:`conftest.py` file.
 
-Unit Testing
-^^^^^^^^^^^^
+Unit Testing and eth-tester Tests
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Unit tests are meant to test the logic of smaller chunks (or units) of the
-codebase without having to be wired up to a client. Most of the time this
-means testing selected methods on their own. They are meant to test the logic
-of your code and make sure that you get expected outputs out of selected inputs.
+Our unit tests are grouped together with tests against the ``eth-tester`` library,
+using the ``py-evm`` library as a backend, via the ``EthereumTesterProvider``.
 
-Our unit tests live under appropriately named child directories within the
-``/tests`` directory. The core of the unit tests live under ``/tests/core``.
-Do your best to follow the existing structure when choosing where to add
-your unit test.
+These tests live under appropriately named child directories within the
+``/tests`` directory. The core of these tests live under ``/tests/core``.
+Do your best to follow the existing structure when adding a test and make sure
+that its location makes sense.
 
 Integration Testing
 ^^^^^^^^^^^^^^^^^^^
@@ -217,25 +215,80 @@ confused with pytest fixtures). These zip file fixtures, which also live in the
 ``/tests/integration`` directory, are configured to run the specific client we are
 testing against along with a genesis configuration that gives our tests some
 pre-determined useful objects (like unlocked, pre-loaded accounts) to be able to
-interact with the client and run our tests.
+interact with the client when we run our tests.
 
-Though the setup lives in ``/tests/integration``, our integration module tests are
-written across different files within ``/web3/_utils/module_testing``. The tests
-are written there but run configurations exist across the different files within
-``/tests/integration/``. The parent ``/integration`` directory houses some common
-configuration shared across all client tests, whereas the ``/go_ethereum`` directory
-houses common code to be shared among respective client tests.
+The parent ``/integration`` directory houses some common configuration shared across
+all client tests, whereas the ``/go_ethereum`` directory houses common code to be
+shared across geth-specific provider tests. Though the setup and run configurations
+exist across the different files within ``/tests/integration``, our integration module
+tests are written across different files within ``/web3/_utils/module_testing``.
 
 * :file:`common.py` files within the client directories contain code that is shared across
   all provider tests (http, ipc, and ws). This is mostly used to override tests that span
   across all providers.
-* :file:`conftest.py` files within each of these directories contain mostly code that can
-  be *used* by all test files that exist within the same directory as the :file:`conftest.py`
-  file. This is mostly used to house pytest fixtures to be shared among our tests. Refer to
-  the `pytest documentation on fixtures`_ for more information.
-* :file:`test_{client}_{provider}.py` (e.g. :file:`test_goethereum_http.py`) files are where
-  client-and-provider-specific test configurations exist. This is mostly used to override tests
-  specific to the provider type for the respective client.
+* :file:`conftest.py` files within each of these directories contain mostly code that
+  can be *used* by all test files that exist within the same directory or subdirectories
+  of the :file:`conftest.py` file. This is mostly used to house pytest fixtures to be
+  shared among our tests. Refer to the `pytest documentation on fixtures`_ for more
+  information.
+* ``test_{client}_{provider}.py`` files (e.g. :file:`test_goethereum_http.py`) are where
+  client-and-provider-specific test configurations exist. This is mostly used to
+  override tests specific to the provider type for the respective client.
+
+
+Working With Test Contracts
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Contracts used for testing exist under ``web3/_utils/contract_sources``. These contracts
+get compiled via the ``compile_contracts.py`` script in the same directory. To use
+this script, simply pass the Solidity version to be used to compile the contracts as an
+argument at the command line.
+
+Arguments for the script are:
+    -v or --version         Solidity version to be used to compile the contracts. If
+                            blank, the script uses the latest hard-coded version
+                            specified within the script.
+
+    -f or --filename        If left blank, all .sol files will be compiled and the
+                            respective contract data will be generated. Pass in a
+                            specific ``.sol`` filename here to compile just one file.
+
+
+To run the script, you will need the ``py-solc-x`` library for compiling the files
+as well as ``black`` for code formatting. You can install those independently or
+install the full ``[dev]`` package extra as shown below.
+
+.. code:: sh
+
+    $ pip install "web3[dev]"
+
+The following example compiles all the contracts and generates their respective
+contract data that is used across our test files for the test suites. This data gets
+generated within the ``contract_data`` subdirectory within the ``contract_sources``
+folder.
+
+.. code-block:: bash
+
+    $ cd ../web3.py/web3/_utils/contract_sources
+    $ python compile_contracts.py -v 0.8.17
+    Compiling OffchainLookup
+    ...
+    ...
+    reformatted ...
+
+To compile and generate contract data for only one ``.sol`` file, specify using the
+filename with the ``-f`` (or ``--filename``) argument flag.
+
+.. code-block:: bash
+
+    $ cd ../web3.py/web3/_utils/contract_sources
+    $ python compile_contracts.py -v 0.8.17 -f OffchainLookup.sol
+    Compiling OffchainLookup.sol
+    reformatted ...
+
+If there is any contract data that is not generated via the script but is is important
+to pass on to the test suites, the ``_custom_data.py`` can be used to store that
+information.
 
 
 Manual Testing
@@ -246,7 +299,7 @@ you can install it from your development directory:
 
 .. code:: sh
 
-   $ pip install -e ../path/to/web3py
+   $ pip install -e ../path/to/web3py
 
 
 Documentation

docs/web3.contract.rst

@@ -610,51 +610,51 @@ Taking the following contract code as an example:
 
 .. doctest:: arrayscontract
 
-    >>> ArraysContract = w3.eth.contract(abi=abi, bytecode=bytecode)
+    >>> arrays_contract_instance = w3.eth.contract(abi=abi, bytecode=bytecode)
 
-    >>> tx_hash = ArraysContract.constructor([b'bb']).transact()
+    >>> tx_hash = arrays_contract_instance.constructor([b'bb']).transact()
     >>> tx_receipt = w3.eth.wait_for_transaction_receipt(tx_hash)
-    >>> array_contract = w3.eth.contract(
+    >>> arrays_contract = w3.eth.contract(
     ...     address=tx_receipt.contractAddress,
     ...     abi=abi
     ... )
-    >>> array_contract.functions.getBytes2Value().call()
+    >>> arrays_contract.functions.getBytes2Value().call()
     [b'bb']
 
     >>> # set value with appropriate byte size
-    >>> array_contract.functions.setBytes2Value([b'aa']).transact({'gas': 420000, "maxPriorityFeePerGas": 10 ** 9, "maxFeePerGas": 10 ** 9})
+    >>> arrays_contract.functions.setBytes2Value([b'aa']).transact({'gas': 420000, "maxPriorityFeePerGas": 10 ** 9, "maxFeePerGas": 10 ** 9})
     HexBytes('0xcb95151142ea56dbf2753d70388aef202a7bb5a1e323d448bc19f1d2e1fe3dc9')
     >>> # check value
-    >>> array_contract.functions.getBytes2Value().call()
+    >>> arrays_contract.functions.getBytes2Value().call()
     [b'aa']
 
     >>> # trying to set value without appropriate size (bytes2) is not valid
-    >>> array_contract.functions.setBytes2Value([b'b']).transact()
+    >>> arrays_contract.functions.setBytes2Value([b'b']).transact()
     Traceback (most recent call last):
        ...
     web3.exceptions.Web3ValidationError:
     Could not identify the intended function with name
     >>> # check value is still b'aa'
-    >>> array_contract.functions.getBytes2Value().call()
+    >>> arrays_contract.functions.getBytes2Value().call()
     [b'aa']
 
     >>> # disabling strict byte checking...
     >>> w3.strict_bytes_type_checking = False
 
-    >>> tx_hash = ArraysContract.constructor([b'b']).transact()
+    >>> tx_hash = arrays_contract_instance.constructor([b'b']).transact()
     >>> tx_receipt = w3.eth.wait_for_transaction_receipt(tx_hash)
-    >>> array_contract = w3.eth.contract(
+    >>> arrays_contract = w3.eth.contract(
     ...     address=tx_receipt.contractAddress,
     ...     abi=abi
     ... )
     >>> # check value is zero-padded... i.e. b'b\x00'
-    >>> array_contract.functions.getBytes2Value().call()
+    >>> arrays_contract.functions.getBytes2Value().call()
     [b'b\x00']
 
     >>> # set the flag back to True
     >>> w3.strict_bytes_type_checking = True
 
-    >>> array_contract.functions.setBytes2Value([b'a']).transact()
+    >>> arrays_contract.functions.setBytes2Value([b'a']).transact()
     Traceback (most recent call last):
        ...
     web3.exceptions.Web3ValidationError:

newsfragments/2797.internal.rst

@@ -0,0 +1,2 @@
+Re-compile all test contracts with latest Solidity version. Refactor test fixtures. Adds a script that compiles all test contracts to the same directory with selected Solidity version.
+

setup.py

@@ -24,7 +24,7 @@
         "configparser==3.5.0",
         "contextlib2>=0.5.4",
         "py-geth>=3.9.1",
-        "py-solc>=0.4.0",
+        "py-solc-x>=1.1.1",
         "pytest>=6.2.5",
         "sphinx>=4.2.0",
         "sphinx_rtd_theme>=0.5.2",

tests/conftest.py

@@ -1,12 +1,17 @@
 import pytest
 
 from eth_utils import (
+    event_signature_to_log_topic,
     to_bytes,
 )
 from eth_utils.toolz import (
     identity,
 )
 
+from web3._utils.contract_sources.contract_data.emitter_contract import (
+    EMITTER_CONTRACT_DATA,
+)
+
 from .utils import (
     get_open_port,
 )
@@ -24,3 +29,78 @@ def open_port():
 
 def pytest_addoption(parser):
     parser.addoption("--flaky", action="store_true")
+
+
+# --- session-scoped constants --- #
+
+
+@pytest.fixture(scope="session")
+def emitter_contract_data():
+    return EMITTER_CONTRACT_DATA
+
+
+class LogFunctions:
+    LogAnonymous = 0
+    LogNoArguments = 1
+    LogSingleArg = 2
+    LogDoubleArg = 3
+    LogTripleArg = 4
+    LogQuadrupleArg = 5
+    LogSingleAnonymous = 6
+    LogSingleWithIndex = 7
+    LogDoubleAnonymous = 8
+    LogDoubleWithIndex = 9
+    LogTripleWithIndex = 10
+    LogQuadrupleWithIndex = 11
+    LogBytes = 12
+    LogString = 13
+    LogDynamicArgs = 14
+    LogListArgs = 15
+    LogAddressIndexed = 16
+    LogAddressNotIndexed = 17
+
+
+@pytest.fixture(scope="session")
+def emitter_contract_event_ids():
+    return LogFunctions
+
+
+class LogTopics:
+    LogAnonymous = event_signature_to_log_topic("LogAnonymous()")
+    LogNoArguments = event_signature_to_log_topic("LogNoArguments()")
+    LogSingleArg = event_signature_to_log_topic("LogSingleArg(uint256)")
+    LogSingleAnonymous = event_signature_to_log_topic("LogSingleAnonymous(uint256)")
+    LogSingleWithIndex = event_signature_to_log_topic("LogSingleWithIndex(uint256)")
+    LogDoubleArg = event_signature_to_log_topic("LogDoubleArg(uint256,uint256)")
+    LogDoubleAnonymous = event_signature_to_log_topic(
+        "LogDoubleAnonymous(uint256,uint256)"
+    )
+    LogDoubleWithIndex = event_signature_to_log_topic(
+        "LogDoubleWithIndex(uint256,uint256)"
+    )
+    LogTripleArg = event_signature_to_log_topic("LogTripleArg(uint256,uint256,uint256)")
+    LogTripleWithIndex = event_signature_to_log_topic(
+        "LogTripleWithIndex(uint256,uint256,uint256)"
+    )
+    LogQuadrupleArg = event_signature_to_log_topic(
+        "LogQuadrupleArg(uint256,uint256,uint256,uint256)"
+    )
+    LogQuadrupleWithIndex = event_signature_to_log_topic(
+        "LogQuadrupleWithIndex(uint256,uint256,uint256,uint256)",
+    )
+    LogBytes = event_signature_to_log_topic("LogBytes(bytes)")
+    LogString = event_signature_to_log_topic("LogString(string)")
+    LogDynamicArgs = event_signature_to_log_topic("LogDynamicArgs(string,string)")
+    LogListArgs = event_signature_to_log_topic("LogListArgs(bytes2[],bytes2[])")
+    LogAddressIndexed = event_signature_to_log_topic(
+        "LogAddressIndexed(address,address)"
+    )
+    LogAddressNotIndexed = event_signature_to_log_topic(
+        "LogAddressNotIndexed(address,address)"
+    )
+    LogStructArgs = event_signature_to_log_topic("LogStructArgs(uint256,tuple)")
+
+
+@pytest.fixture(scope="session")
+def emitter_contract_log_topics():
+    return LogTopics