|
| 1 | +DEVELOPMENT GUIDE |
| 2 | +================= |
| 3 | + |
| 4 | +**Welcome hacker!** |
| 5 | + |
| 6 | +This document will make your life easier by helping you setup a |
| 7 | +development environment, IDEs, tests, coding practices, or anything that |
| 8 | +will help you be more productive. If you found something is missing or |
| 9 | +inaccurate, update this guide and send a Pull Request. |
| 10 | + |
| 11 | +**Note**: `pyenv` currently only supports macOS and Linux. If you are a |
| 12 | +Windows users, consider using [pipenv](https://docs.pipenv.org/). |
| 13 | + |
| 14 | +1-Click Ready to Hack IDE |
| 15 | +------------------------- |
| 16 | +For setting up a local development environment, we recommend using Gitpod - a service that allows you to spin up an in-browser Visual Studio Code-compatible editor, with everything set up and ready to go for development on this project. Just click the button below to create your private workspace: |
| 17 | + |
| 18 | +[](https://gitpod.io/#https://github.com/awslabs/aws-sam-cli) |
| 19 | + |
| 20 | +This will start a new Gitpod workspace, and immediately kick off a build of the code. Once it's done, you can start working. |
| 21 | + |
| 22 | +Gitpod is free for 50 hours per month - make sure to stop your workspace when you're done (you can always resume it later, and it won't need to run the build again). |
| 23 | + |
| 24 | + |
| 25 | +Environment Setup |
| 26 | +----------------- |
| 27 | +### 1. Install Python Versions |
| 28 | + |
| 29 | +Our officially supported Python versions are 2.7, 3.6, 3.7 and 3.8. Follow the idioms from this [excellent cheatsheet](http://python-future.org/compatible_idioms.html) to |
| 30 | +make sure your code is compatible with both Python 2.7 and 3 (>=3.6) versions. |
| 31 | +Our CI/CD pipeline is setup to run unit tests against both Python 2.7 and 3 versions. So make sure you test it with both versions before sending a Pull Request. |
| 32 | +See [Unit testing with multiple Python versions](#unit-testing-with-multiple-python-versions). |
| 33 | + |
| 34 | +[pyenv](https://github.com/pyenv/pyenv) is a great tool to |
| 35 | +easily setup multiple Python versions. For |
| 36 | + |
| 37 | +> Note: For Windows, type |
| 38 | +> `export PATH="/c/Users/<user>/.pyenv/libexec:$PATH"` to add pyenv to |
| 39 | +> your path. |
| 40 | +
|
| 41 | +1. Install PyEnv - |
| 42 | + `curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash` |
| 43 | +1. Restart shell so the path changes take effect - `exec $SHELL` |
| 44 | +1. `pyenv install 2.7.17` |
| 45 | +1. `pyenv install 3.6.12` |
| 46 | +1. `pyenv install 3.7.9` |
| 47 | +1. `pyenv install 3.8.6` |
| 48 | +1. Make Python versions available in the project: |
| 49 | + `pyenv local 2.7.17 3.6.12 3.7.9 3.8.6` |
| 50 | + |
| 51 | +Note: also make sure the following lines were written into your `.bashrc` (or `.zshrc`, depending on which shell you are using): |
| 52 | +``` |
| 53 | +export PATH="$HOME/.pyenv/bin:$PATH" |
| 54 | +eval "$(pyenv init -)" |
| 55 | +eval "$(pyenv virtualenv-init -)" |
| 56 | +``` |
| 57 | + |
| 58 | +### 2. Install Additional Tooling |
| 59 | +#### Black |
| 60 | +We format our code using [Black](https://github.com/python/black) and verify the source code is black compliant |
| 61 | +during PR checks. Black will be installed automatically with `make init`. |
| 62 | + |
| 63 | +After installing, you can run our formatting through our Makefile by `make black` or integrating Black directly in your favorite IDE (instructions |
| 64 | +can be found [here](https://black.readthedocs.io/en/stable/editor_integration.html)) |
| 65 | + |
| 66 | +##### (workaround) Integrating Black directly in your favorite IDE |
| 67 | +Since black is installed in virtualenv, when you follow [this instruction](https://black.readthedocs.io/en/stable/editor_integration.html), `which black` might give you this |
| 68 | + |
| 69 | +```bash |
| 70 | +(sam37) $ where black |
| 71 | +/Users/<username>/.pyenv/shims/black |
| 72 | +``` |
| 73 | + |
| 74 | +However, IDEs such PyChaim (using FileWatcher) will have a hard time invoking `/Users/<username>/.pyenv/shims/black` |
| 75 | +and this will happen: |
| 76 | + |
| 77 | +``` |
| 78 | +pyenv: black: command not found |
| 79 | +
|
| 80 | +The `black' command exists in these Python versions: |
| 81 | + 3.7.9/envs/sam37 |
| 82 | + sam37 |
| 83 | +``` |
| 84 | + |
| 85 | +A simple workaround is to use `/Users/<username>/.pyenv/versions/sam37/bin/black` |
| 86 | +instead of `/Users/<username>/.pyenv/shims/black`. |
| 87 | + |
| 88 | +#### Pre-commit |
| 89 | +If you don't wish to manually run black on each pr or install black manually, we have integrated black into git hooks through [pre-commit](https://pre-commit.com/). |
| 90 | +After installing pre-commit, run `pre-commit install` in the root of the project. This will install black for you and run the black formatting on |
| 91 | +commit. |
| 92 | + |
| 93 | +### 3. Activate Virtualenv |
| 94 | + |
| 95 | +Virtualenv allows you to install required libraries outside of the |
| 96 | +Python installation. A good practice is to setup a different virtualenv |
| 97 | +for each project. [pyenv](https://github.com/pyenv/pyenv) comes with a |
| 98 | +handy plugin that can create virtualenv. |
| 99 | + |
| 100 | +Depending on the python version, the following commands would change to |
| 101 | +be the appropriate python version. |
| 102 | + |
| 103 | +1. Create Virtualenv `sam37` for Python3.7: `pyenv virtualenv 3.7.9 sam37` |
| 104 | +1. Activate Virtualenv: `pyenv activate sam37` |
| 105 | + |
| 106 | +### 4. Install dev version of SAM Translator |
| 107 | + |
| 108 | +We will install a development version of SAM Translator from source into the |
| 109 | +virtualenv. |
| 110 | + |
| 111 | +1. Activate Virtualenv: `pyenv activate sam37` |
| 112 | +1. Install dev version of SAM Translator: `make init` |
| 113 | + |
| 114 | +Running Tests |
| 115 | +------------- |
| 116 | + |
| 117 | +### Unit testing with one Python version |
| 118 | + |
| 119 | +If you're trying to do a quick run, it's ok to use the current python version. Run `make pr`. |
| 120 | +If you're using Python2.7, you can run `make pr2.7` instead. |
| 121 | + |
| 122 | +### Unit testing with multiple Python versions |
| 123 | + |
| 124 | +Currently, our officially supported Python versions are 2.7, 3.6, 3.7 and 3.8. For the most |
| 125 | +part, code that works in Python3.6 will work in Python3.7 and Python3.8. You only run into problems if you are |
| 126 | +trying to use features released in a higher version (for example features introduced into Python3.7 |
| 127 | +will not work in Python3.6). If you want to test in many versions, you can create a virtualenv for |
| 128 | +each version and flip between them (sourcing the activate script). Typically, we run all tests in |
| 129 | +one python version locally and then have our ci (appveyor) run all supported versions. |
| 130 | + |
| 131 | +### Integration tests |
| 132 | + |
| 133 | +Integration tests are covered in detail in the [INTEGRATION_TESTS.md file](INTEGRATION_TESTS.md) of this repository. |
| 134 | + |
| 135 | +Code Conventions |
| 136 | +---------------- |
| 137 | + |
| 138 | +Please follow these code conventions when making your changes. This will |
| 139 | +align your code to the same conventions used in rest of the package and |
| 140 | +make it easier for others to read/understand your code. Some of these |
| 141 | +conventions are best practices that we have learnt over time. |
| 142 | + |
| 143 | +- Don\'t write any code in `__init__.py` file unless there is a really strong reason. |
| 144 | +- Module-level logger variable must be named as `LOG` |
| 145 | +- If your method wants to report a failure, it *must* raise a custom |
| 146 | + exception. Built-in Python exceptions like `TypeError`, `KeyError` |
| 147 | + are raised by Python interpreter and usually signify a bug in your |
| 148 | + code. Your method must not explicitly raise these exceptions because |
| 149 | + the caller has no way of knowing whether it came from a bug or not. |
| 150 | + Custom exceptions convey are must better at conveying the intent and |
| 151 | + can be handled appropriately by the caller. In HTTP lingo, custom |
| 152 | + exceptions are equivalent to 4xx (user\'s fault) and built-in |
| 153 | + exceptions are equivalent to 5xx (Service Fault) |
| 154 | +- Don't use `*args` or `**kwargs` unless there is a really strong |
| 155 | + reason to do so. You must explain the reason in great detail in |
| 156 | + docstrings if you were to use them. |
| 157 | +- Do not catch the broader `Exception`, unless you have a really |
| 158 | + strong reason to do. You must explain the reason in great detail in |
| 159 | + comments. |
| 160 | + |
| 161 | +Profiling |
| 162 | +--------- |
| 163 | + |
| 164 | +Install snakeviz: `pip install snakeviz` |
| 165 | + |
| 166 | +```bash |
| 167 | +python -m cProfile -o sam_profile_results bin/sam-translate.py translate --template-file=tests/translator/input/alexa_skill.yaml --output-template=cfn-template.json |
| 168 | +snakeviz sam_profile_results |
| 169 | +``` |
| 170 | + |
| 171 | +Verifying transforms |
| 172 | +-------------------- |
| 173 | + |
| 174 | +If you make changes to the transformer and want to verify the resulting CloudFormation template works as expected, you can transform your SAM template into a CloudFormation template using the following process: |
| 175 | + |
| 176 | +```bash |
| 177 | +# Optional: You only need to run the package command in certain cases; e.g. when your CodeUri specifies a local path |
| 178 | +# Replace MY_TEMPLATE_PATH with the path to your template and MY_S3_BUCKET with an existing S3 bucket |
| 179 | +aws cloudformation package --template-file MY_TEMPLATE_PATH/template.yaml --output-template-file output-template.yaml --s3-bucket MY_S3_BUCKET |
| 180 | + |
| 181 | +# Transform your SAM template into a CloudFormation template |
| 182 | +# Replace "output-template.yaml" if you didn't run the package command above or specified a different path for --output-template-file |
| 183 | +bin/sam-translate.py --template-file=output-template.yaml |
| 184 | + |
| 185 | +# Deploy your transformed CloudFormation template |
| 186 | +# Replace MY_STACK_NAME with a unique name each time you deploy |
| 187 | +aws cloudformation deploy --template-file cfn-template.json --capabilities CAPABILITY_NAMED_IAM --stack-name MY_STACK_NAME |
| 188 | + ``` |
0 commit comments