5 Tools and Techniques for Better Python Project Setups

Let’s create a well-designed Python project.

Photo by Octavian Dan on Unsplash

1. To Write Better Code and Avoid Common Mistakes

Sometimes when we are adding new features to our application it can affect the existing code or break its functionality.

To avoid this situation we do write unit tests and run them every time to ensure that the new functionality that we have added is not harmful to the application.

So about tests, there are libraries and frameworks for writing better tests and running them easily such as Pytest and tox.

Pytest can help us not just run tests but also configure how to run them, which files to run, etc…Pytest has a configuration file pytest.ini where you can describe its configuration for example which version should be Pytest or which are test files like the following.

# pytet.ini
[pytest]
minversion = 6.0
addopts = -ra -q — cov=src — cov-report=html
python_files = test_*.py

Also, it has integrations one of them is pytest-cov which integrates coverage with Pytest, coverage can help us by providing information about which part of the app is covered by a test and which is not, it also provides a coverage report in various formats XML, JSON, HTML and shows how many percent of code is covered by tests.

Screenshot from https://coverage.readthedocs.io/en/6.3.2/

Tox is a generic virtualenv management and test command-line tool.
Checking that your package installs correctly with different Python versions and interpreters
Running your tests in each of the environments, configuring your test tool of choice
Acting as a frontend to Continuous Integration servers, greatly reducing boilerplate and merging CI and shell-based testing.
Tox also has its configuration file.

[tox]
isolated_build = True

envlist =
    py{38}

[testenv]
usedevelop = true
deps =
    -r src/requirements_dev.txt

These kinds of tools can help you to ensure that any changes done will not affect features done before.

2. How to Avoid Code Smells

Sometimes there can be unused variables in the code or the written code style can not correspond to PEP8 rules. Here static code analyzers come to help us.
So here are a couple of static code analyzers with their configuration example.
Pylint, Flake8, MyPy.
These tools can help us to avoid things like dead code, none applicable code for example breaking PEP-8 rules, unused variables in code, etc...
Pylint config: create .pylintrc file

[MESSAGES CONTROL]
disable=
    missing-docstring,
    too-few-public-methods
[REPORTS]
output-format=colorized
files-output=no
reports=no
evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)

Flake8 config: create .flake8 file

[flake8]
ignore = E203, E266, E501, W503, F403, F401, E402
max-line-length = 120
max-complexity = 18
select = B,C,E,F,W,T4,B9
exclude =
    .git,
    tests

These kinds of tools can help you to define a code style for the whole application and developers will keep it!

3. How to Avoid Breaking the Working Application

Imagine a situation where someone pushed a code without running all the tests and static code analyzers, which may affect existing features.
So to avoid this kind of situation pre-commit came to help us.
Pre-commit is a framework that creates git hooks to make sure your code is written the is corresponding to your defined code style.
It scans your source code and runs all the checkers you will define in your pre-commit config file: .pre-commit-config.yaml

repos:
  - repo: 'https://gitlab.com/pycqa/flake8'
    rev: 3.8.2
    hooks:
      - id: flake8
        name: Style Guide Enforcement (flake8)
        args:
          - '--max-line-length=120'
  - repo: 'https://github.com/pre-commit/mirrors-mypy'
    rev: v0.720
    hooks:
      - id: mypy
        name: Optional Static Typing for Python (mypy)

These kinds of tools can help you to avoid harmful code in your git repositories.

4. Vulnerability Scan

Every application can be hacked and there is a risk of a data breach or source code breach, data encryption, application breakdown, or any other harmful things.
There are a bunch of tools for vulnerability scan, but we will have a look at Sonarqube. Sonarqube is an open-source powerful tool for code quality and security scanning, and one of the leading tools in this industry.
More in the official documentation.
You can set up a local Sonarqube server using a Docker image and define sonar-project.properties

# must be unique in a given SonarQube instance
sonar.projectKey=python_app_blueprint
# --- optional properties ---
# defaults to project key
#sonar.projectName=My project
# defaults to 'not provided'
#sonar.projectVersion=1.0
# Path is relative to the sonar-project.properties file. Defaults to .
#sonar.sources=.

# Encoding of the source code. Default is default system encoding
#sonar.sourceEncoding=UTF-8

5. Structuring Your Application

You probably have read about SRP — single responsibility principle and variable naming, so they are not only about programming in python, java, c#, or any other language, they are general-purpose principles and work almost everywhere even in folder creation.

So it is better to split your application into different folders and give them corresponding names, and store your data separately.
For example:

├── LICENSE
├── Makefile
├── project.toml
├── pytest.ini
├── README.md
├── setup.cfg
├── setup.py
├── sonar-project.properties
├── .ci
│   ├── some_pipeline.yaml
│   ├── scripts
├── src
│   ├── app
│   ├── requirements_dev.txt
│   └── requirements.txt
├── tests
│   ├── integration
│   │   └── __inti__.py
│   └── unit
│       └── __init__.py
└── tox.ini

Here’s an example application blueprint. Thanks for reading.