How to Easily Extend Pylint With Plugins

Customize static Python code analysis for your business-specific goals

image by Paulius Andriekus on unsplash

Static code analysis and linting are foundational for software development's success. One of the most commonly used static code analyzers for python is pylint

In this article, I’ll show you how easy it is to extend Pylint using it’s built-in plugins architecture.

What is static code analysis?

Static code analysis is an automated attempt to evaluate code quality without running it, as opposed to dynamic code analysis which is an automated attempt to analyze code quality at runtime.

So the input of a static code analyzing tool is either the source code itself or its byte code representation (if exists), while its output depends on the goal we are aiming to achieve.

What is a Linter?

Linters are a subtype of static code analyzer that aimed to detect potential bugs, errors, malpractices, and stylistic issues.

So like all static code analyzers, linters take the source code as input and as output, it provides a list of suspicious code statements that developers are ought to improve.

What is Pylint?

Pylint is arguably the most popular linter for python programming language.

Since python is an interpreted language, it has no compilation time.
Because of that, there’s no built-in error detection mechanism.
Linting is the only way for a python developer to validate their code before execution.

In addition to that, pylint performs style checks to make sure the code is pep8 compliant.

Hands-on example

Let's take a look at a short code example:

This is a very simple example, it opens somefile.txt with write privileges, it writes “hello world” to the file.

Let's run this code:

python example.py

And we get a file with “hello world” written in it.

Lets lint this code with pylint, first we have to install pylint from PyPI warehouse:

pip install pylint

Now let's run pylint on our code:

pylint example.py

Output:

************* Module example
example.py:1:0: C0114: Missing module docstring (missing-module-docstring)
example.py:1:7: W1514: Using open without explicitly specifying an encoding (unspecified-encoding)
example.py:1:7: R1732: Consider using 'with' for resource-allocating operations (consider-using-with)
------------------------------------------------------------------
Your code has been rated at 0.00/10 (previous run: 0.00/10, +0.00)

We’ve got 3 hints:

1 — There’s no module docstring.
This is “just” a convention hint and it has no effect on runtime, but it is very important to use docstrings to make sure your code is communicative and readable.

2 — opening a file without explicitly specifying encoding.
The default encoding is utf-8, but failing to specify an encoding explicitly could be problematic.

3 — a refactoring recommendation to use a with statement when opening the file, this is actually crucial refactor!
In this code, I used the open statement without closing the file, I’m in risk of leaking resources!

In addition to these errors, my code got a score of 0 out of 10 — yeah it is that bad!

So lets quickly refactor the code to solve these 3 problems

Now lets lint this code:

pylint example.py

Result is:

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

Great! there are no errors in the code and the score is 10/10!

Extending pylint with plugins

Pylint has a plugin architecture that allows you to extend it’s functionality easily.

To demonstrate the use of such a plugin, let’s write a short code:

this is a very simple scripts that has 3 unused imports, which I do not want pylint to check (hence the hint to ignore unused imports error to pylint)

But as you can see, I have 2 import statements from the sys module that can be merged into just 1.

Lets lint this file with pylint:

pylint ./app/__main__.py

Output:

-------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 8.57/10, +1.43)

No errors.

This is because pylint has no checkers for duplicated import.

So lets create one of our own :)

First, we need to create a plugin module:

I’ve decided to write this in the __init__.py file so this code runs when the module is loaded.
The register function is recognized by pylints’ plugins manager and it used to register a checker to pylints checkers list.

Now let’s create our checker that checks if there are duplicated from imports:

Pylint checker has several components:

1. msgs — a dictionary where the key is a standard pylint hint index.
   The hint index starts with either a c (convention), r (refactor), w (warning), e (error), f (fatal), and then 4 digits that do not conflict with any other pylint number.
   The value of the dictionary is a tuple of 3 elements, the message to be displayed, the message lookup name, and a recommendation.
2. __init__ method wherein we instantiate a set.
3. And the astroid protocol methods.

Astroid is a visitor that has designated hook methods for every node type.
The two methods for each node type are either visit or leave, and the methods are recognized by the formats: visit_<node_type> or leave_<node_type> .

As expected, the visit method is called when pylints parser visits a node and the leave method is called when it leaves the node.

In our example, we’ve got 2 such methods:

1. leave_module — every time when we leave a module we want to clear the set, so that it won’t pass aggregated content from one module to another.
2. visit_importfrom — this is where the validation happens, when we’ve reached an import from statement, we want to check if we have already encountered an import from a statement with the same module name and recommend the developer to merge the two import statements into 1.
   We will first check if the module name is in the set member of the class, if is, then we will send a message of the type duplication-module-imports
   Otherwise will add the module name to the set and continue.
   This is the reason why we’ve selected a set because it doesn’t allow for duplications and it has a constant [O(1)] time complexity for lookup.

Now we need to run pylint with our plugin using pylints load-plugin options.

set PYTHONPATH=.
pylint --load-plugins pylint_plugins.duplicated_imports_plugin app\__main__.py

Result:

************* Module __main__
app\__main__.py:6:0: W8801: Import from 'sys' module is duplicated (duplication-module-imports)
-------------------------------------------------------------------
Your code has been rated at 8.57/10 (previous run: 10.00/10, -1.43)

Cool! pylint used the rules provided by the plugin and detected the error we wanted it to detect!

Now let's refactor our code and merge the duplicated import statements.

Lint again:

set PYTHONPATH=.
pylint --load-plugins pylint_plugins.duplicated_imports_plugin app\__main__.py

And the result is:

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

Awesome!