Plugin Architecture in Python (aka Py3EE)

By Edward D'Souza (June 21st, 2020)

When I came to Python from Java, I reveled in terseness and flexiblity. However, as I'm reading through "Clean Architecture" by Robert C. Martin, I'm seeing that some of the enterprisey ideas from Java are actually quite useful. In this artcle I look at how dependency direction and interfaces can be used to create a plugin architecture in Python.

The toy example here might look over-engineered, but it lets us explore ideas that would be valuable in more realistic contexts when you have multiple people working together.

(The example and many concepts are taken from "Clean Architecture".)

Sections

Example Problem

The example problem we'll use is that you're writing an encrypter program. It has to take in characters from stdin, encrypt them using a translation table, and write the output to stdout.

First, the easy part. The functional data-manipulation component that simply translates a character to its encrypted form using a shift cipher. 

def translate(char: str):
    shift = 1
    letters = string.ascii_lowercase
    if char in letters:
        return letters[(letters.index(char) + shift) % len(letters)]
    return char

Basic Solution

Here is a straight-forward solution to the full problem. It's probably what I would write by default when faced with the above requirements. 

import sys
from translate import translate

def encrypt():
    data = sys.stdin.readlines()
    for line in data:
        for char in line:
            print(translate(char))

encrypt()

The code is beautiful in its terseness, but has a few potential issues.

The main problem is that it's mixing together several responsibilies: getting the characters, translation, and outputting the characters. If we had multiple people that wanted to modify those parts at the same time, it would be a mess getting everyone to work together in such a tight space.

As a high-level policy, it also shouldn't be depending directly on low-level details like the "print" function.

One strategy we can use to fix the above problems is to make the code work with generic interfaces. Specific implementations of those interfaces can then be plugged into the policy to create a working system. The policy is then decoupled from low-level details, and the code is organized so that the input and output concerns are nicely separated.

Enterprise Solution

Welcome to Py3EE.

High-level Policy

import abc
from translate import translate


class CharReader(abc.ABC):
    @abc.abstractmethod
    def read_char(self):
        pass


class CharWriter(abc.ABC):
    @abc.abstractmethod
    def write_char(self, char: str):
        pass


def encrypter(reader: CharReader, writer: CharWriter):
    def encrypt():
        while True:
            try:
                char = reader.read_char()
            except StopIteration:
                break
            encrypted_char = translate(char)
            writer.write_char(encrypted_char)

    return encrypt

Now, we've clearly and explicitly spelled out the high-level policy of reading a character, encrypting it with the translate function, and writing out the encryped character.

The algorithm does not know or care about the stdin module or the print function. It remains open to be re-used in different contexts, with different input sources and output channels.

Enterprise Implementation

Admittedly, like a lot of enterprisey code, our policy is fairly verbose and doesn't actually do anything. To make it useful, we need to implement the CharReader and CharWriter interfaces and then plug them into the policy function. 

import sys
from enterprise import CharReader, CharWriter, encrypter


class MyCharReader(CharReader):
    def __init__(self):
        def get_characters():
            data = sys.stdin.readlines()
            for line in data:
                for char in line:
                    yield char

        self.iter = get_characters()

    def read_char(self):
        return next(self.iter)


class MyCharWriter(CharWriter):
    def write_char(self, char: str):
        print(char)


my_encrypter = encrypter(MyCharReader(), MyCharWriter())
my_encrypter()

What's cool is that the messiness of converting stdin into a stream of characters is contained within MyCharReader. No one else has to be burdened by it.

∎