Plugin Architecture in Python (aka Py3EE)
By Edward D'Souza (June 21st, 2020)
When I came to Python from Java, I reveled in terseness and flexiblity. However, as I'm reading through "Clean Architecture" by Robert C. Martin, I'm seeing that some of the enterprisey ideas from Java are actually quite useful. In this artcle I look at how dependency direction and interfaces can be used to create a plugin architecture in Python.
The toy example here might look over-engineered, but it lets us explore ideas that would be valuable in more realistic contexts when you have multiple people working together.
(The example and many concepts are taken from "Clean Architecture".)
The example problem we'll use is that you're writing an encrypter program. It has to take in characters from stdin, encrypt them using a translation table, and write the output to stdout.
First, the easy part. The functional data-manipulation component that simply translates a character to its encrypted form using a shift cipher.
def translate(char: str): shift = 1 letters = string.ascii_lowercase if char in letters: return letters[(letters.index(char) + shift) % len(letters)] return char
Here is a straight-forward solution to the full problem. It's probably what I would write by default when faced with the above requirements.
import sys from translate import translate def encrypt(): data = sys.stdin.readlines() for line in data: for char in line: print(translate(char)) encrypt()
The code is beautiful in its terseness, but has a few potential issues.
The main problem is that it's mixing together several responsibilies: getting the characters, translation, and outputting the characters. If we had multiple people that wanted to modify those parts at the same time, it would be a mess getting everyone to work together in such a tight space.
As a high-level policy, it also shouldn't be depending directly on low-level details like the "print" function.
One strategy we can use to fix the above problems is to make the code work with generic interfaces. Specific implementations of those interfaces can then be plugged into the policy to create a working system. The policy is then decoupled from low-level details, and the code is organized so that the input and output concerns are nicely separated.
Welcome to Py3EE.
import abc from translate import translate class CharReader(abc.ABC): @abc.abstractmethod def read_char(self): pass class CharWriter(abc.ABC): @abc.abstractmethod def write_char(self, char: str): pass def encrypter(reader: CharReader, writer: CharWriter): def encrypt(): while True: try: char = reader.read_char() except StopIteration: break encrypted_char = translate(char) writer.write_char(encrypted_char) return encrypt
Now, we've clearly and explicitly spelled out the high-level policy of reading a character, encrypting it with the translate function, and writing out the encryped character.
The algorithm does not know or care about the stdin module or the print function. It remains open to be re-used in different contexts, with different input sources and output channels.
Admittedly, like a lot of enterprisey code, our policy is fairly verbose and doesn't actually do anything. To make it useful, we need to implement the CharReader and CharWriter interfaces and then plug them into the policy function.
import sys from enterprise import CharReader, CharWriter, encrypter class MyCharReader(CharReader): def __init__(self): def get_characters(): data = sys.stdin.readlines() for line in data: for char in line: yield char self.iter = get_characters() def read_char(self): return next(self.iter) class MyCharWriter(CharWriter): def write_char(self, char: str): print(char) my_encrypter = encrypter(MyCharReader(), MyCharWriter()) my_encrypter()
What's cool is that the messiness of converting stdin into a stream of characters is contained within MyCharReader. No one else has to be burdened by it.