At LocalStack, we are building a platform that enables development and testing of cloud applications on your local machine. The core is an open source AWS emulator that is primarily written in Python. It is among the top Python projects on GitHub, and has seen a massive uptake in contributions over the past two years. Many Python software developers and architects will relate to the struggles of maintaining a large and complex Python codebase, while keeping developer teams productive. In this talk, we'll explore how we at LocalStack tackle these as we re-create AWS for local development. We'll explain our approaches to automating around AWS specifications, building a highly modular and pluggable system to make it easy for teams to integrate their components, the software patterns we use to keep devs productive, as well as our approach to automated contract testing using pytest.

This talk aims to provide a broad overview of LocalStack and its implementation, and at the same time provide listeners with relatable insights from our experience in architecting, developing, and maintaining a large, complex Python code base.

The outline is roughly as follows:

• Intro: What's LocalStack, who am I, what problems are we solving?
• Software patterns we use, and how they are implemented in Python. These include:
  • A web framework based on WSGI/Werkzeug that implements the chain-of-responsibility pattern: I'll explain how this pattern is similar to WSGI middleware, and demonstrate how it allows us to plug in different types of functionality into the HTTP request processing, when this pattern is beneficial (compared to traditional web frameworks: much more flexible), and what the drawbacks are (hard to debug!)
  • Other patterns found in the code base: Visitor, Adapter, Proxy, Singleton
  • Monkey patching: I will briefly explain how monkey patching works, how it helps us move fast, but also how it can hurt (with real examples of when things blew up in our face), but why the trade-off leans towards using monkey patching.
  • Dynamic code loading: We built an open source framework called Plux that builds a higher-level plugin mechanism around Python entrypoints. will explain both the benefits (super flexible, allows clean code organization) and the drawbacks (pretty hard at times to figure out what's going on) of dynamic code loading.
• Strategies we use to maintain parity with AWS: People are usually skeptical that an idea like LocalStack can even work.
  • We automate a lot around the AWS specs, for example we get weekly PRs when service APIs change. I'll demonstrate how we do server-side code generation from the AWS specs, and how that helps us.
  • Snapshot testing: We have built our own contract testing framework around pytest. We use that to write contract tests against AWS, that we then run against LocalStack to ensure parity. I'll go into a bit of details of the challenges with this approach, but also how we couldn't work without it.
• Wrap up and conclusion: I will tie together all the presented solutions and summarize how they help us solve the challenges. After this, the audience should have a deeper appreciation for how Python helps build something like LocalStack. I'll conclude by pointing out some of the overarching themes in this talk (like the tradeoffs between introducing certain patterns on maintainability).