Python is a popular language for data engineering but has some limitations in performance, concurrency, and production deployments. The Rust programming language offers powerful alternatives with strong compile-time and memory safety guarantees. In this talk, I'll explore how data engineers can leverage Rust to build high-performance data pipelines and processing systems. I'll cover the Rust ecosystem for data work, including frameworks and libraries for working with data formats, databases, streaming systems, and scientific computing. By combining Rust and Python, data engineers can harness the benefits of both languages and build robust end-to-end data systems that scale to meet demanding production needs.

After using Python for almost a decade now, mostly for data engineering, I thought I'd give Rust a try to solve some of the problems I face daily as a data engineer. What I learned doing so is that Rust can be a great and valid additional tool for any data engineer, especially in combination with Python.

I'll elaborate on:

• How the Python ecosystem can benefit from integrating Rust in data engineering workflows
• How Rust can be interfaced with Python
• The strengths of each programming language

Further reading material:

• Blog post
• HN discussion
• LinkedIn post
• Website