This is the website for an older EuroPython. Looking for the latest EuroPython? Click here!
Skip to main content

DuckDB: Bringing analytical SQL directly to your Python shell

Level:
beginner
Room:
terrace 2b
Start:
Duration:
45 minutes

Abstract

In this talk, we will present DuckDB. DuckDB is a novel data management system that executes analytical SQL queries without requiring a server. DuckDB has a unique, in-depth integration with the existing PyData ecosystem. This integration allows DuckDB to query and output data from and to other Python libraries without copying it. This makes DuckDB an essential tool for the data scientist. In a live demo, we will showcase how DuckDB performs and integrates with the most used Python data-wrangling tool, Pandas. Besides learning about DuckDB's main charactestics, users will also experience a live demo of DuckDB and Pandas in a typical data science scenario, focusing on comparing their performance and usability while showcasing their cooperation. The demo is most interesting for an audience familiar with Python, the Pandas API, and SQL.

TalkPyData: Data Engineering

Description

The talk is catered primarily towards data scientists and data engineers. The talk aims to familiarize users with the design differences between Pandas and DuckDB and how to combine them to solve their data-science needs. We will have an overview about five main characteristics of DuckDB. 1) Vectorized Execution Engine, 2) End-to-end Query Optimization, 3) Automatic Parallelism, 4) Beyond Memory Execution, and 5) Data Compression.


The speaker

Pedro Holanda

Pedro Holanda

I’m a Post-Doc based in Amsterdam and a member of the Database Architecture group at CWI. I’m currently working as COO @ DuckDB Labs.


← Back to schedule