In this talk, we’ll present dbt, an open-source data transformation framework that lets analysts and engineers collaborate using their shared knowledge of SQL.
We’ll dive deep into the analyst persona and how dbt empowers them to produce data that their entire organization can trust independently of software engineers.
We’ll explore how dbt embraces software engineering best practices like modularity, version control, testing, and documentation to help analytics teams ship as effectively as software teams.
Using a fictional e-commerce store as an example, we’ll also illustrate how a transformation written in dbt is simpler and cleaner than its Python equivalent.
We’ll then look at how dbt serves as the nexus for analytics workflows at Spotify.
We’ll explore what Spotify’s tech stack used to look like, why they adopted dbt, and what the dbt experience looks like for different stakeholders within the organization.
Finally, we’ll cover what’s on dbt’s roadmap - power user tools like multi-project deployments and a Python SDK - built for software engineers.
__________________________________________________
About Kshitij Aranke
Kshitij is a senior software engineer at dbt Labs, working on the popular open-source data transformation tool dbt. Previously, he worked in analytics at Vouch Insurance (using dbt!) and in big data at Amazon and LinkedIn. He is based in San Francisco and listens to podcasts on long walks in his spare time.
About Brandon Segal
Brandon Segal is a Philadelphian working as a Senior Analytics Engineer at Spotify, working within their internal data platform team to understand how the teams at Spotify are collecting data and transforming it into insights. His ways of working have been greatly influenced by his previous experience as a data scientist and software engineer at Capital One with their ML feature platform. Since joining Spotify, he has worked on bringing software engineering best practices to his insights teams by introducing standard practices like semantic versioning, integration testing, and monitoring to improve data product quality.
__________________________________________________
About the Conference
The Philly Emerging Technologies for the Enterprise (ETE) is the Mid-Atlantic's premier developer's conference. Entering its 17th year, we've brought world-class speakers - including some local favorites - to speak about leading-edge technologies being used today, and emerging technologies that will be important for attendees to know about in the near future.
__________________________________________________
About the Host
Philly ETE is hosted by Chariot Solutions, a software development consultancy.
For over 20 years, companies of all sizes and industries have
looked to Chariot as a partner to help them solve their toughest software challenges, and move their business forward.
Are you a business looking to build and manage software solutions optimized for your unique needs? We're here to help. We’ll apply our strategic, high-touch approach and extensive tech experience to solve your most complex business problems. Reach out today. Visit us at chariotsolutions.com/
__________________________________________________
Sponsored by Pinnacle 21
Employees at Pinnacle 21 do tech work that means something. Their software enables key transitions in the clinical trial data pipeline and streamlines regulatory approvals for drugs and medical devices, getting new treatments to patients faster. Pinnacle 21 supplies its SaaS platform to the US FDA, Japan’s PMDA, 24 of the top 25 biopharma firms, and many more biotechs. It ensures clinical trial data comply with the standards of both government agencies and trial sponsors. On its way from the clinic to a government agency, a data point goes through many steps, and Pinnacle 21 has processed over 30 trillion (with a T!) of them. With strong product-market fit and an aggressive roadmap fueled by their 2021 acquisition by Certara (NASDAQ: CERT), the team is quickly becoming the one-stop data brokerage for the life sciences. Learn more: pinnacle21.com
Sponsored by Lutron
Lutron is the worldwide leader in lighting, automated shade and temperature controls, with headquarters in Lehigh Valley, Pennsylvania and engineering offices in Philadelphia, Boston, South Florida and Austin. We are a privately held manufacturing company with over 4000 design and product patents. We are at the forefront of innovating IoT products for smart homes and connected buildings. We provide solutions for residences that span from 500 sq. ft apartments to 10,000 sq. ft homes and commercial solutions for small office spaces to entire building complexes such as the New York Times Building, Lincoln Financial Field in Philadelphia, Empire State Building and the Guggenheim Museum. Learn more: www.lutron.com/en-US/pages/de...