February 7, 2024

Learning and Sharing in Public with Dagster Lab's Pedram Navid

How to Subscribe
Share In

In this episode Sean is joined by Pedram Navid, Head of Data Engineering at Dagster Labs. They discuss the unique challenges and opportunities in the realm of data engineering, particularly the culture of learning and sharing within the field.

Pedram discusses the traditionally guarded nature of data engineering, contrasting it with the more open-source approach in software engineering. He highlights the potential downsides of this secrecy, such as the difficulty in learning best practices and innovating. The discussion also touches on the balance companies must strike between contributing to communal knowledge and protecting valuable data and intellectual property.

Pedram shares insights from his experiences at Dagster Labs, including the development of the Dagster Open Platform and its impact on fostering a culture of openness in data engineering. Additionally, they explore the future of collaboration in the field, considering emerging technologies and methodologies that could further encourage sharing and innovation over the next 5-10 years.

Links:

Other Podcast

March 27, 2024

Prompt Injection Attacks with SVAM's Devansh

In this episode, we dive deep into the world of prompt injection attacks in Large Language Models (LLMs) with the Devansh, AI Solutions Lead at SVAM. We discuss the attacks, existing vulnerabilities, real-world examples, and the strategies attackers use.

March 20, 2024

The Hidden Hazards of Managing Unstructured Data with Skyflow's Joice John

In this episode, Joice John, Senior Product Manager at Skyflow, joins the show to discuss the complexities of managing privacy and security with unstructured data. Joice explains what unstructured data is and its distinction from structured data, and then dives into the technologies that tackle these challenges.

March 13, 2024

Building Enterprise-grade Security with Skyflow's Daniel Wong

Daniel Wong, Head of Security and Compliance at Skyflow, is back for his third appearance. Daniel discusses his extensive career at the forefront of security engineering, having worked with industry behemoths like Oracle, Salesforce, and CrowdStrike.