Our First-Ever Public Data presentation in New Stars of Data!

On April 5th me, together with my colleague Sai presented our first - ever talk in an amazing conference meant for first-time speakers in the data community- New Stars of Data! And it was soo much fun! A while ago I was looking for a cool dataset to try-out hands-on Databricks skills. In my journey, I learn the best from hands-on, so not in a very long time I found a probably the coolest dataset in the internets to play around with in Databricks.

After a while Sai joined us in our company, and i somehow managed to trick him to join me to play around with this dataset. In a while I got this idea to apply for a talk in the New Stars of Data, I have always dreamt to give a talk in the data community, and I felt a combination of this dataset and the conference is perfect, so tricked Sai to apply together as speakers with equal rights! After initially agreeing on aplication we got to choose a mentor, i knew exactly that I would love Marthe Moengen to mentor us, offcourse if she would agree! And we got lucky! Marthe has been such a supportive person in througout this entire journey! She has not only overlooked our application, found time to meet us both online and in- person but has given us very valuable input!

After some time we got accepted! And it just moved upwards from there...

Until a week before the conference... when I found out that the dataset we chose and based all of our talk got licensed! In a few days i found a solution and a way how to synthesize data in case of a need, but luckily it was not needed. Also Ben has helped me to write a legal letter for the NUFORC (thank you for that Ben, you have no idea how much nerves it has saved me!). And... surprisingly, in the second day of Easter (May 31st night) i have received the best answer from the NUFORC, allowing to not only use the dataset but also wanted to see it if possible!

The day of the conference, we have received so much support from our colleagues!

It turned out, we had the biggest amount of people joining our session!

We have utilized the data found went through quite an adventure, from:

 🌊 Importing the UFO observations to the Datalake (ADLS Gen 2), which served as a storage location

👩‍🚀 Moving the data into the Azure Databricks where we transformed it using the Medallion 🥇 🥈 🥉 architecture and the Unity Catalog.

🌷 Enriched the data using Generative AI and DALL-E-3

📊 Moved to Power BI, revealing the insights the data had all along

Aftermath: One of my best friends from the UK has send me a screenshot of our talk, even though she works far away from our field! I feel so lucky to have been given this opportunity to be in this bubble of hapiness and also allowance and freedom to play with the data I want <3

Thank you for everyone for the support and the great messages we received!

I want to especially thank Are, who has been so supportive throughout my entire career, and I look up to a lot from both technical and a person! I plan to cover some technical stuff from this talk in my other posts, stay tuned!

Here is the full talk for those who have not seen it https://www.youtube.com/watch?v=_2_hRV5CTf4&t=1031s