Launching an Inner Information Market with Atlan
The Energetic Metadata Pioneers collection options Atlan clients who’ve lately accomplished a radical analysis of the Energetic Metadata Administration market. Paying ahead what you’ve discovered to the subsequent information chief is the true spirit of the Atlan group! So, they’re right here to share their hard-earned perspective on an evolving market, what makes up their trendy information stack, progressive use circumstances for metadata, and extra.
On this installment of the collection, we meet Cristina Perez Martinez, Information Engineer and Architect, and Ezequiel Barbero, Market & Enterprise Intelligence Supervisor at Telefónica Tech, who share how a contemporary information cataloging expertise and column-level lineage will assist a broad imaginative and prescient for information democratization.
This interview has been edited for brevity and readability.
May you inform us a bit about your self, your background, and what drew you to Information & Analytics?
Ezequiel Barbero:
I’ve acquired a Masters in Huge Information and have labored in Information & Analytics since 2002. I began at Telefónica in Argentina with the BI Information Crew engaged on ETLs primarily based in SQL. Then I labored in Information Engineering serving to with Information Science, working with the top of that crew in Argentina.
In 2019, I got here to Spain to work with their Information Science crew on Advertising and marketing Intelligence, and in 2021 I joined Telefónica Tech to start out the BI Crew.
Cristina Perez Martinez:
I began working at Telefónica in 2019 as a Python developer, and I moved to Telefónica Tech in 2021. My crew has primarily been working as Information Engineers and Information Architects for the BI crew.
Would you thoughts describing Telefónica, and the way your information crew helps the group?
Cristina:
Telefónica is split into fairly just a few completely different firms, however as an entire, it’s a Telecommunications Enterprise. Right here, in Telefónica Tech, the digital enterprise unit, we’ve been targeted on digital applied sciences comparable to AI & BD, connectivity and IoT, Cybersecurity, Cloud, and Blockchain.
Our crew is split into two, with a part of the crew targeted on structure and engineering, getting uncooked information, then standardizing and reworking it till it goes into Snowflake, our Information Warehouse. The remainder of the crew is targeted on Information Evaluation, primarily based in Snowflake and coding in SQL. From there, they develop dashboards in PowerBI.
Ezequiel:
Telefónica Tech has a crew engaged on IoT and Huge Information for exterior use circumstances, however our crew is liable for inside use circumstances, supporting the corporate. We assist infrastructure, transformation, and for nearly a yr now, Information Governance.
What does your information stack seem like?
Ezequiel:
Our stack is predicated on Microsoft Azure, and we use Information Manufacturing unit for Orchestration. We use Databricks’ ETL instrument, blob storage, and information lake. Snowflake is Telefónica’s information warehouse.
Why seek for an Energetic Metadata Administration resolution? What was lacking?
Ezequiel:
Our firm has over 6,200 individuals, however our crew is small relative to all the group. So if it’s vital to enhance information democratization, then that wouldn’t be potential with out self-service, and with out Information Governance.
Why was Atlan match? Did something stand out throughout your analysis course of?
Ezequiel:
We have been first in search of a cloud-based SaaS resolution that was straightforward to deploy and straightforward to arrange.
Cristina:
Our aim was to have a spot the place we may create a catalog of knowledge that was accessible sufficient to the remainder of the corporate. It was additionally vital to know the lineage between Snowflake and PowerBI. Our main aim was to know the influence that modifying a supply would have on our information warehouse, so column-level lineage ensures end-to-end visibility and traceability. Moreover, we acknowledge the necessity for a sturdy instrument to strengthen safety of our information platform, permitting us to assign roles and permissions to make sure that solely licensed individuals have entry to particular info, in addition to the power to carry out audits which is important to take care of the integrity and compliance of our information operations.
What do you plan on creating with Atlan? Do you have got an concept of what use circumstances you’ll construct, and the worth you’ll drive?
Cristina:
One of many necessities we had is to create considerably of a market for our information, with every thing primarily based on Atlan belongings, and we’re engaged on launching that to start with of this yr. From there, we’re wanting ahead to populating much more metadata in Atlan and Snowflake.
Sooner or later, we’re enthusiastic about the opportunity of utilizing Atlan AI. Our aim is to make accessing information even simpler for individuals, and with the ability to chat with Atlan about information would make it straightforward for individuals to search out what they want.
Photograph by Mario Caruso on Unsplash
