Tobias Macey: Hello, and welcome to the Data Engineering podcast, the show about modern data management.
Data lakes are notoriously complex.
For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst is an end to end data lake has platform built on Trino, the query engine Apache Iceberg was designed for.
Starburst has complete support for all table formats, including Apache Iceberg, Hive, and Delta Lake.
And Starburst is trusted by teams of all sizes, including Comcast and DoorDash.
Want to see Starburst in action? Go to data engineering podcast.com/starburst
today and get $500 in credits to try Starburst Galaxy, the easiest and fastest way to get started using Trino.
Your host is Tobias Macy, and today I'm interviewing Praveen Gujjar about product management for data and analytical platforms. So, Praveen, can you start by introducing yourself?
Praveen Gujar: Yeah. Thanks, first of all, for inviting me to the session. My name is Praveen Gujjar. I currently work as a director of product at LinkedIn,
but my journey in product management has been,
ticket test right now.
Especially after my MBA, I started with Amazon, in a product management capacity,
worked in Amazon Web Services,
specifically in building a lot of the early generations of, AWS or public cloud, products as well.
Then
I joined Twitter,
where I basically,
played a very critical role for a short duration, though,
on moving some of the, you know, on premise workload that we'd all had to public cloud.
Later on for the last, 7 years close to now, I've been with LinkedIn. At LinkedIn, I've paid multiple roles, specifically building data platforms and products and also
digital advertising. So in summary,
my experience spans across data products, obviously,
building data products and platforms,
digital advertising,
AI, and cloud as well. So a lot of,
different experience and in the process, basically, have built, multibillion dollar businesses for these organizations as well.
Fortunately, I have been part of big tech company. I have seen these, internal operations very closely as well. So, yeah, that's my introduction in short. And do you remember how you first got started working in the data space?
Yeah. As I said,
I started with Amazon
after my MBA, but I do have a engineering degree where I was a developer before as well. Transitioned into product management role after
my MBA.
I first started with, Amazon Web Services, as I said, where we built
a lot of the early public, cloud,
products as well.
My area of expertise was, building easy two instances, a lot more focused towards, storage optimized and IO optimized instances.
And these really were instances that were predominantly
catering to the needs of our workloads, such as,
Redshift, for example, Amazon's Redshift,
or, data warehouses, as well as in memory,
in memory workloads like SAP HANA and other things as well. So a lot of analytics product data,
warehouses, data lakes that were basically coming up during that time, we built specifically
storage optimized and IO optimized instances at that time. So that's really how I got into the data product and platform space as well.
Worked in that space for about, you know, in AWS for about 5 years.
Then, again, continued very similar trend with Twitter as well where
we negotiated contract with some of the biggest public cloud providers to move Twitter's workload to public cloud. Ended up, doing, moving their, couple of the Huddl clusters,
to,
cloud as well. And last but not the least, continued the same trend within LinkedIn as well.
Was one of the early PMs who started data products and platforms division within LinkedIn. As LinkedIn
started to become larger organization as well, we clearly saw a need to, have a specific, product management team
leading data product and platform. So there, I built a portfolio of platforms and products as well.
My current work is a little bit different than, just data products and platforms, but I still,
very closely associated with enterprise grade data products in the digital,
advertising space,
specifically catering to the needs of
large customers,
like maybe a media agency,
big brands,
as well.
Tobias Macey: Product management, as it is commonly thought of for people in the engineering space is largely focused on those end user features where a product in that case is maybe a website or
Obviously, for organizations like LinkedIn, Twitter, Amazon, data is a huge component of that functionality. And I'm wondering if you can give some comparison between the typical view of product management
and product management as it exists in this view of being very data oriented and working very closely with the data teams?
Praveen Gujar: Yeah. I think that's a really good question. So I mentor,
multiple product management aspirants as well, and most of them really
carry this view that, product managers build,
UI experiences for our customers,
build apps for our customers alone. But you have product managers that have a varied
experiences in background, and they build different kind of products as well. If I can loosely talk about 3 specific,
product types of product managers.
1,
I would say is a consumer product manager who built in, apps like the DoorDash app, for example, or the Airbnb app as well.
2nd is b to b product managers who build b to b specific products. It can be a mixture of data products as well as
user facing UI experience as well for large, like, for another organization.
These are the b to b product managers. And the third one basically is we are product managers who build data products and platforms
and platforms in general, for both internal and external use cases for organizations.
So these are 3 loose categories.
But if you stick a step back, every organizations,
when they basically scale, they clearly see a need
for building in house data platforms and products as well.
Few things that really, matters the most for them, data obviously, as we say.
It's been old saying now already that data is a new oil.
So data definitely is a clear mode for a lot of these organization as well. So making sure that the data is clean, reliable,
and compliant with regulatory guidelines in a way is very key for these organizations as well.
Second one is scale, the massive amount of scale that this organization has. That means it introduces a lot of complexity into the data ecosystem.
So it's critical to basically build for longer term platforms or products.
3rd, I think, understanding data flow within the ecosystem is very, very key to build every single product and features
at the scale of these larger organization as well. Hence,
investing in data platforms and products in house becomes very important,
because
off the shelf,
our fully managed services
do not often provide the flexibility,
the customization
needed to meet the needs of these larger organization as well.
So where does product manager come in? So you can imagine now that,
you are pretty much building data products and platform for your entire organization. It may be 10,000
plus people or 15 or even 100 k plus people,
depending upon how big the organization is.
And, you need a a in a spoken hub condition, you need a hub kind of a personality or a function
that,
really enables these teams to come together, work on a common vision,
solve the complex problem in a good way because you basically are now interacting with engineers,
back end engineers or infra engineers,
as well as data scientists and other folks as well. So how can you bring everyone together under the common vision becomes important. That's where really product managers play a role. So I was speaking to someone in Stripe.
Stripe, has historically
not had product managers in this particular space, but they clearly have seen need for having product managers in the space in the last 3 to 4 years, and they are building
a product manager,
organization for data platform and products as well, where you can clearly demonstrate,
results from, having a function like this.
Tobias Macey: For cases where you are working on determining what features to provide
during the product ideation phase,
because of the fact that in very data oriented products and capabilities, the data that you have becomes a limiting factor.
Wondering if you can talk through some of the tensions that exist in the process of figuring out if this is something that you would like to do and then doing some of that feasibility analysis and deciding what is a reasonable amount of investment to put into enabling this feature
versus the proposed gain that you're going to get from the amount of engagement you're trying to drive with it, especially in the case where either you don't have that data yet or making that data available
in a very high availability,
low latency use case is going to require a significant technical investment.
Praveen Gujar: Yeah. I think that's a really good question as well. Some of the, like, as an organization, you really
go back and forth in making these decisions as well. And if you are, there are both, cultural and organizational aspects that are necessary, to make this happen, and, also, there are technical,
aspects that are necessary to happen. Let's just talk about the organizational
aspects first to begin with, and then we can talk about the technical aspects as well. First and foremost, I think,
large organization like this have to structure their internal workings to,
enable teams to build for these large data products and platforms as well. That, can be as simple as,
having organization structure. For example, a city organization
that completely manages the infra, the data platforms
necessary as well is very key to the thing. Then basically comes the cultural aspects as well. Rolling out the culture, that is necessary for to these,
teams to actually,
use the data platform and products, that are built in as well. For that, we need to demonstrate value because no team basically will just adapt to our data product or platform if, unless it delivers on the similar value that you have talked about. It has basically
so that use cases,
meets the complexity requirement,
and the performance requirement, be it latency, throughput, or anything else as well. So, it's equally important to basically demonstrate the value, but, culturally,
activating these teams to use these data platform products or some kind of a mandate also becomes very clear as well. So these are definitely organization and cultural challenge that the organization has to go through to ensure
the data platform and products are first built. They are solving key use cases or problems for the, internal customers. And 3rd, they are actually used by these internal customers as well. The second piece,
is very interesting question as well. When you talk about
the feasibility of building the data product, the data availability or anything else. It's a very complex questions as well. I often see this as,
a 3 circular VIN diagrams as well where the user needs are the business needs basically be a very critical role as well. How strategically aligned some of the capabilities that we have to build for business needs is very important. The second, basically, is obviously what you talked about, the technological,
feasibility that includes availability of the data,
available technology to meet the performance requirements, for example, is very critical as well. 3rd, basically, is what is the leverage, that we are creating by building these capabilities as well. I think very initially, we talked about what is the the minimum capability that you can build that produces the highest leverage opportunity.
That is very key as well.
So,
thinking about this intersection is very important as well, and ruthlessly prioritizing
is key.
And, some of this basically may also, you may have to create requirements for other teams that may not be something that you immediately
control or, work with as well.
It can be building data pipelines.
It can be basically upgrading your software of our infrastructure to make sure that we have performance both in terms of latency and throughput, can be managed as well. That's a very key thing as well. Like, if you take an example of today, like, AI is definitely acting as a catalyst for a lot of these capabilities as well. Organizations are rushing into upgrading their infrastructure and capabilities
to ensure,
we go through the massive inflow of data that is expected because of,
AI
related products and features. And in addition to that, like, what is needed to train these models as well. So
for these,
for our organization to be successful,
you need to really think 5 years, 10 years ahead and be committed to invest in these. Because at this scale,
you cannot share a share in 6 months or 1 year. So really planning for the future and working towards it very important.
Tobias Macey: To that point of the significant investment, both in terms of engineering capacity and financial input to that engineering headcount to be able to support these features and the time horizons that you're looking towards,
the fact that you're trying to build these systems in a way that they could be used in an interactive capacity by a large number of users, you ideally don't want to have to create
individual point solutions every single time you want to create a new feature.
And so I'm wondering how you think about that congregation
or grouping of the types of features in terms of the technical platforms that are going to be required to support them so that you can try to build those technical capabilities in a way that they can be used across a wide number of potential use cases both now and into the future?
Praveen Gujar: Yeah. I think that's a really good question.
So platformization
is, kind of,
both tabooed and can be both,
can be, both tabooed and also overused,
within,
organizations
as well. Everyone wants to build a platform, but people are also really,
have a negative reaction to platform because there's a couple lot of time in building, and we often tend to overdo platformization as well. So finding a really,
a good balance is very key as well. What is the right amount of platformization that provides them maximum leverage?
So that's a very critical component of, of it. The second piece of the puzzle also is,
really,
doing extensive research across multiple teams as well to understand what are the common capabilities that are necessary that can be, platformized as well to provide the maximum leverage. I think this goes back to the point of right amount of platformization as well. You want to provide
common set of capabilities packaged together through an API,
but you also want to provide ability for these, teams to customize that on top of,
the platform as well. So that modularity, that customization capability is very key,
important piece of the puzzle as well. So I think that's really where,
having that, these two criteria is very important as well. And the 3rd piece of the puzzle also is,
clearly, in all of these bigger organizations, you have, 2 different types of platforms as well. 1 is a product platform. 1's a tech platform as well.
So there is need to build,
these 2 capabilities very clearly because they serve different
use cases as well and often have overlaps
as well. So for example, our product platform can be a messaging platform that can be used by
multiple teams,
to build messaging capabilities as well. Common set of capabilities that it provides that can can be customized by other, teams as well. Whereas a tech platform can be an experimentation platform as well that can be used by both your b to b organizations as well as b to c organizations as well to run the experimentation
as well. So building these, 2 different platforms, you need to make sure that you invest,
in these 2 different platforms,
as well. And also, as I said before, like, really looking at the long term, you should have a longer term road map, like 5 to 10 years to really understand, like, how do you evolve this, platform as well. All the technology changes every few year as well. Having that vision is very important as well. And as soon as there are critical milestones in technology, you need to make sure that you need to work on these platforms, customer like, evolution of these platforms and products
very clearly as well. So, hence yeah. I think to your question,
longer term vision is very important.
Having a tops down approach is very important as well.
Making sure that you build the, right amount of platformization,
and not fall to the trap, trap of old platformization is important as well. And about all these things, going back to my earlier,
response as well,
having,
an organization structure and culture is very important. We have top down execs kind of mandate sometimes as well in investment in these and making these hard decisions are important.
I think in many organizations,
there will be a time where,
every 5 years or 7 years where you have to see that my infrastructure,
my platform, and product capabilities are now getting accelerated. So I need to deliberately invest next couple of years, for example, to overall all of them to set myself for another 5 to 7 minutes.
This is key aspect of it.
Many of the organizations that I have worked on have deliberately
made this a priority
organization wide. Otherwise, it simply cannot turn you cease to exist.
Tobias Macey: To your point of the technology moving fast, the requirements constantly
substantial amount of early
A substantial amount of early investment in data platforms and data capabilities was oriented around the data warehouse and supporting business intelligence
use cases.
We've now moved beyond that largely to these real time systems being able to do things like embedded analytics, data products for end users
That has substantially changed the types of technologies that we need to be able to support those use cases while also maintaining the analytical capacity of the warehouse and business intelligence.
I'm wondering if you can talk to some of the ways that you see technical teams breaking down the categories of functionality into consumable chunks
so that they can figure out how to build these new technologies or bring in new platforms or components to be able to support these growing use cases and the increased demands on the capabilities of those data platforms to be able to operate in a higher capacity and lower latency format?
Praveen Gujar: That is an awesome question as well. I think, specifically,
as you described, right, this needs a very methodical and deliberate approach,
as well. And this is really where the product management function plays a very critical role as well.
I think the we recently went through something very similar. So I'm gonna basically
layover the framework on what this was actually, did as well.
It all really starts from,
working backwards from the customers as well. At, when I was at Amazon, we used to have this, like, customer obsession story. Right? Like, working backwards from customers,
various organizations,
especially big organizations,
follow different kind of a methodology or framework to,
eventually,
achieve this goal. Like, working back from customers, how can
I ensure
that we build the right product as well? So, one such framework that can be used is jobs to be done as well. So jobs to be done is a framework that basically enables you to first map
all the key jobs or all the key problems that your customers wants to solve, be it internal or external.
And with that as a foundation,
now you can actually build upon that to understand how do they solve this problem and how can we capability how can we build capabilities both in platforms and products to solve this problem as well.
So I think a simple exercise that, teams like this can actually do is
to begin with, map all the jobs to be done. You start with certain key capabilities or key problems that your customers want to solve. It can be basically like what you talked about, real time analytics that should be available as well, and a massive amount of data that can be,
used to try and, train the AI models,
as well, and your infrastructure that basically to support lower latency and higher performance to train these models as well. And,
specifically, if you work in an ecosystem like our ecosystem,
where real time nature is very critical as well to make bidding decisions, to make serving decisions.
How do you basically,
leverage the
upcoming technologies
to increasingly,
reduce the latency in making these decisions as well is where it becomes a very key. So you map all these capabilities
as jobs to be done, framework as well.
Then, what you start doing is writing user stories or
customer, personas and user stories basically to represent these jobs to be done as well. As simple to me, if my user wants to run-in
product, you know, like
embedded analytics, basically,
with my listing. These are the journey that we'll actually go through as well.
So you come you come up with a few user stories
to that. And then each user stories, you start mapping what are the capabilities I need to build to,
achieve these,
fulfill these use cases as well. In those capabilities, what you'll also see is a clear trend of similar capability that are needed by multiple such user stories and multiple such persona as well. That's really the cue for you to, build platforms,
for those meeting those capabilities as well. These personas may be different. These use cases may be different as well, but those commonalities in the user story start emerging. So you map those commonalities
and capabilities,
the common capabilities,
to come up with the platform as well. Now you have jobs to be done, jobs that you have. You have user stories, and now you have collated
common capabilities.
And, the remaining capabilities that we have to build can be the products that are built on top of these capabilities as well.
Now this exercise gives you a huge metrics that tells your organizations
what platforms are either necessary, are missing in our portfolio,
what platforms basically needs to be upgraded
to meet the demand of the next few years as well, and what products are should be customized or built on top of these platforms as well to make sure we serve the needs of our internal customers and also the end consumers who eventually will see the benefit of these as well. So this is really a a elaborate exercise that sometimes can take you months to really go through. And, again, I cannot emphasize the fact that how critical it is to have a culture like this and the top down buying like this to really have this kind of an approach.
And one day, then you can actually fully understand your technology,
tech portfolio,
and the missing gaps in it and what is needed to really
overhaul it for the next few years.
That's really the TLDR
version of how, organizations go about this.
Tobias Macey: In your experience of working in this space of technical product management around these very data focused features, I'm wondering if there is an example of a product either specifically or a general category of product that you can talk through the overall journey from this is the goal that we're trying to achieve through the process of figuring out how to break that down into those technical components and the custom technologies that you've had to build to help people understand all the different stages of the life cycle
and the types of organizational
involvement that you have to bring in to be able to bring that type of project to a successful conclusion?
Praveen Gujar: Yeah. I think that's a great question. Let's take an example of, experimentation,
as well. So if you are a large organization,
you like, you're always experimenting on new capabilities and, features as well, whether it's in the user facing app, whether it's a b to b app that you are building for your customers as well. You are constantly experimenting on look and feel, on models that basically
optimize
either your notification
or your,
the fee that you basically see as well. These are constant,
experimentation as well.
No one
sees a a common experience. It's basically a custom experience that is determined by the experimentations as well. So that can,
so how and,
you can now think about how these experimentations
can be serving different,
platforms to sorry, different capabilities and different,
applications within your organizations as well. So this becomes a very classic use case to actually build a platform,
for to begin with as well. Now the question basically comes in, you have experimentation
capabilities. What are it should basically
encompass it as well. What is that platform definition as well? So the way we, have seen some of these, to play out is,
what are the fundamental things that you need to basically do experimentation as well? So one, you need the ability to understand and target the customers as well. Like,
you when you're on LinkedIn or when you're Amazon, you have billions of customers or billions of users that you, that make purchases or, our decisions on your platform on every single day as well. So, you need the ability to basically segment these customers, and you need the ability to basically run experiments
across these different segments as well. So you need targeting capability to begin with. Right. So then, basically,
is,
ability to basically move through this, targeting segments as well. This becomes a very key aspects,
for example. What do we mean by that? Right? So let's simplify. If you have 1,000,000 users, you want to start a feature sometimes only with 1% of the population exposed, to it as well, where your critical thing is to first evaluate what is the risk of launching this feature as well. We are trying to understand, is it breaking an existing experience?
Is it, causing a negative reaction to certain segment of the population as well? So you randomly select 1% of the population
and introduce this segment as well. But you want to you won't get typically statistical signal data at this point, but you're trying to really evaluate what are the risk of not in the feature as well. Then you basically go up to 50% to understand, like, is this feature really
delivering the value as well? Your goal is to first mitigate risk and then go to the 50% to evaluate
whether the feature is delivering on these capable the capability that we wanted to deliver and the value it is delivering as well. And then if everything looks good and there is statistical signal that this feature has the value, then you go to basically
a 100% ramp and everyone seeing those features as well. And now this is a complex process which needs to be,
managed by the experimentation platform as well. So ramping process directly. And along with that is the a b test capabilities
that are necessary as well. So the at every percentage ramp, we're trying to evaluate different things. So at some point, you are trying to evaluate the risk. At second point, you're trying to evaluate the value addition measured by statistical signals as well. So
that's the key thing. So you now have 3 requirements already within your experimentation platform, the ability to target, the ability to ramp, and the ability to experiment,
our AB test as well. Now not every functions or every capabilities can be experimented as well.
Different teams will have,
our AB tested as well. Different teams will have different needs. So how can they use this basic platform to build additional capabilities?
So our data teams team that is working with,
say, a b to b product can then come and say, I will use the same basic capabilities, but now I do I'm doing
because my products don't get, I cannot AB test on my products, I will use observational,
inference methodology
That is our longer term test, for 6 months to 1 year, and I can basically build on this platform as well. So that's really how you make decisions. 1st, understand, like, what are the common functions that every single team actually need, build those common capabilities.
And on top of it, like, you enable certain APIs and capabilities so that the other teams can take that last mile approach and build cost, solutions for their custom needs as well. And that's really how we think about platforms and,
how the platform enables common set of capabilities that can then be used by other teams to build for their specific needs.
Tobias Macey: In your work of acting in this product management role, bringing requests to the engineering teams, trying to figure out where your goals fit in their overall priorities and capacity.
I'm wondering if you could talk to some of the challenges and tensions that exist in that relationship and some of the ways that you think about how to
engage with the engineering side of the requirements and work with them to build the capacity that's required and understand how to build a kind of synergistic relationship
that they're building something that they're interested in and that they
want to support rather than just building something because they've been told to and they have to?
Praveen Gujar: Yeah. I think that's a good question. So first and foremost,
gone are the days where anyone in the team basically builds whatever they are told to, unless, like, it's really clear top to mind that as well. And this is really one of the biggest thing that I had talked to upcoming product managers as well, how they have to be inspirational to the teams, how they have set the common vision and ask, inspire the team to basically play towards that common vision as well. It's not an easy job for a product manager to begin with. There are a couple of things that, play a very critical role. Completing priorities will always be there. You have always
have very sim smaller resources around,
like, our challenged resources.
Right? So competing priorities were a very critical role as well. The clarity in thinking vision and, is very important as well. And especially when you basically work in a longer term platform roles, you need to ensure that milestones are critical as well. So you deliver on certain milestones
to ensure that there is continuous motivation for the team. There is a sense of accomplishment wins for the team as well. So I think we have to admit
that we all live in the world of instant gratification as well, and building platforms and products are not easy because it takes years sometimes to really see the meaningful results. So really breaking it down into milestones is, very, very key to motivate the team to continue to work on it as well. So all these things are basic necessity that the product manager has to do, like, inspire the team with a common vision,
have a detailed road map, have a a clear milestones, that are delivered on a regular basis to keep the team motivated. These are very important as well. Without this, like, you're not even basically getting started with any kind of a platform product itself. Now comes the challenging part of how to convince basically the team on specific capabilities and other things as well.
So to begin with, the distinction of, building common platform capabilities, like, the bare minimum to this thing is very key as well. This is where a lot of the tension really lies. If a product manager is trying to basically grab an x capability within a platform,
you have to make a very compelling case to the,
to the engineering team as well and why this is going to give the leverage that, the platform is originally intended to do do it as well. It can be,
you like, simple things, like how many teams can basically use these capabilities.
What is the net impact on business basically because of these capabilities
play a very critical role. So leverage and in business impact, you need to delete the most of it as well.
Then basically tying it to the milestones as well.
If these capabilities
take multiple years to basically build, how can you actually break it down into smaller capabilities,
evaluate
the impact basically delivering, and only then build on future milestones is very critical as well. You need to have that clarity with, within yourself first to convince your
engineering team that this is the right thing to do as well. And this is really where having a, a good strong relationship with the engineering team really plays a critical role. I often say to people that I actually mentor that,
there is no better relationship than an engineering and a product manager relationship. The stronger,
your bond is, the easier do it is for you to do your job and also easier for you to basically
have these kinds of an alignment in a more seamless manner as well. And, again, I'm not saying that you can get away with anything as well. You that build that that trust and that relationship with your engineering partner. The primary partner that you have is built by years of experience by doing the right things for our customers, for our product, for our team,
and everyone around me as well. Right? So this is really where I think, finally, you need to have that strong relationship to make sure that lot of the tough decisions are made. If nothing else work, there should be an escalation channel that should be put in place as well that you should be agreed upon priorly and if we are willing to not align on these things to be done. And it happens more much more often than we think about. But the escalation channel basically helps you to make faster decision so you're not really going back and forth for weeks or months, sometimes as well, so that you can actually make a decision. And last but not the least, you need to make sure that in this escalation, there is
very good clarity to the decision makers as well if this is a one way street or a two way street. Right? For example, what do we mean by that? Right? Certain decisions are one way decisions. That means it can significantly
change, and there is no undoing it,
as well.
For example, the experimentation that I talked about. If you are thinking about building a platform that supports targeting, ramping, and experimentation,
it's a significant decision. And if you, build those capabilities for the next 1 plus 2 years and you cannot go back from those decisions, you you can technically go back, but there's a waste of last 2, years. Our 2 way decision is something like,
that can be
undone with minimal damage as well. And for those things, you need to make faster decisions as well. And, really, in this escalation, making sure that that clarity is provided. That, again, really anchors on the product manager doing his or her job to ensure that
I actually you can actually deliver, this escalation message very clearly and get out of a decision as well. So very yeah. I think the order dynamics is very complicated, but really end of the day, the product manager has to step up and deliver on these things to make sure these conflicts are resolved more amicably.
Tobias Macey: In that case of building that relationship with the engineering team and engineering management,
you're also working on the other side with the business and sales side. And one of the oldest tropes in engineering is that salespeople will sell something that you haven't built yet. And so I'm wondering some of the ways that you have in your experience been an ally to the engineering team to help advocate for their needs with the business and sales side of the house
and some of the ways that you act as that kind of peacekeeper between the two sides of the organization?
Praveen Gujar: I think if I get a penny for every time I do this, I would probably be super rich by now. It's really the the right way to begin this.
I think it's a very good observation, and I think it's pretty much industry wide
challenge that we actually face as well.
The sales first,
persons have a very difficult challenge of demanding customers.
They are they do one of the toughest job. I I consider dealing with people. That's one of the toughest job and building technology products in this thing. And, again, an engineer will probably completely disagree with me, but, like, people come in all flavor,
and they have own challenges. But so sales persons basically have to make on the fly decisions sometimes to promise something that may or maybe difficult to get it done as well. And that often comes back as a a heavy business push,
as well. That's really where,
product manager basically,
comes into,
a critical,
plays a very critical role as well. 1, I think having a open communication channel within your sales counterpart and your engineering counterpart is key to the needs as well. But trying to understand each other is very important. So that's number 1.
Second one,
often, like, what comes as a build me a Taj Mahal kind of a request can be broken down into
smaller entities that can actually be delivered in a shorter time frame. And that basically can keep both the salespeople and the customers happy as well. That's just a very common, thing that you can actually more look as a product manager if you don't really dig into the details of what is being asked as well. You have to remember that sometimes
customers may, not know what they need. Sometimes salespeople misinterpret what as well. So really digging into the data, there is no substitute to talk to the customers directly as well. So that's the 3rd piece of the puzzle as well. Understand,
the customer problem really, really deeply,
and then basically whatever is as well. And then last bit of of present really is, like, having a backbone to say no is important to sales cycle or to the business as well.
Despite doing all these efforts, certain promises are something that you may not be able to deliver on.
Maybe you you're technologically not ready at this point, or this may not be in the best interest of your business, for example.
So product manager plays a very critical role in having a backbone and saying no and going through the necessary escalations to basically,
resolve this,
is very important as well. So now we are playing a 2 critical role. 1, you're really trying to understand
the salespeople's
intention behind it and the customer intention behind it as well. And, also, you are trying to basically translate that into,
what isn't feasible from an engineering side. And when things you see that you this is not in the best interest of either the team or the product or the business, then having a backbone to push back is equally important. So you're shielding the engineering team from an unnecessary churn
as well. So I think, yeah, this is where really product manager plays a role. And this is one of the reasons why a lot of the organizations are seeing that there's a need for a product management function
in,
data products and platforms as well because these are complex decisions that you have to make on a day to day basis sometimes, and also make sure that both the different go to market teams and the r and d teams
are in sync, and you freely form a bridge between them.
Tobias Macey: In your experience
of working in this space of technical product management
and working in the data space and with very data heavy products, what are some of the most interesting or unexpected or challenging lessons that you've learned in the process?
Praveen Gujar: Yeah. I think,
the leverage of platform
is unimaginable
sometimes, to be honest as well,
because you actually
irrespective of how much of a research you actually do,
to collect what are the common capabilities we need to build, when you really build a platform and
expose it to your customers, both internal or externals, the amount of creativity they can actually do to build on top of the platform
is amazing as well. So I'll give you a very specific example without diving too much into the details,
at this point. But think about,
compliance and regulations at this point. It's really a common
theme everywhere.
Bigger organizations are putting in privacy regulations.
Bigger government organizations are introducing compliance rules and regulations as well to name a few, GDPR,
CCPA,
and they're upcoming on AI models as well, c 2 p a and everything. They're very critical for organizations to meet those requirements
as well. So in, and often, like, sometimes, like, when these kind of variations come, that's when the platforms that you are actually built can be really, really
useful as well. So I talked about,
our, I worked on this platform called Data Hub, which is now an open source product as well.
The crux of basically what this is is a metadata management,
platform, which understand enables you to understand
effectively the data flow within your ecosystem as well by
building lineages between datasets and,
exposing APIs that can actually be used to understand these relationships over the
data as well.
So we built this platform, and that's when GDPR came into feature as well.
The first of all, understanding of our data ecosystem really helped us to meet the compliant needs of GDPR as well. But,
it's not a one time done activity. Right? Like, you have to constantly monitor if there are any compliance violation that actually happens as well. That's why one of the team basically built, an application called compliance monitoring solution on top of it. It the crux of it is very simple. Built on this,
data management,
platform,
Every time there used to be new regulations,
we used to introduce,
a script to basically
ensure that all the regulations needs are actually met by leveraging or building on this type of platform. So if there is any potential violation, for example,
an individual x was working in team y and had access to a dataset. And now that individual moved from team y to team z and technically,
having still continue to have access to the dataset can be a compliance violation as well. In that case, basically, how do you identify and revoke access or ask the individuals to give up access is a critical thing as well. And this is how the team basically used the data,
by the data metadata management platform to build solutions on top of it. Because now you're bringing in all the data together in your ecosystem, understanding it. So now you're able better able to build these kinds of,
applications.
So,
that's why I say, like, platforms surprise you quite a bit with what kind of applications can be built on top of it. And there are plenty of such examples in my,
own experience as well with the teams even surprised,
or even customers even surprised what you're basically, leveraging, what you have basically built.
Tobias Macey: Obviously, there are major benefits to be had with having a product management capacity
in this data ecosystem.
I'm wondering if there are any examples of cases where either the organization is too small or the focus is not sufficient
where it doesn't matter having a specific product management role for a given organization or a given team.
Praveen Gujar: Yeah. I think definitely. So if you think about
product management,
functions,
but even keep it aside, the product management function. Right? Why do you build in house data platform and products rather than basically,
using,
off the shelf solutions? It's because
you have unique needs. It can be the complexity
of the data ecosystem that you actually have. It can be the scale of the data ecosystem you actually have as well or the customization that you need or ever changing business needs that are important for you to basically have a better control about it. And last but not the least, the compliance as well. Some of these larger organizations,
like,
can be in
severe trouble if there is a compliance violation as well. For example, GTA PR famously
mandated 6% of overall global revenue as well. That's a huge chunk for larger organizations as well. So it's worth every penny
to,
build,
their only now solutions as well to ensure that,
you have better understanding about how you're being compliant and have better control over it as well.
So
there are way too many reasons why large organizations should basically build as well. But if you're a small start up, at this point, I if you're a lot smaller organizations as well, you don't have these complexity, these scales, these requirements as a whole as well. This is really where you have to make more judicious decision about
integrating with the fully managed services
and leveraging open source
platforms too for your needs is a critical thing as well, other than building in house a lot of these capabilities are distinct. And but as you grow,
really making very deliberate choices
about investing,
in some of these,
products and capabilities as well.
Earlier, I alluded every organization
looks back in 5 to 7 years timeline
to see,
do I have to significantly
overhaul some of these capabilities as well? And if you are a start up, you have gone through some of these decision makings, you know that at some point you run into a junction. Like, we are very fair very aware of, the blue whale story of Twitter as well, how they want from a monolithic to,
microservice architecture. Similarly,
every, such junction you need to evaluate if this is the time for you to basically build capabilities in house and search yourself for the next 5 to 10 years of growth. Are you basically
cease to exist in certain case if you don't do that as well. There are multiple instances where organizations basically
stitch together with different capabilities that they actually,
had but did not invest properly,
and couldn't really,
these systems and capabilities couldn't manage the scale.
Lot of this has changed with public cloud right now, with fully managed services as well. So there are organizations that are very effective in not having such large data product
and listing as well, but still manage the business. But when it comes to the cost of lots and flexibility,
expenses sometimes as well. So these are really the trade off that you have to do.
These kinds of organizations, maybe Netflix is probably case in point here, tend to be run very, very lean.
So they manage
their, you know, in that infrastructure for fully managed services and argument it with certain capabilities.
So it's really there is no one size for, fits all kind of a story here. But for certain scale, there is clearly no substitute
building some of these capabilities.
But that's not for everyone for sure.
Tobias Macey: As you continue to work on investing in product capabilities,
leveling up the engineering teams to be more engaged with the organization
and motivated to support the functionalities,
I'm wondering if there are any aspects of the data ecosystem
and the way that you think about the engineering work as being product focused
or the ways that you think about being a product manager in this ecosystem,
if there's any specific advice for people who are looking to get more involved there or things that you're you're thinking about trying to help promote and encouraging people to do?
Praveen Gujar: Yeah.
Again, I think it's a great question. A lot of what basically, people that I mentor who want to be in product management role understand as well. So I don't really understand and ask this question as well.
I think, to be honest, like,
product management is a more, like, really sought out role these especially these days.
Especially, like, I get a lot of requests from MBA students who used to, earlier before management consultant and our investment banking. They now really want to,
be a product manager in, big tech companies, specifically the Silicon Valley.
And, a lot of them don't really understand these nuances as well. So specifically for a product manager who is really want to break into the space, I think I would say certain things are very important.
Number 1 would be,
that, you have to have a typical,
background to some sort.
Can you be successful in this space without technical background? You can be, but that basically means you need to,
ramp up those skills. Because these are technically heavy product products. And without technical capabilities,
you won't first,
have your respect of the engineering team. 2nd, you won't build meaningful products. 3rd, you won't be successful on a product management. So technical skills basically is very important. That's number 1. Number 2 basically is
some of these, products
don't deliver the instant gratification as well. Hard to measure the impact,
sometimes as well. And these are challenges that you should basically be fully aware of that actually comes with a whole set of other challenges as well. So, people are motivated by incentives. So how do you keep your team motivated
when, you have a product that has been working on for 3 years, for example? Right? A platform that you are building for 3 plus years. So really having,
those milestone based approaches shown that in the milestones and molding the team is very critical as well. And for yourself as a as well as a product manager to make sure that you don't change after instant gratification, but you're really trying to build for the next 5 years, is very critical as well. Then I will say it's very to basically come to terms with, to begin with, because we are now,
often seeking instant gratification as well.
And last but not the least,
I think there should be a passion to solve some of these complex problem as well.
I would say this basically trumps everything, to be honest. Some of these space are increasingly difficult space
to lead as a product manager.
So you need to have a passion to solve these complex problems.
Get up every day to basically solve this problem for your customers, for your business, for your team.
And that's very important. So these are 3 skills that are really these 3 things I would say are very important for the upcoming product management and data platform and product space to be pretty aware of.
Tobias Macey: Are there any other aspects of product management in the data ecosystem,
your specific experience in that role, or the technical capabilities required to support these end user facing features that we didn't discuss yet that you'd like to cover before we close out the show? I think,
Praveen Gujar: if you have if you are a product manager without technical background, there are plenty of,
capable
courses that are no are already available as well. And even if you are basically a product manager with a lot of background, as I said, the technology world is very dynamic. You have to basically continue to refresh yourself. I, myself, am going through
much more detailed
AI courses as well to a level, basically, I can code and build my own models as well. Right? Like, really understanding,
going to the depth is very important,
as a product manager, we want to basically,
understand the challenges of your engineering team.
Number 2, basically, is to be in sync with the technology trends as well. So I would say it's very important for you to basically do that. So I think I will, my well, probably kind of an advice would be to
really feel really understand the technology goal or the depth of, understanding it, live a life of an engineer, for example, to be successful product manager,
in this, in this capacity, particularly.
Tobias Macey: Alright. Well, for anybody who wants to get in touch with you and follow along with the work that you're doing, I'll have you add your preferred contact information to the show notes. And as the final question, I'd like to get your perspective on what you see as being the biggest gap in the tooling or technology that's available for data management today. Yeah. I think, like, this shouldn't be a surprise to anyone, but specifically the advent of GAI the last few years,
Praveen Gujar: we are looking at,
astronomical base of innovation that is, powered by AI in the next few futures as well. Every organization is basically investing in some other capacity as well. That basically introduces a whole set of challenge related to data, related to tool,
to make sure that these AI foundational models, number 1, is available
to everyone. They can be basically used in the right manner as well. And any any rack based approach that you actually use to augment these models, you have the right data, you know, that is readily available, can train these models, and go live as well. You have the necessary infrastructure
to experiment
with these models as well. These are all the keys that basically
there are,
there are certain of these areas still have the capabilities, but they are not really upgraded to a level that,
a piece of innovation needs at this point. And I would say, like, a lot of the organizations are really thinking about this, how to basically
upgrade their infrastructure, their own tooling, their own capacity to really cater the deeds of,
AI and the innovation that is now following.
Tobias Macey: Well, thank you very much for taking the time today to join me and share your experience working in this role of product management for data systems. It's definitely a very interesting position.
It's one that is continuing to evolve and change, particularly as the expectations around capabilities for these different end user products are evolving
along with the technology landscape. So it's great to have your perspective and your advice for people
who are working in the space or working with somebody who's in that space.
So thank you again for taking the time, and I hope you enjoy the rest of your day.
Praveen Gujar: Yeah.
Yeah. And
and thanks for having,
me, in the session as well. This was,
it was great to basically share some of my experience as well. And,
yeah. Looking forward to other new things as well.
Tobias Macey: Thank you for listening, and don't forget to check out our other shows. Podcast.netcoversthepythonlanguage,
its community, and the innovative ways it is being used, and the AI Engineering Podcast is your guide to the fast moving world of building AI systems.
Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. And if you've learned something or tried out a project from the show, then tell us about it. Email host at data engineering podcast.com
with your story.
Just to help other people find the show, please leave a review on Apple Podcasts and tell your friends and coworkers.