Unknown: Hello, and welcome to the Data Engineering Podcast, the show about modern data management.
Atlan is the metadata hub for your data ecosystem.
Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan's active metadata capabilities.
Push information about data freshness and quality to your business intelligence,
automatically scale up and down your warehouse based on usage patterns, and let the bots answer those questions in Slack so that the humans could focus on delivering real value.
Go to data engineering podcast.com/atlan
today, that's a t l a n, to learn more about how Atlas Active Metadata platform is helping pioneering data teams like Postman, Plaid, WeWork, and Unilever achieve extraordinary things with metadata.
When you're ready to build your next pipeline or want to test out the projects you hear about on the show, you'll need somewhere to deploy it. So check out our friends at Linode.
With their new managed database service, you can launch a production ready MySQL,
Postgres, or MongoDB cluster in minutes with automated backups, 40 gigabit connections from your application hosts, and high throughput SSDs.
Go to dataengineeringpodcast.com/linode
today and get a $100 credit to launch a database, create a Kubernetes cluster, or take advantage of all of their other services. And don't forget to thank them for their continued support of this show.
Your host is Tobias Macy, and today I'm interviewing Gopal Arunjiparath
about his work at SUST Global, building datasets from geospatial and satellite information to power climate analytics. So, Gopal, can you start by introducing yourself?
Unknown: Tobias, great meeting you today, and thank you so much for having me here. Everyone, thank you for listening.
This is Gopal.
Quick introduction. I'm an electrical engineer turned geodal scientist.
My passion and expertise has been in geospatial data over the last several years.
I started my the early days of my career
working on multimedia
and large scale imagery datasets.
And along the day, got really interested in environmental data and got fascinated by remote sensing datasets
and made the transition from large scale imagery
to large scale geospatial datasets.
And through a serendipitous
sequence of events, ended up
leading analytics and insights at Planet Labs,
where I spent 3 years prior to find founding SAS Global.
While I was at Planet, in addition to exploring remote sensing imagery, I was also curious about what kind of datasets exist
on the changing climate.
And I essentially saw, like, the emergence
of frontier climate models,
and that got me interested in climate modeling.
And a couple of years back, I start I stepped it at full steam ahead
with founding SAS Global and exploring large scale climate
datasets and climate models. So SAS Global was founded with the mission,
to develop data driven products
that enable every decision to be climate informed
so that humanity can thrive on the
forward looking changing planet.
And I would say last several years of my career, I've been focused on product engineering
and research commercialization
on data intensive applications.
Unknown: And do you remember how you first got started working in the area of data?
Unknown: Yeah. So my first experience working with data was looking at imagery.
When I was at Dolby in 2, 007, 2008. Yeah. I I'm not skipping that decade. It was a long time ago. Dolby decided to make this transition from being
the world's
foremost and pioneering
audio technology company to being the world's pioneering
multimedia
technology company
and decided to invest into
an imaging
initiative,
which led to a focused effort around building the world's first
flat panel display for cinematic
imagery calibration
in studios and postproduction houses. So that's when I started dealing with large collections of imagery data
for cinematic
reproduction.
And
as you can imagine,
a movie file has, like, many images. And if you're looking for products that scale across
the gamut
of cinema production, then we're dealing with a lot of imagery for calibration
and for tuning. So that's when I first got started with data data heavy capabilities.
Unknown: And you've mentioned a bit about what SUS Global is and some of the story behind how you got involved in it. And I'm wondering if you can talk a bit about the type of audiences that you're focused on as far as the products that you're building and the data that you're working with?
Unknown: Yeah. Yeah. We're primarily serving
2 camps of very motivated
climate
aware personas at the moment. So 1 of them is around ESG assessment and reporting. As you're all aware, there's been an increasing
interest as well
as increasing awareness,
environmental, social, and governance indicators.
And oftentimes, when you're doing those assessments,
you can need the best validated cleanest climate data that you have access to. So reporting on the risk to a company
from the environment,
1 of the avenues which we help our customers assess.
Secondly, we've been
enabling the creation of new
sustainable finance instruments in the market intelligence space.
So that's the 2nd camp of analysts
and data scientists who we're enabling with clean validated climate data.
And then I would say there's this emergent 3rd camp, which is
rich sets of climate aware
dashboards
and
insights platforms
that are trying to surface climate data in interesting ways. So that's a little bit more nascent of a camp, but quickly emerging
and 1 that we are equally excited.
Unknown: Climate is definitely part of the zeitgeist right now. There are a lot of conversations happening around it, but
the
sort of concrete aspects of it are being able to understand
what it is, what are the changes that are actually happening, what are the impacts that they're having on, you know, business,
on communities,
on
geographical elements. I'm wondering if you can talk to the types of data sources that you're working with and some of the kind of scale that you're dealing with, some of the, maybe, heterogeneity that you have to deal with, and some of the ways that you're processing that data to be able to build some of these derived data products that are more consumable for your customers?
Unknown: That's like a nontrivial exercise. So when we started out looking at
physical climate risk data, and this is a distinction worth making clear. You know, when you think about data in the context of climate,
it's like an ocean like that that hasn't been another.
You're looking at social data, which is human footprint and how that affects climate. There is, economic data, which is, you know, how is climate created,
the changing environment and the changing climate created losses, and how that even loosely connects into
the environmental footprint that is changing,
that is enabling
new schemes for carbon capture of a reduced carbon emission. So that connects loosely into the carbon
offsetting space. And then you have physical climate risk, and that's the space we operate in. There, it's an understanding of what the environment is doing to humans, to property, to the land ecosystem, to the oceanic ecosystem.
So
over there, we are looking at assessments across
physical hazards, which are both acute and chronic. So think of acute hazards as
wildfires,
riverine flooding, coastal flooding,
flash flooding,
and
cyclones,
and then
chronic hazards being heat stress, water stress,
sea level rise like impacts.
So in order to do each 1 of these hazards correctly, you have, like, you know, subsystem
of different
or sub collection of different,
datasets that we need to bring together.
And what is the common ground there? So we can start there, which is inherently climate is,
geospatially
non stationary
process. So different parts
of the land ecosystem are affected very differently by very different hazards.
So
the datasets are inherently geospatial
and have spatial context.
The second piece is they have a temporal element because there is a different incidence of these events over the course of time
and also projected differently over the course of time.
And together,
there's no 1 model of, like, the entire earth system. There are so many different subsystems
to be taken into account. So
bringing those together requires
geospatial
data transformations,
and that's what we've been focused on building. So we tap into
earth observation datasets
primarily because they are good ground truth for what is happening on the ground in the context of these physical hazards.
And then we tap into frontier climate models that project
into the future on what could be
potential impacts and exposure
across different climate scenarios,
which is kind of loosely nominated by the IPCC based on their most recent reports.
So if you look at projected capability
and couple that with observational datasets,
you can then look at hindcast that help calibrate
models against each other. So that's kind of the basic framework we are starting with. And built on top of that, we have
accessed. We have built a more sophisticated geospatial
data processing stack that brings into account land layers,
brings into account machine learning models
so that you can enable asset level assessments of physical climate risk. So going back, going from refining climate data to physical climate data,
to existing datasets,
to what is meaningful
is a journey.
And I would say the primary gap that we're trying to fill is this. When you look at frontier climate models,
especially the ones that are global and we think climate is a global problem, we wanna start with assessments at a global scale. Global climate models inherently are
meant for
understanding
fundamental variables and impacts at a global scale, but they're very coarse resolution.
And when it comes to businesses that we are serving, they don't care about whether a country's at risk or like a state is at risk, but they do care, But they care a lot more about
what's the impact of my holdings
to my assets on the ground,
my
physical footprint,
or my physical supply chain. So
bringing it down to asset level
and regional level
on demand
is the capability that we need to stand up, and that's where you need to use
these data transformations
and machine learning techniques.
Unknown: Yeah. I can definitely imagine a whole host of challenges and potential approaches that you could take to being able to do things. There are whole businesses that are focused on some of these subcategories of problems of, you know, saying, okay.
Based on public records data, I'm able to determine that this, you
know, bounding box at these latitude, longitude key pairs are a physical kind of business that a company owns and being able to identify that company and the entity resolution aspects. Like, talk to people where that's their whole business. It's just that piece of it. I've talked to people whose whole business is just being able to say, I can handle this mapping layer and be able to
overlay some of this kind of, you know, social and civic data on top of these, you know, raw map tiles to be able to give you some ability to analyze some of this information. That's a whole business right there. And you're saying, okay. We'll take all of that and add another layer on top where we also wanna be able to understand
what are the actual weather events, what are the long term trends, how does that map from, you know, discrete weather events into the overarching climate challenges.
So definitely interested in understanding kind of how much of the
technology stack and the data manipulations that you're doing are things that you're able to pull off the shelves because other people have kind of treaded some of that ground for you, and how much of it is just we have to build all of this ourselves and invent new technologies
because
what you're trying to do is so kind of vertically integrated and complex and inherently needs to be able to address all of those concerns at the same time? That is a trade off, and I feel in in this particular context,
Unknown: we are building on the top of years decades of research and work.
So, you know, climate models that exist today,
kind of end generation, like I'd like to call it. IPCC calls them 6
or the couple model India comparison project version 6 0, but they're built on top of generations of climate models. Some of them written in, like, many years ago,
and there's a lot of science and gifted thinking that goes into them. So rather than reinvent that, we've been focusing the problem more on what's most how do you make that data meaningful to a business
on the ground? Same thing with remote sensing. You know? Like, if you look at Earth observation datasets,
this has been an explosion
in the context of, like, the amount of data generated from satellites in space.
So finding the right datasets, the right sensors
that can be used most effectively
for datasets is kind of where we're in a very opportunistic time to be able to make those choices. So that's kind of where we are focused our time on. In terms of the
Unknown: ways that organizations are using these data products that you're building, you mentioned that they wanna understand, okay, what is the overall impact of climate on my assets, my physical holdings?
I'm just wondering if you can talk to the types of questions that they're asking and some of the ways that you think about structuring
your
derived data products in a way that allows them to be able to explore those answers?
Unknown: My role at SaaS Global is a deliberately crafted unique 1. I head up both the tech and the product functions. And primarily, it comes down to, okay, let's build the right tech and invest in the right technologies necessary to answer very,
very specific and very meaningful
sets of questions.
So I would broadly
set
questions
and
query set, if you might,
for climate data or physical climate data in today with 4 caps.
So the first set of questions is around localization and reporting. So where is their clear concentration
of climate exposure,
and
what is the expected severity? What is the estimated impact to my set of assets? So that's take 1 set of questions you can answer, which is given a large collection of globally dispersed
assets,
where's concentration, where is exposure,
and what's the estimated impact? So that's 1 set of questions we'd have answered.
The second 1 is I would call relation,
which is
how does has an exposure
from the physical climate
lead to impacts in either the distribution,
the sourcing,
the supply chain,
or even the allocation of capital.
So that's the relation side of questions.
The 3rd bit, I would say, or the 3rd camp would be
what I like to think of as optimization.
So if you were to deploy capital
across a set of instruments or a set of assets in a climate and air fashion,
how would you do it? If you were to deploy capital towards building new infrastructure,
which is gonna happen over the next few years
and make that new infrastructure
informed by the changing climate,
how would you do that, and how would you make those climate investments?
That's the optimization set of questions.
And then lastly,
I like to think of the the camp of adaptation, which is
how could
teams of users and our customers
introduce adaptation measures
in a capital efficient way
to protect structures,
species,
land, and natural capital for the decades and generations to come.
So at SaaS, at the early days of helping
our customers build
the understanding and awareness,
and that's largely the reporting. That's where the market's at today. But longer term, we are building the software
of climate adaptation.
We're building the pathway from answering the basic questions, which is localization and reporting,
to relation and optimization
that help you connect to larger scale problem sets and larger scale business impacts,
and eventually to adaptation,
which is kind of where the world needs to be in the years to come if you are to be serious about adapting to the changing climate, and that's the software we build.
Unknown: Everybody. 1 of the interesting aspects of this too is the question of education where because
climate modeling is a very sophisticated
undertaking, it does require a lot of contextual and domain knowledge to be able to
interpret all of the ramifications
of the information that you're generating.
I'm wondering how you think about the customer education aspect of the product that you're building and being able to say, okay. This is the information that we're producing.
These are some of the ways that you can and should be thinking about aggregating it because as with anything that you're doing with analytics, there are certain transformations that are legitimate and actually do provide additional kind of meaning and
nuance to a problem. And then there are other ways that you can run an aggregation that, well, it will produce a result, doesn't actually make any sense.
And I'm curious how you've approached that challenge when working with your customers to be able to say, okay. You know, these are the datasets. These are kind of the contextual semantics around it and being able to bring that into their own
analytical stacks and their own data products to be able to say, okay. You've
combined our climate information with your information about the
amount of capital that you have invested in a building or the amount of inventory that you're storing there. You know, these are the ways that you can combine them to be able to get a useful answer and useful insights out of it. But if you do it this other way, it's actually just gonna give you a number that has no meaning to anybody.
Unknown: Yeah. Yeah. And that's a darn trivial problem, especially right now with so many misconceptions
around
the changing climate. Many of the times people misunderstand
weather for climate. Many of the times people misunderstand
hazards. Many of the time people misunderstand
causality.
So I would say the way we have looked at it, and we're still in the early days of doing this, and we haven't cracked it 100 percent yet. We've tried to work on what I think of as a common
domain language,
which is, okay, what are the basic set of primitives on which we can start conversing on this topic, this fairly sophisticated topic, without misunderstanding
each other? And I would say that starts with data literacy.
And
in this context, it's climate data literacy.
And
I've kind of
I've seen
what Valerie, Logan, and the data lodge
have tried to do in terms of going to digital transformation
and going through data transformations and enabling companies to do that as, like, a pioneer in that space. And I try to borrow some of those methodologies
into what I like to think of as climate data literacy
and climate data as a second language. So
that I take on what are the primary sets of things people need to have a common understanding of
and distill that into
workshops,
hackathons,
and deep dives we organize with our customers
because this is not an established domain. Many of the customer
like, outside of catastrophe modeling,
reinsurance
and insurance markets, like, people don't really know what to do with this data, but they have the right motivations.
And I don't wanna trivialize that. I feel like people have I shouldn't say people don't know how to. I think it's a function of
you need an increasing awareness of the capabilities
to be able
to shape
these solutions. Right? And towards that end, you know, we create venues where data scientists, data engineers,
and from the customer side
brought in solution engineers
and data scientists from our side, climate data scientists at our side to create that shared knowledge base. And once you've done it a few times, you see the replicable pattern,
and you can then create the knowledge base that helps make it easier to do it in the future. And to a great extent, almost any technology, like, even data engineering, be it, you know, some cloud native capability or a new open source
open source package
has kind of evolved that way, man. 1 use case has been clear, it's been documented, and then you suddenly have a community of people
relating to that same problem set and then going from there.
Unknown: RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control.
Their SDKs make event streaming from any app or website easy, and their state of the art reverse ETL pipelines enable you to send enriched data to any cloud tool. Sign up for free or just get the free t shirt for being a listener of the data engineering podcast at dataengineeringpodcast.com/rudder.
Digging into the technology stack of how you're designing and building your platform, I'm wondering if you can talk to some of the architectural elements there and some of the unique challenges that you've had to address, some of the, you know, engineering that have been
maybe most valuable, but also most kind of challenging or frustrating?
Unknown: Yeah. Yeah. I see that
the exciting as well as challenging elements a lot of, like, building this climate data warehouse
where you have the most relevant datasets, and I would say all the cloud providers are kind of going that direction. They are having either the planetary computer system from Azure slash Microsoft
to, like, what's in the Google Earth Engine datasets today,
but they're not quiet there in terms of all the capabilities and
the stack you need. So we've been focused
on building on top of those capabilities towards a climate data warehouse, which is very specific
to what we need,
and keeping that fresh is a nontrivial challenge.
Keeping that well maintained and organized is a nontrivial challenge.
And then
enabling a feedback structure where
on ground intelligence
and intelligence from our customers can help us
learn to keep that warehouse focused,
updates that warehouse effectively
is another nontrivial challenge.
We've also focused a fair bit on building a cloud native workflow. So
we have built
a distributed
data processing system in Golan,
and we have spent a fair bit of time on building out data pipelines that can that can process global scale or planetary scale data
for large collections. If you're studying the supply chain of a commodity, that's like
1, 000 or 100 of 1, 000 of assets. So being able to process that is something we've been doing.
And I would say
we kind of tuned our system a little bit more for batch mode operations.
So I would say now the challenge we're looking at is, okay, how do we also enable
rapid translation
and being able to
focus more on caching and querying
because the use cases are expanding, and customer sets we are serving are expanding. So there's some new patterns that we need to support.
And I would say in terms of the technology,
the most exciting thing that I've discovered through this journey of building out the platform has been
building the operational efficiencies where you can have
these live streams, be it to satellite or to on ground sensing,
get plugged into the warehouse very efficiently
and retrain models on demand
or in on demand based on certain triggers you said. So we've architected a system, I would say, the beginnings of a system that does that effectively,
and we intend to, like, build that out over
the the years to come.
Unknown: To that question of sensing,
1 of the other
interesting problems in this space is the fact that there has been so much rapid innovation and evolution of the sensing capabilities where
it went from you had to rely on
government owned satellite imagery that had maybe a daily update cycle if you were lucky
to having
commercially owned and rapidly
expanding satellite imagery capabilities with, you know, different, maybe, wavelengths of light that they're dealing with or, you know, dealing with not just light based imagery, but radio sensing or radar, things like that
that maybe have hourly update cycles in some cases, at least over certain geographies.
That's another aspect is the fact that there is maybe heterogeneity in terms of the update cycles for those different geographies, or maybe North America has a rapid update cycle.
You know, Antarctica is obviously gonna be very undercover
because it's such a hard orbital pattern to maintain.
And I'm just curious how much of that is something that you've had to engineer around or have to worry about, or are there ways that you're able to kind of normalize the problem so that it is kind of pushed to the boundaries and you don't have to worry about that all the way through to the kind of center of your systems?
Unknown: Yeah. So you raise a very interesting
point, which is when you have
very sophisticated
imaging capability, you have to deal with all these boundary issues.
And the big point to take away here is
raw satellite data is often not very useful in this context.
So when the context of satellite imagery, we often talk about these processing levels. So natively, what a sensor captures
is kind of what people like to call L1,
and then there's a process to auto rectify, which is down lies to the ground
version of it, which they call l 1 b. And then there is a version which is Mosaic, which is kind of what you see on Google Earth, which is your l 2.
And then l 3 is a stacked version of it, which has
the time dimension. So Google Earth changing every day, so 1 can think of parallel imagery closer to that.
When it gets to that l 3 scale,
the data becomes useful
for the kind of
spatial temporal analysis that we are talking about, which is primarily what you need for geospatial analytics related to climate.
We've specifically
gone with datasets where
where that's either already available or very easy to derive.
So that's kind of the qualifying criteria, which kind of immediately
shrinks your
your dataset space to a very select few. And then you add on top of that, you want global coverage, and then
it it becomes even simpler for you to make the choice.
So
we've tried to push the problem
more upstream
and focus more on the application there because that's the thing we uniquely know how to do.
Unknown: In terms of the
ways that the data is presented to your end users, I'm curious what types of data assets you're dealing with, the shape that the data is taking. Are you delivering
image data with overlays? Are you delivering
tabular or semi structured data? I'm just curious kind of what are the assets that your end users are actually interacting with to be able to then form their own analysis?
Unknown: Yeah. So we currently serve 2 products. We have a data API product that provides a base structure. It is not imagery driven. It's largely
asset driven. So it's tabular.
By asset, I mean, you know, physical
location driven, which is the request which conforms to the request that the customers make to our API.
And then the second product capability is that of a dashboard,
which you can think of as a thin client built on top of
the API or the data API, which enables you to visualize
the climate related insights
and serves the analyst or, like, the non developer audience.
So the data API basically allows
the collection of insights and the serving of insights,
which
are pretty structured.
So we looked at tiled datasets and essentially connecting this to the spatial
of web feature services
or web mapping or web tile services, but
those are things that we have on the road map for the future.
Primarily, the data API is serving
the analyst and the data scientist
personas very effectively at the moment because they have the geospatial metadata, so it's very easy to put it on the app and visualize
as needed.
So the way we're trying to focus on that is on the API connected to
we've stood up a Python client
and,
collection of notebooks and make it very easy for people to visualize it in their own workflows, and that provides a quick start
for
any
Pythonic
development workflow.
And for others, we can easily solution engineer and stand those up too.
Unknown: Another interesting area to explore is
in that question of performing analyses,
there's always an element of
forming a kind of shared understanding, building a story around the
insight that you're trying to provide to the people who are interacting with that output.
I'm curious what you have found, both you and your customers,
is useful kind of patterns or lessons to be able to
bring some of that shared understanding
to the end consumers of that analysis
because of the fact that it is relating to climate data, which does have this very a a lot of people have this complicated relationship with that concept where,
you know, it's something that is both inherently local as well as global. And, you know, everybody has a part to play in it, but everybody has their own kind of concepts about how they think about that broader
global impact and global network of involvement?
Unknown: I would say for our customers,
the problem set is very clear based on the camps or the questions that I mentioned earlier. So
I think the confusion is a little less on that aggregation. So if you provide data at the
asset or property level,
then it is very easy to group, sort, or aggregate the risk
to the macro scale. Right? So
that's, like,
issue of a security, then
they can look at their footprint and come up with a consolidation.
So we kind of work with them and enable them to come up with that logic. We have some patents which we can obviously get them started with. So I think the confusion there is less. I think it comes down to
based on your
on ground footprint or the tangible asset footprint, your profile can look very different.
There's also this element of
background or benchmarks. Right? So
you might have
an asset or a site
or a property
in an area which has very high exposure in the background,
and you might not be much worse off or much different from the background.
But that might still be highly elevated risk profile or exposure profile compared to another area or another site.
So that context is often very important, especially in use cases around site selection
where you're confined to a region,
but you're looking for the lowest level
of physical risk exposure
in that region. And that's kind of inverting the problem towards saying, okay, I know it's this is the region, but here's where if I were to pick a parcel of land for a specific opportunity, where would I do that? So that's different from, I know this is the parcel of land I care
about. What's the risk then? So you're kind of looking at both sides of that. So I would say that's macro versus the micro.
Unknown: Another
layer that
people are, I'm sure, curious about is this question of kind of time scales where
you're looking mostly, I'm sure, at what is my risk right now? What is my risk projected into the near term future or as far out as I could reasonably predict?
But there's also value in understanding,
okay. I have this information about now, maybe near into the future, but what if I look backwards and I'm looking at annual time scales of what is the impact of climate on these locations
and being able to then build up that view of how have changes in the climate progressed from
time into the past to where we are now so that I can then get a more nuanced understanding of what these future projections actually mean for me because I can build up this trend of analysis to be able to say from the past through to now into the future.
I'm wondering what you see
as ways that that factors into some of these questions and the decision making process.
Unknown: Yeah. I feel like that definitely helped shape a deeper understanding
because what has happened is something people can relate to. What could happen in the future is something that is still unknown, and no model can say that with absolute certainty.
So
being able to surface the data
on the historic dimension
and
surfaces at a cadence that is
can be correlated with other outputs and benchmarks
is definitely very valuable towards developing a deeper understanding. So
being able to enable
insights and indicators that are relatable in history
help
make those even more relatable for forward looking projections. And the forward looking projections is just hard primarily because
when you look
at
a climate with a 30 year horizon,
so many things that could change in that time frame. So that's the reason why even with the
6th assessment report
and the IPCC,
they documented a set
of bundled assumptions
as what they call socioeconomic
pathways
or shared socioeconomic pathways, the SSPs.
And that's basically saying, okay, if there are if you make these assumptions, this is how you can enable a projection.
So that ties into all the different warning scenarios
indirectly. So that's kind of what you see in the current literature.
Unknown: Bigeye is an industry leading data observability
that gives data engineering and data science teams the tools they need to ensure their data is always fresh, accurate and reliable.
Companies like Instacart,
Clubhouse, and Udacity use BigEyes automated data quality monitoring,
ML powered anomaly detection, and granular root cause analysis to proactively detect and resolve issues before they impact the business.
Go to data engineering podcast dot com/bigeye
today to learn more and keep an eye on your data.
And when people are building these
kind of aggregate, you know, layered historical views projecting into the future,
are there certain
kind of key indicators of the climate that they're typically looking at? Like, are they looking at what is the average kind of seasonal temperature and its variance over these years? Are they looking at, you know, rainfall?
Like, what are the types of kind of climate information that they're looking at, and how do you help to
prevent the pitfall of looking at specific weather events in that process of building the analysis?
Unknown: Yeah. Yeah. Great question. So I feel like it's oftentimes the questions start with, like, temperature and precipitation, right? These are more fundamental variables, but now we are seeing increasingly awareness around
hazards, like the acute hazards, because it's immediately loss making, like, because of certain increased cycle risk in certain areas. People in in the Gulf of Mexico, in Florida, getting used to the fact that during hurricane season, Zafavi Ben still have to live somewhere else. So I almost feel that awareness is there. We've we've seen that expand to other other
hazard dimensions,
for example, heat stress and water stress. As you probably know, there's been this
increasing water stress and drought across many parts of the US, and that's expected to just get worse as as there's more warming.
So
what does that impact look like? So
1 of them is the absence of water, but then what about the businesses that are heavily dependent on water, like a mine or a bottling factory? And
what does that look like
across these scenarios,
across these longer time horizons?
And then if you were investing into those businesses,
how would you think about the operational burden from drought and water stress in the future? So those are the kind of problems we're looking more closely into
and enabling teams
to answer.
And
in terms of be like, beta functions,
that's a lot more near term,
and there are different set of solutions related to that, which we introduce our customers to. We're not trying to solve that problem. We're not trying to
do better seasonal
prediction
because that requires a different set of expertise.
Unknown: Another
interesting way to think about interacting with this climate information,
particularly since you're looking at near future projections, is
the question of being able to
maybe play with some of the outcomes where you say, based on
this action or, you know, the assumption that I'm going to act in this way and that in aggregate, you know, people who are in my industry are going to have similar behaviors,
what
impact might that have on future climate projections, you know, particularly if you're looking at
industrial scale operations where, you know, you have 1, 000, 000, 000 of dollars worth of economic activity happening.
You know, if those players are saying, okay. Based on the data that I have and my assumptions about future climate impact,
I am going to actually make this investment in reducing my emissions by this percentage,
assuming that everybody else in the industry is going to do the same. Like, how much of a kind of meaningful
insight will that provide, and and sort of what are the levels of scale that are necessary to be able to actually understand what type of meaningful impact might that have given these set of assumptions?
Unknown: Yeah. I feel like
to some extent, the scenario you just laid out
is the broad narrative for offsets.
And what I'm looking at is and many of the times, like, there is the understanding that when you have
heavy emissions,
you're indirectly doing harm to the environment and triggering
accelerated warming. So
if
1 thing you could do is, like, stop that, the second thing you could do is kind of offset that.
And that's where I feel like the credit schemes
indirectly
or the credit schemes enable
businesses to be able to contribute back
towards the system and the holistic system.
But the big challenge in this ecosystem
of
assessments
is the time frame.
So if you're emitting
if you're having heavy emissions today, so say
you emitted n tons of CO 2 into the environment,
like, nothing changes immediately. Right? There might be some
or n tons of CO2 equivalent. It might not be exactly carbon dioxide, or you were involved in some wasteful process. The
impacts of them,
from those are not apparent
right away. It's apparent years from now
globally might not even impact
your specific operation.
So
I feel
for having a system of
adaptation
there,
1, you need the
climate
mitigation measures, which are kind of common offsetting reforestation,
sequestration.
And on the other side, you need the adaptation measures
where
you would think that the incidence of these events is gonna increase over time. So balancing those out is kind of the complex
bit, and
the unknown is
when you hit certain tipping points, the impacts can be a lot worse
than right now. And the unknowns also, what level of emissions trigger those tipping points.
Unknown: In terms
of being able to get started using the products that you're building at SUST, I'm wondering if you can talk to some of the
assumptions that you have about the kind of analytical or data platform capabilities that your customers are working with or the types of technologies or interfaces that you're able to provide and just some of the overall steps involved in being able to get onboarded as a customer and be able to start taking advantage of the kind of derived data products that you're building to be able to incorporate into some of these business questions and being able to help feed that into the decision making processes?
Unknown: Yes. I would say some of the assumptions, there's always, like especially if you're looking at things from
an API first standpoint.
The assumptions are often an understanding a shared understanding of
the data interface.
Like, how does data enter your system? How is data that you are serving back
the customer
used in their environments?
So we focused a fair bit on that, and we've worked through some early assumptions
and some false assumptions that we've since corrected
into the capability we currently sell. I would also say
the way we have been able to work through these assumptions
is just like most commercial
APIs providing, like, effective documentation,
a solid API reference
has been very useful that someone can actually pick up, read, and implement. Secondly, providing some good examples on how the data can be used
downstream
and integrated in workflows is another very valuable thing. So we've kind
of gone in with certain assumptions, and then in cases where those assumptions are not 100% valid, we're trying to showcase
how we can solution engineer around that
rather than serve a bunch of features that we gotta maintain over time. I'm sure that's gonna expand over time. Like, we're gonna have to build more, but we're trying to be very deliberate around
the data capabilities we are serving
because we wanna serve it for the long haul.
And inherently, the onboarding customers today, that's what they'll have to that's what they're getting used to in a production setting this year. And once these go into production, ideally,
you don't wanna change unless the use case is involved.
Unknown: In your work of building this platform
and working with your customers, what are some of the most interesting or innovative or unexpected ways that you've seen your products used?
Unknown: It was a pleasant surprise when we had, like, a few teams looking
at municipality
level insights
when it comes to city planning. And, you know, originally, we didn't think of that as, like, a use case, but that was an emergent capability.
The second bit is around
when you think about
you just touched briefly on carbon offsets, but
1 of the capabilities we spend a fair bit of time
time on in terms of fleshing out is a wildfire projection capability.
So
as you know, 1 of the examples
of carbon offsetting is planting trees and planting forests.
So if there were an afforestation
or a reforestation
project
that is getting commissioned and funded,
does the assumption that
when that forest goes live many years from now, it's gonna suck carbon from the atmosphere,
And based on the capacity of that, you're already providing offsets onto that project.
Now that whole assumption,
that
whole workflow, as you said, the forest is untouched
and thriving,
but if you have forest fires,
that's no longer true. So
this is where
the interaction
between the urban violent interface and wildfires
directly impacts the efficacy of a carbon credit.
And that's a use case that came from our customers,
which is very innovative that we really wanna support because it it really
pushes
the carbon credit ecosystem to the harsh reality
of what things look like in the next 20, 30 years.
Unknown: Yeah. It's definitely
interesting and great that the information that you're providing is able to feedback into not just corporate decision making, but also a better understanding
of
the programs that have been put in place to account for climate change and the ways that people think about being able to
mitigate risk or offset their own activities and
kind of digging into the assumptions and, you know, making sure that they are actually accurate.
Unknown: Absolutely. And, you know, testing those assumptions
and holding accountable,
like, the whole process for credits. And I think that's largely what that ecosystem needs because
outside of a few
standardization
bodies, there's kind of like the wild wild west.
Unknown: And in your own work of building this product, building this business, working in this ecosystem, what are some of the most interesting or unexpected or challenging lessons that you've learned in the process?
Unknown: So I would say 1 unexpected lesson is, you know, when you're
starting off with, like, a very small team
or, you know, just the team of founders,
the first set of customers you onboard
really
shape the trajectory
of your 1st few months or even sometimes 1st few years.
So
I would say most people talk about picking founders or investors when it comes to entrepreneurship,
but I like to invert that and say what you care about the most is picking
the customers you go after
because
that shapes the narrative of whether you wanna be self funded or venture funded
or related.
The second bit is I feel founding SAS Global. I've been pleasantly surprised by the kind of learnings,
like, from cross model meeting, across
remote sensing, data engineering, machine learning, and climate modeling, all of which are related. Data is, like, the the common thread among all these domains,
but
they're all so rich in terms of context of the data
that it's been a constant learning journey for myself and for my team,
And building out
a interdisciplinary
set of individuals to come together on this shared mission
has been a very exciting journey for me, and I'm also pleasantly surprised on what
a small team of engineers can accomplish.
And the other learnings being, you know, we're looking at things
with a fresh look and not be biased by what has existed before because we're operating in a different environment.
Unknown: For people who are interested in accounting for climate information in the work that they're doing and in the decisions that they're making, what are the cases where SUST Global is the wrong choice? I would say, like, you touched on a little earlier,
Unknown: better if you care about very near term
days weeks rather
than months
years, then we're not the right
solution.
And if you care about, you know, ESG is very broad. So if you care about social and governance angles,
Unknown: you're probably not the right solution. We're very focused on physical environment and physical risk at the moment. As you continue to build and iterate on the products that you're building and exploring more of the space and understanding
what are the
questions that are being asked and ways that you can address them. What are some of the things you have planned for the near to medium term?
Unknown: You know, our
near term focus is to get more developer friendly.
We're seeing a remarkable
set of opportunities in terms of serving,
not just analysts
and data scientist personas, but also developers who
want to integrate our capabilities into
new dashboards,
new products that they are building.
So more work than new sustainable applications powered by SAS.
SAS Global's capabilities is, 1 of the things we are planning for in the future.
And then, you know, going from
hazard exposure to advanced adaptation measures, we're building the software of climate adaptation,
and we're just in the early innings of that. So a lot more to follow on for that evolution.
Unknown: Are there any other aspects of the work that you're doing at SUST Global and this overall space of climate analytics and climate data that we didn't discuss yet that you'd like to cover before we close out the show?
Unknown: I would say, you know,
climate impacts
everybody
and all aspects of business. So while we are being very focused on
the commercial opportunities, we also live our mission
and in enabling humanity
to thrive
on the changing planet
with
the evolving climate, we prioritize
decisions that
minimize even our own carbon footprint. And even when we pick external collaborations, we try to see, is there opportunities for us to collaborate with, like, nonprofits,
enable them to have access to our data? Does it doesn't cost us much when you are still operating the same data stack
and thereby enable a net positive impact.
So we've been very focused on that, and I feel like that's baked into our culture as a team. So as we onboard new individuals onto our team, as employees, as partners, as advisors,
we look for that, and that's helped shape our DNA.
Unknown: And as a meta note too, if you wanted to call out your kind of hiring efforts, this is a good time for you to do that. Yeah. Absolutely. So we are hiring for
Unknown: a senior back end engineer as well as, like, a lead platform engineer to help us bolster more of our platform.
Folks listening have enjoyed hearing, I would love to hear from you. And if you're interested in learning more about these roles, visit the careers page at susglobal.com.
Also,
for any others who are working on large scale imagery or environmental
or climate data
in the context of data warehousing, as well as data processing
work. If the problem sets that I've described are interesting to you, whether you're looking for new opportunities or not, I would love to stay in touch and talk more. So feel free to reach out. LinkedIn is the best place to reach me and find me, so direct message me there.
Unknown: Alright. Well, for anybody who does want to get in touch with you and follow along with the work that you're doing, I'll have you add your preferred contact information to the show notes. And as the final question, I'd like to get your perspective on what you see as being the biggest gap in the tooling or technology that's available for data management today.
Unknown: That's a biggie. I would say,
you know, when it comes
to tooling and data management,
people look at, like, data management. They're all looking at it from the lens of the tools they currently use,
and thereby, they go natively to the cloud provider and the tool stacks that exist there. So I would say
some of those are already
from our perspective, we're familiar with the kind of tools and capabilities that exist, but the biggest gap would be looking across those tools.
Some of which are built within the cloud
provider's
environment and some of them which are not. So for example,
if you're using
Mixpanel for evaluating
product metrics and Datadog for evaluating
and for
incidents management
and incidents response.
You kind of need to manage different accounts for that. So I would say having a holistic system
where your data management as well as data metrics and insights management are all easy to do together,
which is widely known and used as 1 gap I'm seeing. I'm sure there is some solution out there which tries to solve that, but it hasn't become mainstream where I've seen that might be deployed. So I'd love to hear more about that too.
Unknown: Alright. Well, thank you very much for taking the time today to join me and share the work that you're doing at SUST Global and your
efforts to help bring more visibility
to the information about climate and the ways that it impacts businesses and their operations and the ways that they make decisions. So appreciate all of the time and energy that you and your team are putting into that, and I hope you enjoy the rest of your day. Yeah. Thank you so much. It's been a pleasure to be joining you, and thank you so much for having
Unknown: me. Thank you for listening. Don't forget to check out our other shows, podcast.init,
which covers the Python language, its community, and the innovative ways it is being used, and the Machine Learning Podcast,
which helps you go from idea to production with machine learning.
Visit the site at dataengineeringpodcast.com
to subscribe to the show, sign up for the mailing list, and read the show notes. And if you've learned something or tried out a product from the show, then tell us about it. Email hosts at data engineering podcast.com
with your story. And to help other people find the show, please leave a review on Apple Podcasts and tell your friends and coworkers.