Unknown: Hello, and welcome to The Machine Learning Podcast. The podcast about going from idea idea to delivery with machine
learning.
Unknown: Your host is Tobias Macy, and today, I'm interviewing Piero Molino about the application of declarative ML in a world being dominated by large language models. So, Piero, can you start by introducing yourself?
Unknown: Sure. Thank you, by the way, for having me, Tobias. I really appreciate the opportunity.
I'm Pierre, and I'm the CEO of PlaidiBase, which is a company bringing declative machine learning,
to the enterprise. And previously, I've been working at many different organizations. In particular,
my last stint was,
4 years at Uber where I was doing research and applications
and where I developed Ludwig, which is an open source declarative machine learning framework that now is the foundation of the technology that we are building at private base.
Unknown: And do you remember how you first got started working in machine learning?
Unknown: Yeah. Well, it was quite quite a long time ago, honestly,
about
more than 10 years now, probably 13, 14 years. My my introduction to machine learning was through actually for recommender system
because I was interested in information retrieval and filtering, and I was working on this
little path project of mine
about
how to induce serendipity
in recommender systems.
That's what got me started and interested in figuring out
machine learning in particular for personalization,
but then that brought it up into, like, just natural language processing and then later on many different topics surrounding machine learning.
But it all got started from the perspective of,
retrieval information and information filtering.
Unknown: As with so many projects that get people into a particular field, they usually end up getting put to the side in favor of all the other work that comes along. I'm curious if you ever actually got to a satisfactory point of being able to actually introduce that serendipity in the recommender systems.
Unknown: I would say not really. Although I wrote a few papers about it and my my bachelor thesis, I would say,
about it. And,
that paper, despite the fact that it didn't really have
a lot to it, was more in terms of solution,
but it actually posed the problem
in an interesting way.
For a long time, it was my most cited paper despite being my first and worst paper,
but,
that then it was superseded by the my more recent work. But for a long time, it actually was was was my most cited 1.
Unknown: And so
bringing us to the topic at hand now, for anybody who hasn't been living under a rock for the past 1 or 2 years, large language models have definitely been
taking the ML and AI world by storm. They have broken out of being a subject purely for academia or industry insiders and have now been introduced into the general population. Everybody's talking about it. I'm wondering if you can just start by summarizing your perspective on the effect that these LLMs have had on the AI and ML industry.
Unknown: Yeah. It's certainly like a pretty big effect from many perspectives. On 1 hand, you have the fact that
companies that before companies and also individuals
that before were not thinking
at AI at all or they were not thinking about it as something to pay attention to immediately or they were thinking about it as something futuristic that eventually they would have started to consider.
Now by the availability
of something like ChargeGPT and interfaces,
they make it pretty
tangible
that these models have capabilities
that are particularly useful for solving tasks and also for just interacting with with with data
that opened up
the possibility set of possibilities and imagination
of people and companies.
From the perspective of someone building
machine learning
tooling
that made it so that there's way more people that now are interested in the space
and way more demand.
At the same time,
the that also comes with a caveat, which is,
most of these people,
may not have a deeper understanding
of the implications
and the complexity
of building
infrastructure
and tooling
around the machine learning.
And so there's, like, a little bit of
education that we need to do more than what we were doing before in order to be able for people to fully grasp
what actually we are doing.
The other aspect to it is that
these new mechanisms through using LLMs in particular
lower the barrier of entry for many tasks
to people who
maybe do not have
training data.
Before, in order to be able to start to do some machine learning, at least you needed to start from some training data. Now you can start without it, And there are limitations obviously, but it's it's a much lower barrier of entry,
for nontechnical people. And that's something that obviously is changing the way the industry is thinking about it. And also we are thinking about it as a company.
Unknown: And to that point of
LLMs lowering the barrier to
getting some sort of product level capability
deployed,
Wondering what are the potential pitfalls
or some of the
issues that you've seen
teams or individuals
encounter in their haste to be able to bring this new capability
either into their product or even just in the experimentation
phase, maybe some of the ways that LLMs are failing to live up to the hype?
Unknown: Yeah. So that's,
definitely something that that that we are observing. I will say, on the 1 hand, you have
really great and new capabilities in particular around generation
and in particular about
creating content and
reformatting,
if you want,
even
content like structured information into something that is textual and colloquial and understandable.
And so these capabilities, like, are really
impressive and unprecedented
for the
in in the machine learning space and in the AI space. Right? That said, obviously, there's some, pitfalls, though, that you were mentioning.
1 of them is absolutely the fact that,
there is a mismatch in terms of expectations
of what these models
can actually provide in terms of answering questions
and what actually the questions that people really want to ask are.
Some clear examples about it are, like,
the kind of interaction that some of the users we have seen having with these models is kinda like trying to,
ask an oracle
about
how how to answer something, but it's really
not what these models are. These models are generating the next token and doing it in a smart way, but they cannot really
predict the future. They cannot really
answer
questions if the data to support it is not given to them. Right?
Another pitfall that we are seeing is,
also
expectations on what the output will look like.
And so,
these models, again, are trained to generate the next token and then fine tuned for following instructions,
but still
their
goal is to produce text
even despite anything, right, despite everything else. Meaning that even in situations where
there's
no clarity
or that they like, the model is not certain about something, they will generate it anyway
and,
in most situations.
And that means that it could generate things that are
untrue, could generate things that are
imprecise.
It could generate there's no grounding
that actually happens
at the time of the generation of the outputs.
And there's techniques to mitigate that, like, for instance, providing additional context, providing some evidence, and so on, but still,
it's not always the case. There's there's no guarantees. Let me put it this way. Right? So these are definitely pitfalls. And then I would say the last 1 that I would mention is the fact that
the
way through which some of these models are consumed right now, so mostly through APIs,
may not be the best 1 for for everyone. There's there there are, like, a few pitfalls with that too,
lock in and also
throttling that API providers can do, or also the APIs providers
can change the models under the hood for you and, you know, then what worked before could not work anymore.
And all of these things you have no control over. Right?
So that's part of the reason why we believe that the platform, in particular, the Clarity platform using open source models like us,
can be really valuable in this new landscape.
Unknown: On that point of open source models,
the
providers that brought about this initial wave of excitement, particularly in the general community,
is the OpenAI,
chat GPT, and GPT series of models. But as you mentioned, there are a number of open source offerings that have been fast followers and that are focusing on different niche use cases or different specific applications. I'm wondering if you can give a summary of your understanding of what that open source model landscape looks like and some of the ways that those models are looking to differentiate from OpenAI in particular, but also the other commercial,
LLM providers.
Unknown: Yeah. Absolutely. So, and we're, you know, strong believers in open source. All our technology is built on top of open source projects,
and also the models that we serve and use
are are open source, so, you know, it's we are really vested in in this space. Right?
I can tell you there are a few aspects about the open source that are particularly interesting. 1 is the speed at which the
field is moving. There's literally a new model every week, or actually more than that, but 1 new very relevant model every week,
and the pace of progress is very, very fast, and the community is rallying together
in making these models available for everybody, right? And also some bigger players like Meta, for instance,
have started to make their own models
available under permissive licenses, so there's a lot of, excitement,
that is building in the open source space,
and different models have different
skills and characteristics.
The particular interesting thing in my mind
is that despite the fact that most of these models are actually trained more or less on the same data and more or less fine tuned on the same instructions datasets.
The possibility
of having them, for instance, trained on
the the datasets in different languages,
and the availability of models in different languages that is something that is exciting, that is not really present in the commercial offerings as much, and also the fact that models can be fine tuned for specific
needs and specific customers'
adaptations,
that is also something that the,
let's say, larger providers
do, but to to a really small extent,
and also don't give you ownership of, you know, whatever you've been
fine tuning and adapting,
and so the personalization
aspect is,
1 that makes for, like, a really exciting use cases for open source models.
Unknown: For people who are using these LLMs as a component of their stack, there have been
a number of different architectures that have grown up around it, in particular
using things like vector databases for generating the context embeddings to be able to
feed into the model to customize it to your particular application.
I'm wondering if you can talk to some of the ways that,
what you're building at prediabase and the overall concept of declarative ML can
be used to maybe simplify the
operational and architectural aspects of working with these LLMs and incorporating them into the workflow and serving?
Unknown: Absolutely. Absolutely.
I would say the,
just
for for for the sake of giving, like, a more complete answer,
what
the approach that we're following with the Clari. M l, really, the way it consists of is you can write configurations
of your machine learning pipelines
that are like YAML files and really short and human readable, and that makes it possible
for you to go from, like, 0 to, like, a deployed model
very, very fast without having to write low level machine learning code and do the orchestration of putting together components like, you know, vector DBs and other things like that, right?
What we have been working on
is to add these kind of capabilities
into our both open source and platform,
and basically what we want to try to do is to handhold
users
from the very beginning of a journey, which, like, it's literally
deploying a model to query
it, all the way to, like, fine tuning it, and in the middle, there's, let's say, different degrees of application of models. So on 1 hand, once you deploy them, then you may want to run them
both for real time prediction, but also on batches of data, and if we have optimizations
for running them on batches of data,
then you may want to do,
the very first thing that you should try when you want to adapt a machine a large language model on your data is actually retrieval augmented using something like a vector DB, and the reason for that is that it's very cheap to do compared to the fine tuning, and so it's
the obvious first thing to try and do, and so
you can write a configuration very easily to specify
what are
the
prompts
and the parts of the prompt
that retrieve the information,
from from the backdoor database within the configuration file,
and that, you know, makes it possible for you to iterate on prompts by changing just the configuration file.
Unknown: Another interesting aspect of
the idea of LLMs,
especially in juxtaposition to the idea of declarative ML is that they're both aiming to provide a similar level of ease of access and a shallower on ramp to being able to take advantage of AI capabilities
for product or business reasons.
I'm curious what you see as some of the different ways that people are coming into
1 or both of these tool chains and approaches
of, the the types of problems that they're trying to solve and, some of the maybe misconceptions about the capabilities of LLMs that they have and some of the ways that your work at prediabase and, undeclarativeml
helps to address some of those shortcomings.
Unknown: So, yeah, I would say the intent, to a certain extent, is similar. Right?
The clarity of machine learning wants to lower the barrier of entry for building machine learning tools,
and LLMs actually lower the barrier for using
what underneath is a machine learning tool. Right?
I would say
the way the things unfold is slightly different, though, because
with the creative machine learning,
the output that you have on process
is
actually a machine learning pipeline
that then you can use for, you know, your own use cases or to adapt to your data and do whatever you want with it, right?
You don't obtain like a general
system, really, but you get like a system that is task specific, right? You can also train something like an LLM, but
it's pretty costly. You can definitely do it. It's valuable in some situations in particular when your data is very bespoke
and doesn't
really jive with like general data coming from the internet, but in general, the goal is to build the model
that you want to deploy and use. Right?
With LLMs,
the model is already built for you and you're just querying it and interacting with it, and as we were discussing, there are mechanisms for adding
knowledge about your data into the process, like with
retrieval augmented generation
or also with fine tuning them, but
the
first barrier of entry is substantially different. Right?
And, again, for us, these 2 things come together in the form of providing also the Clarity configurations
for the, deployment models,
adding
retrieval augmented generation to them and also fine tuning them. So basically, we bridge the gap from what does it take to start using a model to adapting it to your data through providing
information in the context,
that you
are giving to the model for generating
and also using that data for adapting the model to your use case and everything in between. Right? So we bridge the gap from
0 to,
an entire model fine tuned for you and everything in between.
Unknown: Another aspect of the
hype around LLMs that is interesting is that it seems to have completely eclipsed the conversations around transformer models that were the previous hype cycle, at least within the ML ecosystem.
I'm wondering what you see as the relative
use cases for transformer models as compared to LLMs and maybe some of the reasons that these LLMs have
become much more
widely adopted,
maybe some of the challenges that existed in the transformer architecture that are simplified in LLMs and where you still might wanna use the transformer approach.
Unknown: I would say, you know,
LLMs are transformers. At least right now, there's, like,
competing,
architectures,
but I think the the the main difference is not really in terms of the details of how the models are provided,
but more
the mechanism for using them, and so for instance, if you're thinking about a transform model like BERT,
they were trained on large amounts of data, but not for generating text,
but there was, like, a side,
let's say, task. There was basically infilling of missing pieces of text, but the goal was not really to use them for that task. That task was an auxiliary task for being able to obtain good embeddings that then could have been used for downstream tasks.
The model would generate embeddings and those embeddings are then used
for fine tuning or
for training a smaller model, a smaller head
on a specific task. Right?
Using that obviously requires some
machine learning
knowledge,
and again, for instance, this is something that you can do
with our declarative ML approach in Ludwig and in PratyBase. You can just say, I have an input to this text and I want to use BERT for encoding it, and I want to have an output that is whatever, a category.
We made it as simple as that, but it's still different than typing text in a chat
box and and getting answers out of it. Right?
Despite the fact that underneath trans the the architecture that is used for both these LLMs that are deployed
and the BERT architecture. There's very, very little difference between the 2,
so this is like something
that has changed, I would say, but on the other hand, transformer models like BERT and like more, recent versions of it are actually used for
still, for instance, for retrieval augmented.
When you're putting your data in a vector DB, you need to provide a vector for that data, and, these kind of models are used for that purpose.
So they're still part of the, conversation, although they are not the ones that people talk about the most.
Unknown: And, also,
LLMs have been widespread
for these textual and natural language
applications, but there are still a number of different modalities for which machine learning is still very useful and valuable. I'm curious what you have seen as some of the ways that people are trying to address the multimodal,
use cases, particularly dealing with images or
dealing with sensor information
or some of the other applications where pure natural language is not sufficient?
Unknown: Yeah, absolutely, and again,
the declarative ML approach is particularly well suited for that. 1 of the reasons is in the configurations
files that we make very easy to write in Ludwig and Pradibase,
you can specify the data type associated
with each input that you are providing to the model, and those data types could be,
both tabular data types, for instance,
like categories, binary, and numerical values, more complex data types, like text, image, and audio, and also, like, time series, like time series coming from sensors,
geospatial data,
more complex structures like SATS and BAGS,
so
the multimodality
is
definitely an aspect
that is
is being addressed by
declarative ML, and declarative ML makes it very easy
to, train models on these multiple modalities all at once.
The, you know, interesting thing is that you can also reuse some of these LLM capabilities, for instance, when you decide how the text is treated within this larger model that the Claritima configuration
defines.
You can also use
the models that,
the LLM models just instead of
repeating the predictions
to generate a token at a time, you're just up at the level of obtaining the embeddings from them.
There's a lot of work that is happening in this side. Obviously, now it's a bit less flashy and a little bit less
in in the forefront of people's imagination,
but the the,
industrial and enterprise applications
of multi of merging multiple modalities
are
potentially even stronger than the applications of
of of just text interfaces.
Unknown: Another shortcoming or constraint that I've seen brought up a number of times also is the question of the size of the context windows that you're able to achieve, and there's definitely still ongoing research to expand that. And I'm wondering what are some of the ways that that manifests
in terms of
what applications
are viable with LLMs and which still require custom model development to be able to accommodate
either larger context windows
or being able to customize to your specific data set
in lieu of having that larger context window to give the model enough information to work with.
Unknown: Yeah. No. It is it is a really
a really glaring
limitation. Right?
1 example of what we have seen
unfolding
on
why that's particularly relevant and why that's particularly important
for real world applications
is that,
for instance,
even for retrieval augmented,
generation, which is to a certain extent a mechanism
to overcome
that limitation in context because instead of
giving like an entire document, maybe a document that will be longer
than
the window sides,
you can give just
a few snippets from the document that are retrieved
to fill that window size with only only the relevant pieces of information.
Still, that that may be not sufficient. I can give you, like,
some examples that we're seeing in particular when you need to
provide
information about
all of the elements of something, like, for instance, all the passages of text
within within a document to be able to actually obtain
the correct answer, then you're out of luck. Right? That goes beyond what most models now there's, like, some large ones, that make it possible to use
very long window sizes,
but still, it's not sufficient.
1,
like, 1 customer that we've been working with,
has been
trying to answer questions on their
SQL,
data, and,
they wanted to actually
do aggregates
over over their information
despite the fact that that may not have been, like, the best,
idea to do it with an LLM as opposed to, like, running a SQL query. But,
the kind of question that they were asking was not a question that, like, a SQL query could could answer,
but they could not provide the entire data to be able,
for the model to do aggregate
reasoning on top of it,
and this kind of limitation,
you know, is is is because the window size is not large enough.
Now there's a lot of research that's been happening in increasing that window size,
although I think a combination
of
better, faster models that
can work on larger window sizes
with,
better retrieval mechanisms,
to retrieve
the relevant parts
that are, like,
sufficient for answering the questions.
It's still open,
it's an open research to find the right balance between these 2. Right?
I've seen recently
work coming from
the HAZY Lab at Stanford that I was part of in the past,
that is, like really goes deep in the direction of long sequences using state models, and I think that's a very,
valid
line of research that could bring us improvements in this direction.
Unknown: The other thing that is brought up a lot for people who are actually working with large language models, even just recreationally, is the the challenge of hallucination
where the models just
completely forget what it is that they're trying to do, or they,
base successive predictions on faulty information.
And I'm wondering what you have seen as some of the useful mechanisms for being able to
identify the occurrence of those hallucinations
and,
potentially
counteract them or ways
to,
prevent those hallucinations from occurring in the first place?
Unknown: Yeah. So
the those are like a big problem. I actually don't really like the term hallucination,
because it makes you look like it's
a bug while it's definitely a feature of these models. That's how they are trained. That's what they are supposed to do, really, generating text, so, you know, I wouldn't consider it.
It is a bug in the term in in terms of what we want this model to do,
but
it's not a bug in terms of what they are supposed to do.
So it is a big issue though. In particular,
there's a lot of work that has been going on on the alignment side to try to limit,
but it's kinda like a a a whack a mole, if you want.
You identify
areas
where models,
confidently provide answers and kinda
address them partially through, like, multi
hop reasoning and multi hop kind of interactions
and,
like, constitutional
approaches
to make
to to limit the impact of it. I don't think anyone has found yet a mechanism
for
guaranteeing
any,
avoiding the problem altogether.
That said, you know, there are also retrieval augmented mechanisms, for instance, are a way to
partially address it, and
sometimes you go can go surprisingly far
in
actually specifying to the model what you want
them to do or not to do. I can give a concrete example. If you use retrieval augmented and you retrieve some snippets of text, you can specify
only use
words, only use
information
from the retrieved text to answer the original question, and the model, most of the times, will try to comply to that.
Again, you have no guarantees,
but expert like, empirically, you can see that in most cases, that actually,
ends up happening. I think the problem there is
if a model like that is then exposed to end users that may be
trying to
stress it or be malicious about it, then that may not
work anymore because they could send prompts that are contradicting
the request to be,
truthful
to the,
information retrieved. Right? So
it it it really depends on the, final application. Right? But a combination of retrieval augmented
with constitutional,
training of models, I think it's the best that we have found so far.
Unknown: The other aspect of these LLMs that have proven challenging
is the
specifics of the data that they're being trained on and some of the questions around bias,
inclusivity,
the
types of information that they have available, particularly when you're starting to deal with non English
language,
use cases.
And I'm wondering what your,
thoughts and experience have been around
ways that
we, as an industry, can try
to address some of those challenges
or at least
improve the visibility of those shortcomings to the end consumers of these models.
Unknown: This is a really complex topic, and it's,
let's say, difficult to answer in a way that in a short way that is not, like,
dismissive of many of the aspects there. I'll try to do my best, but
I would say
on the 1 hand, you have,
the issue of the preselection of the data. Right?
The models are only
they don't have
any
inherent bias, really,
if not if it's not coming from the data that they've been trained on. Right?
So that is definitely
1 aspect that is important.
They're just reflecting or amplifying
the issues that are already there in the original data.
If you,
again, train on data that contains them,
maybe, you know,
finding the right balance in
large datasets
that
are less biased
is is is difficult because
empirically, the more token you train these models on, the better, but also the higher quality of the data that you're training them on, the better, and it was demonstrated pretty clearly by
by both versions of LAMA, for instance,
that
just exactly the same architecture but trained for longer and on better data and on bigger data actually
produces better models. Right?
On the other hand, there are,
aspects of
like, what about
the
the systematicity
of the use of these models. Right?
Because even if a model is biased,
if it, if that bias
is mitigated
in many different ways, like, for instance, there's multiple models
and you, you know,
mix them or average them or have something that,
and maybe this model is trained on different types of data. Obviously, if you train on all the same data, the same biases will emerge. That could be a mechanism for for limiting biases, and it's not super different from what it seems to be the mechanisms that both,
palm and the more recent gpt4
have been trained using a mixture of experts,
but the issue is still if a model produces
consistently a biased answer to the quest to to questions and
all people ask similar questions, they will all receive the same biased answer, and so a bias that was maybe minimal in the data can be amplified dramatically
by the use of the models,
and there's no clear answer there, on how to address this problem.
1 interesting
talk that was, like, on a different topic that is seen given by Michael Jordan from Berkeley was about
routing users
of,
map system
towards a destination,
and the
idea there was,
clearly, if there's a short
route from point a to point b but you give the same suggestion to all your users,
then that route is not the shortest anymore, right, because there's going to be traffic induced by the fact that you are the 1 providing these suggestions
to everyone.
I think a similar kind of dynamic
happens with
large language models,
and so maybe
a potential solution there is to actually start to give
different answers and
be more, I don't know, sampling
more with higher temperature
from these,
from these models than what we've been doing before to try to diversify the outputs.
Unknown: In your experience of working with large language models and with your customers
and finding
the ways that declarative ML and LLMs
can be used in conjunction or declarative ML can help with the
most interesting or innovative or unexpected ways that you've seen them applied?
Unknown: Yeah. So I think on on 1 hand, you have, the tasks. So there's,
many interesting tasks that, you know, I was not
expecting before,
so I've seen quite a few
interesting users. 1 the latest 1 that I've seen is applying,
LLMs
with
with data,
coming from sport events
to get an edge in playing fantasy football, and it was, like, a pretty fun application that I've seen recently,
but I would say the most surprising users more than surprising, I would say the most interesting users for me are users that are beyond
tracked users.
For instance,
information extraction is a very interesting application.
We have been, you know,
for instance, demonstrating it by applying it to, datasets of CVs and extracting information that is not
100%
there in the CV, but can be inferred from the CV.
That was like a a fun application
or in general applications
of mapping back,
back and forth,
data that is structured
into data that is like textual, so generating out of structured information,
and
on the opposite from
unstructured information, textual generating structured information.
Those kind of applications,
I think,
are really great for enterprise applications because then you can have,
systems that are,
certain systems, like, you know,
databases
and and
structured SQL queries
on top of information that is unstructured and vice versa.
You can generate human readable
human readable versions of structured information
for users to consume
without having to look at tables of data. So I think both, are are pretty interesting applications.
Unknown: And in your experience of working in the space of declarative ML, particularly in this time where LLMs
have been taking over a lot of the attention? What are the most interesting or unexpected or challenging lessons you've learned in the process?
Unknown: Yeah. I would
say
probably the most interesting,
lesson that I've learned is that,
and it may be a little bit obvious, but there's no silver bullet. We have seen applications and customers
for which the pretrained models were
worked really, really well on their tasks,
other customers that really needed to fine tune their models
on the,
on their data set to get good performance because retrieval augmented generation was not good enough,
for their use cases,
and also customers that had, went had to go all the way to train models from scratch
for applying them to their
to their use cases because
the pre trained LMs
will not, like,
really work well for their use cases. Right? And so the, let's
say, meta learning there is that,
which is also what we're trying to put in the platform as as, as a consequence,
is that every use case is really different,
and what you want to have is a platform that supports you
in being able to experiment
with all forms of,
techniques
because,
you cannot know
beforehand
which 1 will be the 1 that delivers the value,
in terms of
capabilities, performance,
and good trade offs with speed and performance that makes them usable in your for your use cases in your applications.
The learning is that
all of these things can work in different use cases, and you need to figure them out,
empirically on your use case,
Unknown: before being able to apply them for real. And so for people who are interested in being able to incorporate AI into their product offerings or even just internal capabilities, what are the cases where an LLM is the wrong choice?
Unknown: Yeah. I would say LLM is different to the wrong choice for use cases where there's really no data in the open that relates to that. 1 example that we give, using openly available datasets,
by the way, so
it could have been the case that the model could have been trained on them,
but,
so we have this example that we show to users
of
a dataset of Twitter users,
with some
labels that
telling if they are
bots or humans,
and there is textual information about them. There's, like, the self provided description.
There's also some structured information about it,
like the number of followers and the number of,
tweets that they post and things like that, and also, like, images, they profile pictures. Right?
And if you ask an LLM to provide some predictions,
on on a user like that, if it's a human or a bot,
The best LLMs are actually the ones that tell you that they don't have enough information to be able to make the determination,
which basically tells us that this
is 1 example use case, but there's many like this where training models
from scratch
is actually still the preferred way of solving
your task,
and,
you obviously need to collect the data, in order to be able to train these models from scratch,
but
that kind of investment
is is worth if you are,
if you want to have good performance at the task. Right?
Unknown: And so
as you continue to
build and iterate on prediabase and declarative ML capabilities
and help to support the LLM ecosystem and applications, what are some of the things you have planned for the near to medium term or any particular projects or problem areas you're excited to dig into?
Unknown: Yeah. So there's a couple. The first 1 and most important is,
the fact that we're coming up with a new release of Ludwig, 0 dot 8 release,
that,
includes all these LLM capabilities
through the clouded configurations
that we've been discussing, and we are gonna release it in, like, very shortly.
And I'm super excited about it because it's the 1,
stop,
place where you can do all these things at once
very very easily,
so, super excited about it.
And then,
in terms of applications,
we are working heavily
on a few, including
information extraction,
and in particular, information extraction
that, you know, we have seen
many companies being super interested in,
and
it's
a very promising space because
it's a it's a, like, a step function in the
level of performance given the amounts of data that
is needed
compared to previous to previous solutions, and so, it's it's a very exciting new space for us to explore this information extraction.
Unknown: Are there any other aspects of this overall space of large language models and declarative ML and the work that you're doing at predabase that we didn't discuss you'd like to cover before we close out the show?
Unknown: Yeah, I would say 1
important topic in my mind is
the data aspects.
Obviously,
large language models, like every other machine learning model and also every piece of software really,
is garbage in and garbage out, so the quality of the data
is very, very important,
still in a world where we are interacting with text with with our AI
models. Right?
And to to this extent, you know, we have been doing a lot of work
in connecting the dots between
the place where the data lives,
the, you know, data warehouses, databases, Snowflake, Databricks, places where
the data deals,
and the place where the model live, which is, like, a platform like ours,
and basically connect the dots and making it so that the quality of the data that is piped through these pipelines
is good enough to be able to obtain really good results,
and those actually the outcomes of the
high quality pipelines
are very important
to avoid
situations
where
there's data loss. And so the
monitoring of of the quality of the data and the connection between
the data sources and models only in 1 place makes it possible for,
like, very fast pipelines and very reliable pipelines,
and having the capability of
putting these things together, the data and the models, in 1 place, is 1 of the added value,
that our platform provides that we're really proud of and that we think is going to be particularly valuable for customers.
Unknown: Alright. Well, for anybody who wants to get in touch with you and follow along with the work that you and your team are doing, I'll have you add your preferred contact information to the show notes. And as the final question,
Unknown: and
I don't think that there's
a huge barrier of adoption
in terms of
how easy it is to use and interact with the models today. I think
the barrier is more on the disconnect
between the expectations of what the capabilities of the models are and what the models can actually deliver,
and so
I think education
and conversations like this,
are actually the key to be able to set the right expectations
and for people to adopt the right solution for the right problem.
Unknown: Alright. Well, thank you very much for taking the time today to join me and share the work that you're doing at prediabase and your thoughts on the large language model ecosystem and ways that declarative ML can help to
simplify and improve
on the application of these new capabilities
to different business problems. So I appreciate all the time and energy that you and your team are putting into that, and I hope you enjoy the rest of your day. Thank you so much, Tobias. Thank you for having me. Thank you for listening. And don't forget to check out our other shows, the Data Engineering Podcast, which covers the latest in modern data machine
Unknown: learning
podcast.com
to
subscribe
to the show. Can visit the site at the machine learning podcast.com
to subscribe to the show. Sign up for the mailing list and read the show notes. And if you've learned something or tried out a project from the show, then tell us about it. Email host at the machine learning podcast.com
with your story.
To help other people find the show, please leave a review on Apple Podcasts and tell your friends and coworkers.