The gen AI integration playbook – an interview with Klippa's CTO Jeroen Bobbeldijk

The CTO vs Status Quo series studies how CTOs challenge the current state of affairs at their company to push it toward a new height … or to save it from doom.

“I think integrating AI into software people already use is the most powerful approach.”

If you can’t decide where to apply generative AI and you’re stuck in analysis paralysis, the cure is simple. Look for a repetitive, time-consuming manual process and automate it – says Klippa’s CTO, Jeroen Bobbeldijk.

But Jeroen has much insight available for anyone interested in venturing into the territory of gen AI or Artificial Intelligence in general:

• Are you trying to figure out the best way to start a conversation about AI in your company?
• Are you considering building your own foundation model like GPT or Llama?
• Are you afraid that you don’t have enough data to train your AI solution successfully? 

Dive deep into gen AI implementation with Jeroen.

About Jeroen & Klippa

Klippa’s partner network

Arkadiusz Kowalski: Hello Jeroen. I know that Klippa cooperates with partners to implement AI-driven document automation, and you’re currently expanding your network. Can you tell me more about that and explain what role your partners play in delivering value?

Jeroen Bobbeldijk: Hi Arek! In the past, Klippa focused on selling APIs directly to companies for integration into other software. Now, we’re expanding into building a platform where people can log in, build automation, and deliver these services to end consumers through our partner network.

We see ourselves as a technical company that builds solutions, but we don’t directly implement them for customers. Our partners implement our software themselves. If they have any technical issues or any technical questions, we help them out with that.

Klippa delivers customizable AI-driven solutions across various industries, such as finance, real estate, tourism, transportation, etc. Many of these companies may not be so technologically advanced. Would you say that Klippa’s AI-driven software is a chance for them to grow, or do they need to meet a certain standard before benefiting from it?

If a company doesn’t have a development team, it can be more difficult for them to use the software.

Our solutions integrate into the internal processes of organizations, often through external software and development partners. Our new platform makes it easier for companies to implement solutions themselves through drag-and-drop interfaces. 

However, it still requires some technical knowledge. That’s where our partners come into play – they’re experienced in automating processes, and we’re one of the tools in their toolbox.

Gen AI integration playbook

Companies today are under pressure to try gen AI but often lack the necessary expertise. From what I gathered, some try to ignore it as much as possible in response, while others jump in without a strong foundation – they subscribe to 3rd party tools and tell the world they’re AI-driven but, in reality, don’t even have a good grasp of their data. What’s your take on this situation?

I don’t think people should start using AI directly, like just giving them ChatGPT. I’m a big fan of integrating AI solutions into other software to help people with their work, for example, by giving suggestions in emails or written documents.

Integrating AI into software people already use is the most powerful approach. It doesn’t require knowledge about prompts or trusting the output. It helps people, and they don’t really have to do anything to get those benefits.

On the other hand, when you give AI tools like ChatGPT to your employees, they can become really creative and find ways to use it that you didn’t think of yourself. It depends on the type of company. If you’re an old-school company that isn’t used to technology, I wouldn’t give people something like ChatGPT directly.

Since we’re a tech company, it’s different for us. We use gen AI to help developers write code by providing suggestions in their code editor and during merge reviews. We also offer a self-hosted ChatGPT for employees to aid them in writing marketing copy or emails. 

However, we always caution people not to trust the output, especially for factual information, blindly. Use it to be creative, but don’t use it like it’s a guaranteed fact.

How difficult is it for an average IT company – with no prior AI experience – to use pre-built gen AI models in its own software?

It’s becoming easier. APIs from OpenAI and other providers are similar. They use the same schemas, so you can easily switch. They’ve added features to generate responses in specific JSON schemas, making it easier to implement in other software. You just provide input text or image and specify the desired output format. 

AI is getting faster and cheaper, allowing for more context and expanding the possibilities.

I wouldn’t advise anyone to build or host their own language model as a starting point. Go with a big provider, or use Azure if you’re worried about privacy. It’s complex and expensive to host yourself. Providers are currently losing money to gain market share. Once you reach a certain usage level, then consider hosting it yourself.

Since different LLM providers are fighting for market share, one will ultimately win. Do you think the cost will change then?

I think so. I suspect they’re burning money to train new models and bring them to the market. I don’t know for sure if that’s the case, but we’re in the AI model space, so we know how expensive it is to train and host models.

Right now, providers keep dropping their prices because their competitors keep getting better. They’re all similar, and it’s very easy to switch between them, so they either have to provide better results, better speed, or lower costs.

It’s the classic playbook for startups that want to become unicorns – give it away almost for free to gain market share, and once the winner emerges, they’ll start raising the prices.

Creating value with gen AI

Let’s talk about generative AI in the short, medium, and long term. Say we’re at the office of a CTO who was given some time and resources to brainstorm AI initiatives that could create business value.

In the short term, we’re usually looking to optimize existing processes or save money. So, how do you find opportunities for generative AI usage in your business?

The trick is to look for tasks that require a lot of manual, repetitive labor that can be automated using generative AI. Consider where you can assist people by integrating gen AI services into existing products to give hints or pre-fill fields. It’s about helping employees work more efficiently or faster by skipping certain steps by using AI.

Also, look for existing tools that integrate with your current systems, like CRM or goal management systems. If a plugin allows you to use generative AI, it can be an easy step toward implementing AI in your company. We’re seeing more and more SaaS companies that are implementing gen AI to make their products more powerful.

How can you measure short-term success with such generative AI initiatives?

Don’t try to force it completely into a whole process. Start small so you can go through ideas quickly and change how you use generative AI. Start with proof of concepts in various places, measure the wins in that process, and decide early on whether to continue or not.

AI has been around for a long time, but now it’s more accessible than ever, and there’s immense hype around it – so there’s a lot of pressure for companies to use it. Sometimes, it ends up being used in processes where it doesn’t fit or doesn’t provide any real benefits. That’s why I think it’s always good to do a proof of concept.

Have a few people from different departments involved. Not just tech people but also the end users. Talk about AI, test it out, don’t rush it. And then you’ll know if it’s worth it or not.

In many companies, various processes are still manually performed by humans even though they no longer require human involvement.

At Klippa, we focus on document processes like invoices, where a lot of manual labor is involved in entering them into systems and validating them. We try to automate these systems, freeing people from boring manual tasks and making them more efficient in their work.

In the medium term, CTOs can look for ways to use generative AI to add new features or attract new users. We’ve been working with a company that switched from acquiring individual users to getting them in batches through deals with large companies. This overwhelmed their team because they still had to onboard each user manually, but they were able to automate onboarding with AI. What do you think about AI’s potential to create business value this way?

I think it’s really powerful.

The most important use case of AI is integrating it into products that assist people in performing tasks more efficiently.

If manually onboarding people takes a long time, that’s a perfect use case for AI. I see the future in integrating AI into products rather than having people use AI directly.

So far, you have said a lot about existing AI tools. But when a company invests in AI in the long term, and its entire product and business model begin to revolve around AI, isn’t it time to think about building their own AI model?

It depends on what you see as a fully-fledged AI product.

For example, you can use other tools to build models trained on your own data for a specific use case. This approach allows you to own the model without knowing how to build it from scratch. I don’t think it’s necessary to be an AI company to do this.

Regarding reliance on a particular provider, all these APIs are similar, so you can migrate to another provider quite quickly if needed. The issue might be that the new provider could be more expensive. At some point, it might be worth looking into making the model yourself. The future will show if the prices will grow.

Right now, you can’t build and host a model for the same cost as using a 3rd party API; it’s just impossible. So, it makes more sense to keep using APIs instead of building your own models. 

At Klippa, we make small models for specific use cases that are affordable to host on a single GPU. This approach could be the future of AI – smaller, more specific models that are more accurate for particular tasks. But, it does require in-house knowledge of how to make these models.

Speaking of in-house knowledge, what kind of specialists do companies need to hire or foster in-house to do this kind of work?

You need different tech wizards. You need people who build models, i.e., data scientists, to ensure you have the correct data and backend engineers to scale the solution in production. 

AI model engineers focus on model performance, while back-end engineers focus on cost-effectiveness and faster results. Those are the two main types of developers you need to build AI solutions.

It looks like you need a methodical approach and a modest test drive to think about seriously starting an AI initiative.

Yes, for sure.

If you don’t have in-house technical knowledge, try to get a partner on board who is familiar with different AI technologies and tools. Let them help you do a proof of concept. 

For example, our related company, DataNorth AI, provides workshops on using AI within companies, identifies processes suitable for AI, and guides companies in using AI or even building custom solutions. This allows companies to use AI without having to acquire specific in-house knowledge.

But I’ve heard from some clients that short workshops aren’t enough to make a big difference. What would you say to them?

It all depends on the situation.

We offer a multi-step plan, not just a workshop. After the initial workshop, we look into company processes to identify potential AI use cases, which can lead to proof-of-concept projects. 

I agree that a one-hour workshop won’t make anyone proficient in AI, but it helps people understand what AI can do for them and how it works.

Our service can include strategy and roadmap development and proof-of-concept delivery. If the customer so wishes, we can go from start to finish.

The feedback so far has been very positive. People are often amazed by AI’s potential, especially when they see specific use cases implemented in their company that they hadn’t thought of themselves.

Gen AI technology stack

Do you have recommendations regarding technologies or products that CTOs should consider to improve processes or save money with AI?

For developers, GitHub Copilot is something they really enjoy and find powerful. But apart from that, we don’t really buy too many tools. We build our own.

If you build AI models or work with them directly, you do that in Python because that’s what all the tools and libraries are made for. You can build, train, and host models yourself, but that’s the most complex and expensive route.

When using generative AI from providers like OpenAI, it’s just an API that you can use with any language, like Java or .NET.

If you want more control and prefer not to use a proprietary model like the ones from OpenAI, there are also open-source models from Microsoft and Meta that you can use straight away. This can be a powerful way to use AI without training or building models yourself.

What’s your advice for companies interested in training a model to be the foundation of their business?

Probably don’t do it unless you have a lot of money. Instead, train more specific models. Look at the real problem you’re trying to solve and try to make a really specific model for that. 

Look for research papers addressing similar problems and look at the types of models they’ve been using. It’s much easier to make a small model for a specific case and implement, build, and host it yourself than to take a large language model and try to make it do something new. 

Start small, go from there.

Getting enough data to train your own models may be difficult, though. Is synthetic data a good solution for this problem?

For sure, we often generate synthetic data. For example, we can take an existing invoice, mark the places where we want to change data and generate new content in those places. This way, we can generate hundreds of thousands of samples to train our model without collecting real data.

Often, there’s just not enough data to train a reliable model on. It’s a big problem. But the data you have can be changed to become something new. Data is really similar – you have an invoice with different text, but often, you don’t have many different suppliers. So you don’t really need a lot of different invoices; you just need different variations of the same invoice. 

The same goes for ID cards or other use cases we have experience with. In this context, synthetic data can be really powerful as well.

Another important factor is that you don’t have to annotate the data. Since you already know what’s on there, you put it there yourself. You skip a whole step in model training because you have the ground truth. The data can go directly into the training pipeline.

What about working with companies that sell synthetic data for others to use?

We do it in-house, but there are many companies that make synthetic data for you to use in your models. It takes quite some time to do it the way we do it, so I think it makes sense to go with a service like this when you’re just starting out.

Resources

Do you have any learning resources that CTOs can use to educate themselves about AI?

Try to look for specific papers that concern the same kind of problems that you want to solve. Look at the type of models they’ve used and their methods. 

Then, if you have the resources for it, the best way to learn is to do a proof of concept. Define your problem, look for anyone else who has tried to solve it before, try to replicate their methods, or build something yourself.

What’s next? Four actions for CTOs to take

Ready to start implementing AI in your organization?

For the best ROI in gen AI, remember that it’s best to:

• find a manual process that could be automated and start with a small proof of concept,
• test it out and talk about it with people from different departments and the end users, not just tech people,
• get as much data quality and quantity as you can within a short time, but discard the idea quickly if there’s no improvement,
• keep going until you find a case where AI provides real benefits.

Good luck with your gen AI initiative!