Interoperability in AI Solutions
GPT-4 is the dumbest model any of you will ever have to use again.” OpenAI CEO Sam Altman recently told a room at Stanford University…“We can say right now with a high degree of scientific certainty, GPT-5 is going to be a lot smarter than GPT-4. GPT-6 is going to be a lot smarter than GPT-5 and we are not near the top of this curve”.
Altman is of course right (and would soon prove himself correct upon the release of GPT-4o later that month). In terms of GenAI solutions, we have barely begun to experience what GenAI can do.
At the same time, what is is becoming increasingly clear is that GenAI solutions might not always have to involve proprietary Large Language Models such as GPT-4o (or 5 or 6 or…).
Which is why at ACQUAINTED we advise all clients who who are embarking on building GenAI solutions that they do so with interoperability in mind. In short, model-specific problems require model agnostic solutions. But what do we mean by this?
Vendor, Vidi, Vici
Interoperability essentially means the capability to integrate and exchange data or services among different AI models or systems without being locked into one vendor or platform.
This flexibility is crucial for avoiding vendor lock-in, which can restrict an organisation's ability to adopt new technologies, negotiate pricing, or customise solutions to meet specific needs. Interoperability allows companies to switch providers or integrate multiple solutions without significant barriers or costs.
Being able to integrate and leverage the best features of various models means that companies aren't limited to the capabilities of a single provider but can combine strengths from multiple sources.
Different tasks might require different AI capabilities and interoperability allows for the customisation of solutions by mixing and matching AI models that are best suited for specific tasks.
Not only that, but relying on a single AI model or vendor can be risky if that model fails or the vendor experiences issues. Interoperable systems can mitigate this risk by allowing an easy switch to alternate systems or spreading dependency across multiple platforms.
Bet On Open Source
A week after Meta released its family of open-source LLM Llama 3 models, it was reported that they were downloaded over 1.2 million times. In that same timeframe, developers shared over 600 derivative models on the open source data science and machine learning platform Hugging Face, and, at the time of writing, the Llama 3 GitHub repo had already passed 17,000 stars (which is a lot).
More impressively, Llama 3 70B Instruct was tied for first for English-only evaluations on the LMSYS Chatbot Arena Leaderboard — the crowdsourced open platform for LLM evaluations. Overall, it sat in sixth place making it the highest ranked openly sourced available model, just behind closed proprietary models.
Initially positioned as less refined but accessible alternatives to proprietary software, open source tools are progressively encroaching on territories once dominated by expensive, high-performance solutions. As we will continue to see, the gap between open source and proprietary models is now virtually non-existent — a game of inches that will continually close time and time again.
In functionalities like chatbots, open source models now match proprietary ones in key performance areas like task completion rates and accuracy. Open source language models also offer complete access to their source code, architecture, and training datasets, providing a level of transparency that supports thorough testing and adjustment that proprietary models just don’t provide.
As a result, and as language model applications continue to evolve, often driven by user innovation, open source platforms are becoming crucial in fostering diverse and creative technological advancements.
There are also adjacent benefits of using open-source. From putting an end to the intellectual property debate by disclosing their training data, or more generally fostering much-needed trust in technology thanks to their transparent nature, there are many benefits to open-source that are rarely mentioned.
In fact, there is even an argument that open models can also drive sustainability in GenAI by enabling the sharing of pre-trained model weights, reward models, and other artifacts which are resource-intensive to produce.
Now this is not an advertisement for Meta, it is a forewarning for enterprises beguiled by the likes of OpenAI’s ChatGPT and other proprietary models and who perhaps don’t understand that there are other often cheaper (sometimes better) alternatives. Open-source Llama is one such LLM, but in the future, it is likely even LLMs might be the perfect match either…
Go Small, Be Mighty
Domain-specific models are likely the next port of call for enterprise use cases of GenAI. Decentralised, privacy-first and highly specialised, domain-specific models are built for specific tasks or industries in mind.
It makes sense that this is the trajectory. Larger, broader models make for larger, broader problems and wrangling information out of maelstroms of data — rather than more specific, smaller datasets — is quite clearly not the most efficient way of doing things.
Instead, domain-specific models are tailored and fine-tuned for specific fields or areas of expertise. This specialised focus allows these models to excel in intricate and heavily regulated sectors where accuracy is paramount, providing essential support to the professionals and teams employing the technology, such as lawyers, medical practitioners, and financial analysts.
This goes for both language and vision models. While LLMs did well with applying knowledge sourced from datasets such as Common Crawl into proprietary use cases, vision models do not have that same luxury. If you trained a vision model on an internet full of cats and memes, it would not be effective at applying itself to medical documents, construction parts etc.
Now, whether or not you need a specific model is another story altogether. Depending on the complexity and intricacies of the task at hand, a specialised model might be necessary. If your tasks demand nuanced understanding or deep knowledge within a specific area, a tailored approach becomes vital.
Additionally, consider the complexities of your industry — does it involve unique processes, specialised terminology, strict regulatory frameworks, compliance requirements, or particular ethical considerations? If the answer is yes, then investing in a specialised model equipped with the appropriate context and expertise is not just beneficial, but likely essential. These factors collectively determine the suitability and necessity of opting for a domain-specific model.
Next, are the considerations around what type of domain-specific model to use? Will you build one from scratch? Build on top of an existing model? Or perhaps utilise one of the many growing foundational domain-specific model providers? This is a question for another article — get in touch today if you want to know the answer.
As for GPT-4 being the dumbest model you ever will use, we can’t help but agree. So make sure you don’t cross the Rubicon if you don’t have to.
This article first appeared in The Playbook, download the entire edition for free today.