Skip to main content

Room for Bigger Models

· 4 min read
Florian Brunner
Product Owner @ ADITO Platform
Tags:

The ADITO AI Runtime has a lot more compute behind it. We can now host and serve models up to ten times the size of what we could run before, through the same API you already use. They are available under the alias adito-llm-max.

This post covers what that means for your integrations and why it is worth trying. At the end there is one small addition: adito-llm-mini.

What changed​

So far, the hardware decided how big a model we could serve. Every model in the runtime had to fit into that budget, which made each choice of base model a compromise between quality and what would actually run.

That budget is now about ten times larger. Model sizes that used to be out of reach for a self-hosted runtime are now on the table. How you call the runtime stays the same: the endpoints, the authentication and the OpenAI-compatible API do not change.

Data sovereignty does not change either. The models run on ADITO's own infrastructure, and your prompts are not used for training. AI Compliance has the details.

Why it is worth using​

Larger models do better on exactly the work where small ones tend to struggle:

  • Long, messy input such as contracts or long email threads. A larger model keeps track of detail across the whole text and is less likely to lose the thread halfway through.
  • Tasks with several steps, where the model has to plan, combine CRM records or call tools in the right order. Fewer wrong turns means fewer retries.
  • Prompts with a lot of constraints at once ("use this format, leave out X, answer in German"). They get followed more reliably.

It also saves you prompt engineering. Many of the workarounds people build around a small model, like extra examples, splitting a task into tiny steps or a second pass to repair the output, stop being necessary.

If you have an integration that works most of the time but still needs someone to check every result, try that one first.

Versioning stays the same​

The larger models are served under their own floating alias, adito-llm-max, and follow the same scheme as every other model in the runtime. The alias points at the current model and moves on its own, just like adito-llm does. Each model also gets a pinned codename that never changes, so you can pin it when your integration needs stable output. Every codename goes through Active (GA), Deprecated and Retired, and the transition window is published before the alias moves.

The full rules are in Model naming and versioning.

Also new: adito-llm-mini​

Some tasks do not need a large model at all. Classifying a ticket, pulling a date out of an email or sorting a short text into one of five buckets are simple jobs, and they often run in large numbers. For work like that, a cheaper model is the better fit.

That is what adito-llm-mini is for: a smaller text-generation model that sits next to adito-llm and costs less to use. It uses the same chat completions API, so switching means changing the model name in the request. Versioning works the usual way: adito-llm-mini is the floating alias, and a pinned codename is available if your integration needs one.

My rule of thumb: start with adito-llm. Move a task up to adito-llm-max when it needs more depth, and down to adito-llm-mini once you know it is simple and you want to keep costs down.

Where to go next​

Current models, codenames and their status are listed in AI Models. Request parameters and code examples are in Text Generation.

./Flo