Connect Your LLM of Choice to the ABBYY Document AI Platform

Large language models (LLMs) have changed what teams expect from document automation. But plugging an LLM into a real business process is not as simple as pasting a prompt into a chat window. You need control over which model runs, where your data goes, and how it all fits into a governed pipeline.

The ABBYY Document AI platform now lets you connect your own LLM directly to your document processing workflows, so you can pair ABBYY's document expertise with the generative model that best suits your needs. Here is how it works.

Key takeaways

• Integrating an LLM into business workflows demands careful consideration of control, data governance, and compatibility with existing processes.
• The ABBYY Document AI platform enables direct connections to your preferred LLM, ensuring flexibility and alignment with your unique requirements.
• By combining ABBYY's document processing expertise with tailored generative models, organizations achieve smarter automation while maintaining oversight, reliability, and enterprise-grade control.

Supported LLM providers and connections

You bring the model; the platform handles the integration. Right now, you can connect five LLM providers natively through Connections, and access a sixth as an ABBYY-managed option:

How to choose which LLM to use:

• Anthropic gives you access to the Claude model family, known for strong reasoning and long-context document handling. Setup requires just an API key and a model name, making it straightforward to add if your team already uses Claude.
• OpenAI ChatGPT offers the simplest setup, with just an API key and a model. It is a good fit if your team already works on the OpenAI platform.
• Microsoft Foundry connects to OpenAI and a plethora of other models hosted on Azure. It suits enterprises that need data-residency controls, Azure integration, or custom model hosting. You provide the full endpoint URL, including the deployment path and API version.
• Google AI Studio (Gemini) gives you cost-effective access to Google's multimodal models, ideal if you already run on Google Cloud.
• Mistral AI provides a range of efficient, open-weight models well suited to structured extraction tasks. Setup requires an API key and a model name.
• PHOENIX Plus is ABBYY's managed LLM connection. There is nothing to configure: no API key, no endpoint, no model selection. ABBYY provisions, hosts, and maintains the model on your behalf. It is purpose-built for document processing and integrates directly into your existing IDP pipeline. See our dedicated Feature Spotlight for a full overview of PHOENIX Plus.

We are actively working on expanding the list of supported models (including proprietary and self-hosted models) to be able to serve the needs of all our customers.

How to set up a connection

Connecting a model takes just a couple of minutes and no code. You will need administrator access to Configuration and valid API credentials from your provider. Here is the six-step process:

1. Open Configuration. In the ABBYY Vantage interface, find the ADMIN section in the left sidebar and click Configuration. You will see the Connections menu.
2. Add a connection. Click Add Connection, then pick your provider: Anthropic, Google AI Studio (Gemini), Microsoft Foundry, Mistral AI, or OpenAI ChatGPT. If your tenant has the PHOENIX Plus entitlement, the ABBYY PHOENIX Model connection appears automatically and requires no setup.
3. Configure the details. Give the connection a clear name, select the model you want to use, and enter the required credentials for your provider.
4. Test it. Click Test. A green checkmark confirms the connection works. A red error means you should double-check your key, URL, or model name.
5. Save it. Once the test passes, save the connection. It becomes immediately available to every skill designer in your tenant.

Because connections live at the tenant level, your whole team can reuse them. Tenant administrators can also override the default connection per environment without republishing a skill.

For more information on how to set up an LLM connection in your Vantage tenant, visit the documentation.

Demo video: How to set up a connection

Where LLM connections are used

Once your connection is live, you can put it to work inside the Prompt-based Extraction activity in Advanced Designer. This is where a connected LLM helps you extract data from documents using natural language instructions.

We covered that feature in depth in its own Feature spotlight article: "Prompt-Based Extraction: How to Use LLMs Within Your Governed Document Processing Pipeline"

Connecting other models today

Not every model sits on the native provider list yet. If your organization runs a proprietary, self-hosted, or otherwise unsupported LLM, you can still connect it today through the Custom Activity.

A Custom Activity collects document data from a transaction and sends it to any external system over an HTTP request, using a script you configure. The response comes back and saves to the transaction data. In practice, that means you can:

• Route documents to a self-hosted or proprietary model behind your own endpoint.
• Choose which text, fields, and images to send to the external system.
• Keep control over exactly what data leaves your pipeline.

The Custom Activity requires a secure SSL connection, and you can place it inside a For Each Document activity to process one document at a time.

What's next with ABBYY’s LLM connections

LLM Connectivity gives you real flexibility from day one: native support for leading providers, a quick setup, and a script-based path for everything else. And we are just getting started.

ABBYY is actively expanding native LLM provider support, including proprietary and self-managed models, with more integrations on the way. Until those land natively, the Custom Activity bridges the gap, so you never have to wait to connect the model your workflow depends on.

Ready to connect your first model? Head to Configuration > Connections in Vantage and add a connection today.

Discover the full potential of the ABBYY Document AI platform and explore all of its robust capabilities by visiting abbyy.com/ai-document-processing/.