Continuum AI is now publicly available. Try out the first confidential LLM platform!

Continuum AI

Use LLMs, keep data private.

Continuum protects user prompts from the AI provider, and prevents model weights from being leaked.

What is Continuum?

Continuum is a framework for deploying LLMs and other AI models. It enables the creation of ChatGPT-style services in which both user prompts and model weights are shielded throughout.

With Continuum, the infrastructure and the service provider can never access user prompts and model weights in plaintext.

Continuum integrates with well-known inference servers like NVIDIA Triton, vLLM or Hugging Face TGI.

Continuum will be released as open source in H2/2024.

Continuum AI illustration

Why Continuum?

For model owners

Protect your weights while deploying to untrusted environments.

Provide model weights as encrypted files. Only attested workloads gain access to decrypted weights.

For service providers

By deploying LLMs with Continuum, user requests and responses are kept inaccessible to the inference service provider and infrastructure.

With Continuum, you can offer best-in-class privacy guarantees to users of your platform.

Public preview

You can test Continuum AI now with this public preview, featuring the Mistral 7B model, with Llama 3 70B coming soon.

Continuum AI ensures that no one can access your prompts. Not the model provider. Not the cloud provider. Not us.

Continuum preview screenshot

Why not just ___

___use TLS encryption?

When encrypting network traffic with TLS, the service provider still has access to prompts and weights on their VMs. Malicious or defective workloads can leak prompts and models e.g. by sending them over the network.

___use disk encryption?

When encrypting disk with e.g. LUKS, the service provider has access to the disk decryption key. Malicious or defective workloads can leak model weights e.g. by sending them over the network.

___run on premise?

When running on premise, user prompts and weights are always accessible by the machine and service operator.

Want to chat?

Leave your email or send us your questions.