AI chat

pleate_backend.ai_chat stores conversations and streams answers from a model provider, with reasoning, sources, stop, regenerate and a daily token quota per user. It backs ChatThread, Message, Composer, StreamingText, Reasoning, ModelPicker, PromptSuggestions and SourceList. The provider is a setting: Anthropic, OpenAI, any OpenAI compatible server, or your own class.

Providers

Anthropic

pip install "pleate-backend[anthropic]". Thinking deltas become reasoning events, web search citations become sources, images go as base64 image blocks and PDFs as document blocks.

PLEATE["AI"] = {
    "PROVIDER": "pleate_backend.ai_chat.providers.AnthropicProvider",
    "PROVIDER_OPTIONS": {"api_key": env("ANTHROPIC_API_KEY"), "max_tokens": 8192},
    "MODELS": [
        {"id": "claude-sonnet-5-5", "name": "Sonnet", "description": "Balanced",
         "params": {"thinking": {"type": "adaptive"}}},
        {"id": "claude-opus-5-5", "name": "Opus", "description": "Most capable"},
        {"id": "claude-haiku-4-5-20251001", "name": "Haiku", "description": "Fastest"},
    ],
    "DEFAULT_MODEL": "claude-sonnet-5-5",
}

OpenAI and compatible servers

pip install "pleate-backend[openai]". Works with OpenAI and anything that speaks its API: LiteLLM, vLLM, Ollama, OpenRouter. A model entry can carry its own options (base URL, key), so one picker can mix providers. Usage comes from stream_options.include_usage, and reasoning_content deltas (DeepSeek, vLLM, LiteLLM) become reasoning events.

PLEATE["AI"] = {
    "PROVIDER": "pleate_backend.ai_chat.providers.OpenAIProvider",
    "PROVIDER_OPTIONS": {"api_key": env("OPENAI_API_KEY")},
    "MODELS": [
        {"id": "gpt-5", "name": "GPT-5", "description": "Smart"},
        {"id": "llama", "name": "Llama", "description": "On our servers",
         "api_model": "llama-3.3-70b",
         "options": {"base_url": "http://litellm:4000/v1", "api_key": env("LITELLM_KEY")}},
    ],
    "DEFAULT_MODEL": "gpt-5",
}

A MODELS entry may set provider and options (its own provider class and constructor options; the global PROVIDER_OPTIONS only apply to the global PROVIDER), api_model (the id sent upstream, defaults to id) and params (passed to the SDK call: temperature, max_tokens, thinking, reasoning_effort...). Only id, name and description are public.

Echo, for development

The default EchoProvider needs no key: it answers "You said: ..." word by word. [error] in a message makes it fail, [sources] makes it cite a source, and the delay option slows it down.

Your own provider

from pleate_backend.ai_chat.providers import Provider, ProviderError

class CookbookProvider(Provider):
    def stream(self, messages, *, model, system, options):
        # messages: the history (role, content, attachments); options: the model's params
        try:
            for chunk in my_sdk.stream(model=model, system=system, messages=messages, **options):
                yield {"type": "delta", "text": chunk.text}
        except my_sdk.RateLimited as exc:
            raise ProviderError("provider_rate_limited") from exc
        yield {"type": "usage", "input_tokens": 812, "output_tokens": 240}

Events a provider yields: delta, reasoning, sources and usage. A ProviderError code reaches the user in the error event.

Endpoints

Mounted at /api/ai, signed in users only. Conversations and messages of other users answer 404.

The stream

Sending answers with Server-Sent Events, unless the request has Accept: application/json: then it waits and answers 201 {user_message, assistant_message} (regenerate answers 200 {assistant_message}). Events, in order:

event: user_message
data: {"message": {"id": "…", "role": "user", "content": "Which products are low on stock?", …}}

event: start
data: {"message": {"id": "…", "role": "assistant", "status": "streaming", "content": "", …}}

event: reasoning
data: {"text": "The user wants inventory below the threshold."}

event: delta
data: {"text": "Three"}

event: delta
data: {"text": " products are below 10 units:"}

event: done
data: {"message": {"id": "…", "status": "complete", "content": "Three products are below 10 units: …", "input_tokens": 812, "output_tokens": 46, …}}

• The stream never breaks with a 500. A provider failure sends error, saves the message with status error and error_code, then done.
• Validation and quota errors happen before the stream starts and are regular JSON errors.
• The assistant message is saved about every second while it streams, and at the end, also when the client disconnects. Reloading the page mid answer shows what was written so far.
• The first exchange titles an untitled conversation with the first message, cut to 60 characters on a word boundary.
• Regenerate keeps previous answers in variants, oldest first.

Messages have role, content, reasoning, status, error_code, model, sources, attachments ([{id, name, content_type, size}]), variants, input_tokens, output_tokens and created_at. Attachments are ids of files from the files app owned by the user; useChat uploads Files from the composer for you.

On the frontend

const models = useModels();
const chat = useChat(conversationId, {
  model: models.model,
  onConversation: (c) => router.replace(`/chat/${c.id}`),   // when the first message creates it
});
const suggestions = useSuggestions((prompt) => chat.send({ text: prompt }));

<ChatThread>
  {chat.messages.map((m) => (
    <Message key={m.id} role={m.role}>
      {m.reasoning && <Reasoning active={m.status === "streaming"}>{m.reasoning}</Reasoning>}
      <StreamingText text={m.content} streaming={m.status === "streaming"} />
    </Message>
  ))}
</ChatThread>
<Composer {...chat.composerProps} toolbar={<ModelPicker {...models.modelPickerProps} />} />

useChat returns messages (history plus the live answer), streaming, error, send, stop, regenerate, deleteConversation, hasOlder and loadOlder for scrolling back, and composerProps. useConversations lists conversations for a sidebar, useAiUsage reads today's quota.

Settings

PLEATE["AI"]:

Error codes