The Prompt Playground will automatically be available through a Step if a generation object is passed to that step.

Chainlit integrations will take care of this for you if the framework you are using supports it.

​
Example

For this example we are going to create a completion with OpenAI chat API and create a ChatGeneration reflecting the API call made to OpenAI.

Chainlit supports other LLM providers and let’s you implement custom ones. Learn more here.

from openai import AsyncOpenAI

from chainlit.playground.providers import ChatOpenAI
import chainlit as cl

client = AsyncOpenAI(api_key="YOUR_OPENAI_API_KEY")

template = "Hello, {name}!"
inputs = {"name": "John"}

settings = {
    "model": "gpt-3.5-turbo",
    "temperature": 0,
    # ... more settings
}


@cl.step(type="llm")
async def call_llm():
    generation = cl.ChatGeneration(
        provider=ChatOpenAI.id,
        inputs=inputs,
        settings=settings,
        messages=[
            cl.GenerationMessage(
                template=template, formatted=template.format(**inputs), role="assistant"
            ),
        ],
    )

    # Make the call to OpenAI
    response = await client.chat.completions.create(
        messages=[m.to_openai() for m in generation.messages], **settings
    )

    generation.completion = response.choices[0].message.content

    # Add the generation to the current step
    cl.context.current_step.generation = generation

    return generation.completion


@cl.on_chat_start
async def start():
    await call_llm()

​
Result

Once the step is sent, a new button will be available next to it. Clicking on that button will open the Prompt Playground in the context of that step.

Open the Prompt Playground

Was this page helpful?

OverviewTemplate & Variables