Completions

POST https://api.cerebras.ai/v1/completions

Python

import os
from cerebras.cloud.sdk import Cerebras

client = Cerebras(
    api_key=os.environ.get("CEREBRAS_API_KEY"),  # This is the default and can be omitted
)

completion = client.completions.create(
    prompt="It was a dark and stormy night",
    max_tokens=100,
    model="llama3.1-8b",
    logprobs=5,
)

print(completion)

Response

{
    "id": "chatcmpl-b1743f2d-8c20-4ad5-a77c-426521ae1b1c",
    "choices": [
        {
            "index": 0,
            "finish_reason": "length",
            "logprobs": {
                "text_offset": [30, 34, 40, 44, 47, 52],
                "token_logprobs": [
                    -3.3912997245788574,
                    -4.322617053985596,
                    -0.3532562553882599,
                    -1.7001153230667114,
                    -2.187103271484375,
                    -2.4667720794677734
                ],
                "tokens": [" and", " there", " was", " no", " moon", ","],
                "top_logprobs": [
                    {
                        ".": -1.1334871053695679,
                        ",": -1.3366121053695679,
                        " when": -2.3366122245788574,
                        " in": -2.9381747245788574,
                        "...": -3.3287997245788574,
                        " and": -3.3912997245788574
                    },
                    {
                        " I": -0.9554294943809509,
                        " the": -1.7679295539855957,
                        " a": -2.2366795539855957,
                        " all": -3.1898045539855957,
                        " we": -3.4241795539855957,
                        " there": -4.322617053985596
                    },
                    {
                        " was": -0.3532562553882599,
                        " were": -1.6813812255859375,
                        " I": -4.0251312255859375,
                        " wasn": -4.1345062255859375,
                        " had": -4.1970062255859375
                    },
                    {
                        " a": -0.3719903528690338,
                        " no": -1.7001153230667114,
                        " an": -3.614177942276001,
                        " this": -4.129802703857422,
                        " nothing": -4.184490203857422
                    },
                    {
                        " electricity": -1.3433531522750854,
                        " one": -1.6949156522750854,
                        " moon": -2.187103271484375,
                        " way": -3.038665771484375,
                        " sign": -3.054290771484375
                    },
                    {
                        ".": -0.8495846390724182,
                        " to": -2.1386470794677734,
                        " in": -2.2870845794677734,
                        ",": -2.4667720794677734,
                        ".\n": -3.2714595794677734
                    }
                ]
            },
            "text": " and there was no moon,",
            "tokens": null
        }
    ],
    "created": 1769297019,
    "model": "llama3.1-8b",
    "object": "text_completion",
    "system_fingerprint": "fp_feb5e1faa8274e54bef0",
    "time_info": {
        "completion_time": 0.002412253,
        "prompt_time": 0.000347391,
        "queue_time": 0.000245702,
        "total_time": 0.004331350326538086,
        "created": 1769297019.2632425
    },
    "usage": {
        "completion_tokens": 6,
        "prompt_tokens": 8,
        "prompt_tokens_details": {
            "cached_tokens": 0
        },
        "total_tokens": 14
    }
}

Generate text continuations from a single prompt string. Best for simple text generation, autocomplete, and single-turn tasks.

Request

prompt · string | array

The prompt(s) to generate completions for, encoded as a string, array of strings, array of tokens, or array of token arrays. Default: ""

model · string · required

The model to use for completion. See Supported Models for a list of available models.

stream · boolean | null

If set, partial message deltas will be sent. Tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: [DONE] message.

Default: false

return_raw_tokens · boolean | null

Return raw tokens instead of text.

Default: false

max_tokens · integer | null

The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model’s context length.

Default: null

min_tokens · integer | null

The minimum number of tokens to generate for a completion. If not specified or set to 0, the model will generate as many tokens as it deems necessary. Setting to -1 sets to max sequence length.

Default: null

grammar_root · string | null

The grammar root used for structured output generation. Supported values: root, fcall, nofcall, insidevalue, value, object, array, string, number, funcarray, func, ws.

Default: null

seed · integer | null

If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. Determinism is not guaranteed.

Default: null

stop · string | array | null

Up to 4 sequences where the API will stop generating further tokens. The returned text will not contain the stop sequence.

Default: null

temperature · float | null

What sampling temperature to use, between 0 and 1.5. Higher values (e.g., 0.8) will make the output more random, while lower values (e.g., 0.2) will make it more focused and deterministic. We generally recommend altering this or top_p but not both.

Default: 1.0

top_p · float | null

An alternative to sampling with temperature, called nucleus sampling, where the model considers the tokens with top_p probability mass. For example, 0.1 means only the tokens comprising the top 10% probability mass are considered. We generally recommend altering this or temperature but not both.

Default: 1.0

echo · boolean

Echo back the prompt in addition to the completion. Incompatible with return_raw_tokens=True.

Default: false

user · string | null

A unique identifier representing your end-user, which can help Cerebras to monitor and detect abuse.

Default: null

logprobs · integer | null

Return log probabilities of the output tokens.

For example, if logprobs is 5, the API will return a list of the 5 most likely tokens. The API will always return the logprob of the sampled token, so there may be up to logprobs+1 elements in the response.

The max value is 20.

Default: null

Setting logprobs to 0 is different than null. When set to null, log probabilities are disabled entirely. When set to 0, log probabilities are enabled but it does not return top_logprobs.

Completion Response

choices · object[] · required

The list of completion choices the model generated for the input prompt.

Show properties

finish_reason · string | null

The reason the model stopped generating tokens. This will be stop if the model hit a natural stop point or a provided stop sequence, length if the maximum number of tokens specified in the request was reached, or content_filter if content was omitted due to a flag from our content filters.

index · integer

logprobs · object | null

Show properties

text_offset · array

Number of characters since the prompt.

token_logprobs · array

Logprob value for each token.

tokens · string

The tokens.

top_logprobs · array

List of the most likely tokens and their log probability, at this token position. In rare cases, there may be fewer than the number of requested top_logprobs returned.

text · string

The generated completion text.

tokens · array | null

The raw tokens of the completion, returned when return_raw_tokens is set to true.

created · integer | null · required

The Unix timestamp (in seconds) of when the completion was created.

id · string

A unique identifier for the completion.

model · string

The model used for completion.

object · string · required

The object type, which is always “text_completion”

system_fingerprint · string

This fingerprint represents the backend configuration that the model runs with.

Can be used in conjunction with the seed request parameter to understand when backend changes have been made that might impact determinism.

usage · object

Usage statistics for the completion request.

Show properties

prompt_tokens · integer

Number of tokens in the prompt.

completion_tokens · integer

Number of tokens in the generated completion.

total_tokens · integer

Total number of tokens used in the request (prompt + completion).

prompt_tokens_details · object

Detailed breakdown of prompt token usage.

Show properties

cached_tokens · integer

Number of prompt tokens that were served from the cache and reused from a previous request. See Prompt Caching for more information.

time_info · object

Performance timing information for the request.

Show properties

queue_time · number

Time spent in queue waiting for processing (in seconds).

prompt_time · number

Time spent processing the prompt/input tokens (in seconds).

completion_time · number

Time spent generating the completion/output tokens (in seconds).

total_time · number

Total time for the entire request from submission to completion (in seconds).

created · number

Unix timestamp (in seconds) of when the time_info was recorded.

Python

import os
from cerebras.cloud.sdk import Cerebras

client = Cerebras(
    api_key=os.environ.get("CEREBRAS_API_KEY"),  # This is the default and can be omitted
)

completion = client.completions.create(
    prompt="It was a dark and stormy night",
    max_tokens=100,
    model="llama3.1-8b",
    logprobs=5,
)

print(completion)

Response

{
    "id": "chatcmpl-b1743f2d-8c20-4ad5-a77c-426521ae1b1c",
    "choices": [
        {
            "index": 0,
            "finish_reason": "length",
            "logprobs": {
                "text_offset": [30, 34, 40, 44, 47, 52],
                "token_logprobs": [
                    -3.3912997245788574,
                    -4.322617053985596,
                    -0.3532562553882599,
                    -1.7001153230667114,
                    -2.187103271484375,
                    -2.4667720794677734
                ],
                "tokens": [" and", " there", " was", " no", " moon", ","],
                "top_logprobs": [
                    {
                        ".": -1.1334871053695679,
                        ",": -1.3366121053695679,
                        " when": -2.3366122245788574,
                        " in": -2.9381747245788574,
                        "...": -3.3287997245788574,
                        " and": -3.3912997245788574
                    },
                    {
                        " I": -0.9554294943809509,
                        " the": -1.7679295539855957,
                        " a": -2.2366795539855957,
                        " all": -3.1898045539855957,
                        " we": -3.4241795539855957,
                        " there": -4.322617053985596
                    },
                    {
                        " was": -0.3532562553882599,
                        " were": -1.6813812255859375,
                        " I": -4.0251312255859375,
                        " wasn": -4.1345062255859375,
                        " had": -4.1970062255859375
                    },
                    {
                        " a": -0.3719903528690338,
                        " no": -1.7001153230667114,
                        " an": -3.614177942276001,
                        " this": -4.129802703857422,
                        " nothing": -4.184490203857422
                    },
                    {
                        " electricity": -1.3433531522750854,
                        " one": -1.6949156522750854,
                        " moon": -2.187103271484375,
                        " way": -3.038665771484375,
                        " sign": -3.054290771484375
                    },
                    {
                        ".": -0.8495846390724182,
                        " to": -2.1386470794677734,
                        " in": -2.2870845794677734,
                        ",": -2.4667720794677734,
                        ".\n": -3.2714595794677734
                    }
                ]
            },
            "text": " and there was no moon,",
            "tokens": null
        }
    ],
    "created": 1769297019,
    "model": "llama3.1-8b",
    "object": "text_completion",
    "system_fingerprint": "fp_feb5e1faa8274e54bef0",
    "time_info": {
        "completion_time": 0.002412253,
        "prompt_time": 0.000347391,
        "queue_time": 0.000245702,
        "total_time": 0.004331350326538086,
        "created": 1769297019.2632425
    },
    "usage": {
        "completion_tokens": 6,
        "prompt_tokens": 8,
        "prompt_tokens_details": {
            "cached_tokens": 0
        },
        "total_tokens": 14
    }
}