Creates a model response for the given chat conversation.

post https://integrate.api.nvidia.com/v1/chat/completions

Given a list of messages comprising a conversation, the model will return a response. Compatible with OpenAI. See https://platform.openai.com/docs/api-reference/chat/create

Body Params

model

string

Defaults to nvidia/llama3-chatqa-1.5-8b

messages

array of objects

required

length ≥ 1

A list of messages comprising the conversation so far. The conversation optionally starts with one or more context or context_asset_id messages. The roles of the messages after that must be alternating between user and assistant. The last input message should have role user with content populated.

Messages*

role

string

required

The role of the message author.

content

Defaults to what is the percentage change of the net income from Q4 FY23 to Q4 FY24?

The contents of the message.

temperature

number

0 to 1

Defaults to 0.2

The sampling temperature to use for text generation. The higher the temperature value is, the less deterministic the output text will be. It is not recommended to modify both temperature and top_p in the same call.

top_p

number

0 to 1

Defaults to 0.7

The top-p sampling mass used for text generation. The top-p value determines the probability mass that is sampled at sampling time. For example, if top_p = 0.2, only the most likely tokens (summing to 0.2 cumulative probability) will be sampled. It is not recommended to modify both temperature and top_p in the same call.

frequency_penalty

Defaults to 0

Indicates how much to penalize new tokens based on their existing frequency in the text so far, decreasing model likelihood to repeat the same line verbatim. https://platform.openai.com/docs/api-reference/completions/create for details. Higher values increase the penalty.

presence_penalty

Defaults to 0

Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

max_tokens

integer

1 to 1024

Defaults to 1024

The maximum number of tokens to generate in any given call. Note that the model is not aware of this value, and generation will simply stop at the number of tokens specified.

seed

Defaults to null

If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result.

stream

boolean

Defaults to false

If set, partial message deltas will be sent. Tokens will be sent as data-only server-sent events (SSE) as they become available (JSON responses are prefixed by data: ), with the stream terminated by a data: [DONE] message.

Responses

Language

Credentials

Bearer


xxxxxxxxxx
22
1
curl --request POST \
2
     --url https://integrate.api.nvidia.com/v1/chat/completions \
3
     --header 'accept: application/json' \
4
     --header 'content-type: application/json' \
5
     --data '
6
{
7
  "model": "nvidia/llama3-chatqa-1.5-8b",
8
  "messages": [
9
    {
10
      "content": "what is the percentage change of the net income from Q4 FY23 to Q4 FY24?",
11
      "role": "user"
12
    }
13
  ],
14
  "temperature": 0.2,
15
  "top_p": 0.7,
16
  "frequency_penalty": 0,
17
  "presence_penalty": 0,
18
  "max_tokens": 1024,
19
  "seed": 42,
20
  "stream": false
21
}
22
'

Click Try It! to start a request and see the response here! Or choose an example:

application/json

text/event-stream

200Invocation is fulfilled

202Result is pending. Client should poll using the requestId.

422Validation failed, provided entity could not be processed.

500The invocation ended with an error.