Overview

The Large Language Model (LLM) NIM API endpoints provide simple access to use natural language based generative AI. This single API endpoint provides access to top models for use in a wide range of tasks including: chat, instruction following, question answering, summarization, creative text generation, and code generation.

NOTE: Select models are available as downloadable container images and supported with an NVIDIA AI Enterprise entitlement. These select models have additional OpenAI API spec details for running self-hosted localized NIMs. Please refer to the Downloadable NIM documentation for additional information.

URL: https://integrate.api.nvidia.com

Endpoint: POST /v1/chat/completions

Models

01-ai

Endpoint	Model
Create chat completion (yi-large)	Create chat completion (yi-large)

abacusai

Model	Endpoint
abacusai / dracarys-llama-3.1-70b-instruct	Create chat completion (dracarys-llama-3.1-70b-instruct)

aisingapore

Model	Endpoint
aisingapore / sea-lion-7b-instruct	[Create a chat completion (sea-lion-7b-instruct)](ref:create_chat_completion_v1_chat_completions_post

bigcode

Model	Endpoint
bigcode / starcoder2-7b	Create code completion (starcoder2-7b)
bigcode / starcoder2-15b	Create code completion (starcoder2-15b)

databricks

Model	Endpoint
databricks / dbrx-instruct	Create a chat completion (dbrx-instruct)

deepseek

Model	Endpoint
deepseek-ai / deepseek-coder-6.7b-instruct	Create code completion (deepseek-coder-6.7b-instruct)

google

Model	Endpoint
google / gemma-2b	Create chat completion (gemma-2b)
google / gemma-7b	Create chat completion (gemma-7b)
google / gemma-2-2b-it	Create chat completion (gemma-2-2b-it)
google / gemma-2-9b-it	Create chat completion (gemma-2-9b-it)
google / gemma-2-27b-it	Create chat completion (gemma-2-27b-it)
google / codegemma-1.1-7b	Create chat completion (codegemma-1.1-7b)
google / codegemma-7b	Create chat completion (codegemma-7b)
google / recurrentgemma-2b	Create chat completion (recurrentgemma-2b)
google / shieldgemma-9b	Create chat completion (shieldgemma-9b)

ibm

Endpoint	Model
Create chat completion (granite-34b-code-instruct)	ibm / granite-34b-code-instruct
Create chat completion (granite-8b-code-instruct)	ibm / granite-8b-code-instruct

mediatek

Model	Endpoint
mediatek / breeze-7b-instruct	Create chat completion (breeze-7b-instruct)

Model	Endpoint
meta / codellama-70b	Create chat completion (codellama-70b)
meta / llama2-70b	Create chat completion (llama2-70b)
meta / llama3-8b	Create chat completion (llama3-8b)
meta / llama3-70b	Create chat completion (llama3-70b)

microsoft

Model	Endpoint
microsoft / phi-3-medium-128k-instruct	Create chat completion (phi-3-medium-128k-instruct)
microsoft / phi-3-medium-4k-instruct	Create chat completion (phi-3-medium-4k-instruct)
microsoft / phi-3-mini-128k-instruct	Create chat completion (phi-3-mini-128k-instruct)
microsoft / phi-3-mini-4k-instruct	Create chat completion (phi-3-mini-4k-instruct)
microsoft / phi-3-small-128k-instruct	Create chat completion (phi-3-small-128k-instruct)
microsoft / phi-3-small-8k-instruct	Create chat completion (phi-3-small-8k-instruct)

mistralai

Model	Endpoint
mistralai / codestral-22b-instruct-v0.1	Create code completion (codestral-22b-instruct-v0.1)
mistralai / mathstral-7b-v0.1	Create code completion (mistralai-mathstral-7b-v0.1)
mistralai / mistral-large-2-instruct	Create chat completion (mistral-large-2-instruct-infer)
mistralai / mistral-7b-instruct	Create chat completion (mistral-7b-instruct)
mistralai / mistral-7b-instruct-v0.3	Create chat completion (mistral-7b-instruct-v0.3)
mistralai / mixtral-8x7b-instruct	Create chat completion (mixtral-8x7b-instruct)
mistralai / mixtral-8x22b-instruct	Create chat completion (mixtral-8x22b-instruct)
mistralai / mistral-large	Create chat completion (mistral-large)

nvidia

Model	Endpoint
nvidia / llama3-chatqa-1.5-70b	Create chat completion (llama3-chatqa-1.5-70b)
abacusai / dracarys-llama-3.1-70b-instruct	Create chat completion (llama3-chatqa-1.5-8b)
nvidia-nemotron-4-340b-instruct	Create chat completion (nemotron-4-340b-instruct)
nvidia / nemotron-4-340b-reward	Create reward model (nemotron-4-340b-reward)

qwen

Model	Endpoint
qwen / qwen2-7b-instruct	Create model for given chat completion (qwen2-7b-instruct)

rakuten

Model	Endpoint
rakuten / rakutenai-7b-chat	Create chat completion (rakutenai-7b-chat)
rakuten / rakutenai-7b-instruct	Create chat completion (rakutenai-7b-instruct)

seallms

Model	Endpoint
seallms / seallm-7b-v2.5	Create chat completion (seallm-7b-v2.5)

snowflake

Model	Endpoint
snowflake / arctic	Create chat completion (arctic)

upstage

Model	Endpoint
upstage / solar-10.7b-instruct	Create chat completion (solar-10.7b-instruct)