Model Description

RakutenAI-7B is a systematic initiative that brings the latest technologies to the world of Japanese LLMs. RakutenAI-7B achieves the best scores on the Japanese language understanding benchmarks while maintaining a competitive performance on the English test sets among similar models such as OpenCalm, Elyza, Youri, Nekomata and Swallow. RakutenAI-7B leverages the Mistral model architecture and is based on Mistral-7B-v0.1 pre-trained checkpoint, exemplifying a successful retrofitting of the pre-trained model weights. Moreover, we extend Mistral's vocabulary from 32k to 48k to offer a better character-per-token rate for Japanese.

This model is ready for commercial use.

Model Developer Rakuten Group, Inc.

License This model is licensed under Apache License, Version 2.0.

Language(s) Japanese, English

Third-Party Community Consideration:

This model is not owned or developed by NVIDIA. This model has been developed and built to a third-party’s requirements for this application and use case; see link to Non-NVIDIA 7B Instruct Model Card.

References:

RakutenAI 7B instruct Model Card on Hugging Face

RakutenAI 7B instruct paper

Model Architecture:

Architecture Type: Transformer

Network Architecture: Mistral-7B

Model Version: 0.1

Input:

Input Type(s): Text

Input Format: String

Input Parameters: Max Tokens, Temperature, Top P

Limitations & Bias:

The suite of RakutenAI-7B models is capable of generating human-like text on a wide range of topics. However, like all LLMs, they have limitations and can produce biased, inaccurate, or unsafe outputs. Please exercise caution and judgement while interacting with them.

Output:

Output Type(s): Text

Output Format: String

Software Integration:

Supported Hardware Platform(s): NVIDIA Lovelace

Supported Operating System(s): Linux

Training Datasets:

• JSNLI
• RTE
• KUCI
• BELEBELE
• JCS
• JNLI
• Dolly-15K
• OpenAssistant1

Inference:

Engine: Triton Inference Server

Test Hardware: NVIDIA L40 Systems

Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please report security vulnerabilities or NVIDIA AI Concerns here.