> For the complete documentation index, see [llms.txt](https://paper.lingyunyang.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://paper.lingyunyang.com/paper-list/artificial-intelligence/language-models.md).

# Language Models

* Grok-2 \[[Blog](https://x.ai/blog/grok-2)]
  * xAI
  * Grok-2 Beta was released on 2024/08/13.
* Gemma 2: Improving Open Language Models at a Practical Size (arXiv:2408.00118) \[[arXiv](https://arxiv.org/abs/2408.00118)] \[[Code](https://github.com/google-deepmind/gemma)]
  * Gemma Team, Google DeepMind
  * **Gemma 2**
  * Models: <https://www.kaggle.com/models/google/gemma>
* The Llama 3 Herd of Models (arXiv:2407.21783) \[[arXiv](https://arxiv.org/abs/2407.21783)] \[[Blog](https://ai.meta.com/blog/meta-llama-3/)] \[[Code](https://github.com/meta-llama/llama3)]
  * MetaAI
  * **Llama 3**
  * Models
    * Llama 3 8B: <https://huggingface.co/meta-llama/Meta-Llama-3-8B>
    * Llama 3 70B
    * Llama 3 405B
* Mixtral 8x7B (arXiv:2401.04088) \[[arXiv](https://arxiv.org/abs/2401.04088)] \[[Blog](https://mistral.ai/news/mixtral-of-experts/)] \[[Code](https://github.com/mistralai/mistral-inference)]
  * Mistral AI
  * **Mixtral 8x7B**
  * Model: <https://huggingface.co/mistralai/Mixtral-8x7B-v0.1>
* Llama 2: Open Foundation and Fine-Tuned Chat Models (arXiv 2307.09288) \[[Paper](https://arxiv.org/abs/2307.09288)] \[[Homepage](https://ai.meta.com/llama/)]
  * Meta AI
  * **Llama 2**
  * Released with a *permissive* community license and is available for commercial use.
* LLaMA: Open and Efficient Foundation Language Models (arXiv 2302.13971) \[[Paper](https://arxiv.org/abs/2302.13971)] \[[Code](https://github.com/facebookresearch/llama)]
  * Meta AI
  * **6.7B, 13B, 32.5B, 65.2B**
  * Open-access
* PaLM: Scaling Language Modeling with Pathways (JMLR 2023) \[[Paper](https://www.jmlr.org/papers/v24/22-1144.html)] \[[PaLM API](https://developers.googleblog.com/2023/03/announcing-palm-api-and-makersuite.html)]
  * **540B**; open access to PaLM APIs in March 2023.
* BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (arXiv 2211.05100) \[[Paper](https://arxiv.org/abs/2211.05100)] \[[Model](https://huggingface.co/bigscience/bloom)] \[[Blog](https://bigscience.huggingface.co/blog/bloom)]
  * **176B**
  * open-access
* OPT: Open Pre-trained Transformer Language Models (arXiv: 2205.01068) \[[Paper](https://arxiv.org/abs/2205.01068)] \[[Code](https://github.com/facebookresearch/metaseq/tree/main/projects/OPT)]
  * Meta AI
  * Range from 125M to 175B parameters.
  * Open-access