project 1 | Tianze Luo

The Panda project, launched in May 2023, is a pioneering open-source initiative in the field of large language models, comprising three sub-projects: PandaLLM, PandaLLMOps, and PandaIndex. We achieved 1000+ stars on GitHub.

Released Models

7B models

Panda-7B: Base model (pretrain) with 7B parameters.

Panda-OpenLLaMA-7B: Base model (pretrain) with 7B parameters, based on OpenLLaMA.

Panda-Instruct-7B: Model with pretrain and instruction-tuning, 7B parameters.

Flan-LLaMA-7B: Fine-tuned LLaMA-7B model with the Flan dataset.

13B models

Panda-13B: Base model (pretrain) with 13B parameters.

Panda-LLaMA2-13B: Base model (pretrain) with 13B parameters, based on LLaMA2 model.

Panda-13B-Chat: Chat model (pretrain & SFT) with 13B parameters.

Panda-LLaMA2-13B-Chat: Chat model (pretrain & SFT) with 13B parameters, based on LLaMA2 model.

Special models

Code-Panda-13B-Python: Python code completion model with 13B parameters.

Legal-Panda-13B-Chat: Legal service and consultation model with 13B parameters.

Panda-index-large-en: An information retrieval model with 1.3B parameters.

Technical Report

2023

arXiv

Panda LLM: Training Data and Evaluation for Open-Sourced Chinese Instruction-Following Large Language Models

Fangkai Jiao*, Bosheng Ding* and Tianze Luo* and Zhanfeng Mo*

arXiv preprint , 2023

Abs PDF

This project focuses on enhancing open-source large language models through instruction-tuning and providing comprehensive evaluations of their performance. We explore how various training data factors, such as quantity, quality, and linguistic distribution, influence the performance of instruction-tuned models trained on publicly accessible high-quality instruction datasets for both English and Chinese languages. Our goal is to supplement evaluation with quantitative analyses, providing valuable insights for the continued advancement of open-source chat models. Our model, data, and code are publicly available for others to use and build upon.