提交 3682020c 作者: liang tongtong

添加webui readme说明

上级 57c3cac3
......@@ -10,8 +10,26 @@
✅ 本项目中 Embedding 选用的是 [GanymedeNil/text2vec-large-chinese](https://huggingface.co/GanymedeNil/text2vec-large-chinese/tree/main),LLM 选用的是 [ChatGLM-6B](https://github.com/THUDM/ChatGLM-6B)。依托上述模型,本项目可实现全部使用**开源**模型**离线私有部署**
## webui
![webui](./img/ui1.png)
根据上图依次点击step1~3,即可完成模型加载,加载文件,以及对话历史记录的查看
### TODO
- [ ] 添加模型加载进度条
- [ ] 添加输出内容及错误提示
- [ ] 国际化语言切换
- [ ] 引用标注
- [ ] 添加插件系统(可基础lora训练等)
## 更新信息
**[2023/04/11]**
1. 加入Webui V0.1版本,同步当日之前的更新内容;
2. 自动读取knowledge_based_chatglm.py中LLM及embedding模型枚举,选择后点击setting进行模型加载,可随时切换模型进行测试
3. 可手动调节保留对话历史长度,可根据显存大小自行调节
4. 添加上传文件功能,通过下拉框选择已上传的文件,点击loading加载文件,过程中可随时更换加载的文件
5. 底部添加use via API可对接到自己系统
**[2023/04/07]**
1. 解决加载 ChatGLM 模型时发生显存占用为双倍的问题 (感谢 [@suc16](https://github.com/suc16)[@myml](https://github.com/myml)) ;
2. 新增清理显存机制;
......
......@@ -10,7 +10,26 @@
✅ In this project, [GanymedeNil/text2vec-large-chinese](https://huggingface.co/GanymedeNil/text2vec-large-chinese/tree/main) is used as Embedding Model,and [ChatGLM-6B](https://github.com/THUDM/ChatGLM-6B) used as LLM。Based on those models,this project can be deployed **offline** with all **open source** models。
## Webui
![webui](./img/ui1.png)
Click on steps 1-3 according to the above figure to complete the model loading, file loading, and viewing of dialogue history
### TODO
-[] Add Model Load progress bar
-[] Add output content and error prompts
-[] International language switching
-[] Reference annotation
-[] Add plugin system (can be used for basic LORA training, etc.)
## Update
**[2023/04/11]**
1. Add Webui V0.1 version and synchronize the updated content before the current day;
2. Automatically read knowledge_ based_ Enumerate LLM and embedding models in chatglm.py, select and click 'setting' to load the model. You can switch models for testing at any time
3. The length of the conversation history can be manually adjusted and can be adjusted according to the size of the video memory
4. Add the upload file function, select the uploaded file from the dropdown box, click loading to load the file, and the loaded file can be changed at any time during the process
5. Add use via API at the bottom to connect to your own system
**[2023/04/07]**
1. Fix bug which costs twice gpu memory (Thanks to [@suc16](https://github.com/suc16) and [@myml](https://github.com/myml)).
2. Add gpu memory clear function after each call of ChatGLM.
......
......@@ -27,8 +27,8 @@ embedding_model_dict = {
llm_model_dict = {
"chatglm-6b": "THUDM/chatglm-6b",
"chatglm-6b-int4": "THUDM/chatglm-6b-int4",
# "chatglm-6b-int4-qe": "THUDM/chatglm-6b-int4-qe",
"glm-6b-int4": "THUDM/chatglm-6b-int4",
"glm-int4-qe": "THUDM/chatglm-6b-int4-qe",
}
chatglm = None
......
Markdown 格式
0%
您添加了 0 到此讨论。请谨慎行事。
请先完成此评论的编辑!
注册 或者 后发表评论