手把手一起在本地CPU上部署ChatGLM3-6B

迷南。 2024-04-20 11:46 31阅读 0赞

### 1 下载ChatGLM3-6B代码 ###

如下图所示，下载链接: [ChatGLM-6B][]

![在这里插入图片描述][87415ce173ad4b2eb7bf276e3e9ce78a.png_pic_center]

### 2 下载ChatGLM-6B模型 ###

可以在modelscope下载模型，下载地址[ChatGLM-6B-Models][]

![在这里插入图片描述][85db69ee4f474630b699dc6038c5d06e.png_pic_center]  
模型文件如下图所示：

![在这里插入图片描述][59315cad52c148bb9770fdde9e185a73.png_pic_center]

### 3 C/C++编译环境安装 ###

安装TDM-GCC，下载地址[tdm-gcc][]

![在这里插入图片描述][f2825377444a49c79a6804007b1cfb8f.png_pic_center]  
开始安装软件，点击Create

![在这里插入图片描述][58ada25236e64858bca441b9b936dffe.png_pic_center]  
选择安装版本，然后Next

![在这里插入图片描述][40f9c5f318b44f2da9f6e966d4d50b12.png_pic_center]

选择安装路径，然后Next

![在这里插入图片描述][7fee85081923406199efddbae45f0f32.png_pic_center]  
打开gcc选项，向下找openmp

![在这里插入图片描述][0e253478841b4270b06351a67e1c74ae.png_pic_center]  
勾选openmp，然后Install

![在这里插入图片描述][d56e261fb5504d718cb1ddc89029e575.png_pic_center]  
等待安装完成

![在这里插入图片描述][4d8e016336504bd5afcc38a278204b38.png_pic_center]  
安装完成

![在这里插入图片描述][7466161dde0a4f49b6197d985a0617ed.png_pic_center]  
检查是否安装成功

![在这里插入图片描述][dac02c5bbc8b42db9e8322a67a1047a9.png_pic_center]

### 4 ChatGLM-6B运行环境 ###

进入pytorch官网，查找安装指令

![在这里插入图片描述][aa394d9557904e2580469c89b9db888a.png_pic_center]  
安装cpu版本pytorch，首先进入自己的虚拟环境，可参考该博客链接: [Python环境的安装（Anaconda+VS code+Win 10）][Python_Anaconda_VS code_Win 10]

![在这里插入图片描述][ccb7b2ec0aa040b99723ccbac6615493.png_pic_center]  
开始安装pytorch

![在这里插入图片描述][471b8346176a4d42926e1dd28f3bb0c4.png_pic_center]  
正在安装，如图所示

![在这里插入图片描述][e96e92a6a93146e2be32c53bf7c4ca58.png_pic_center]  
检查是否安装成功

![在这里插入图片描述][7afe025b9b9245e49098658d4630a315.png_pic_center]  
安装transformers，如图

![在这里插入图片描述][bdafd472283b4d84aa20c5c92afcb13c.png_pic_center]  
安装 chardet，如图

![在这里插入图片描述][64a4918f13374012a567127b89b57306.png_pic_center]  
安装 sentencepiece，如图

![在这里插入图片描述][31cd1ac29dcc431f99565d2af259f527.png_pic_center]  
因为本人环境中已经安装过其它包，故本次仅安装以上所列，大家可以根据开始下载的ChatGLM3-main程序包中requirements.txt文件所列进行安装

### 5 开始运行测试 ###

准备好环境后，便可以进行测试，运行代码如下

import os
    import streamlit as st
    import torch
    from transformers import AutoModel, AutoTokenizer
    
    MODEL_PATH = os.environ.get('MODEL_PATH', './THUDM/chatglm3-6b')
    TOKENIZER_PATH = os.environ.get("TOKENIZER_PATH", MODEL_PATH)
    
    st.set_page_config(
        page_title="AI_ChatGLM_Assistant",
        page_icon=":robot:",
        layout="wide"
    )
    
    @st.cache_resource
    def get_model():
    
        tokenizer = AutoTokenizer.from_pretrained(TOKENIZER_PATH, trust_remote_code=True)
        # model = AutoModel.from_pretrained(MODEL_PATH, trust_remote_code=True, device_map="auto").eval()
        model = AutoModel.from_pretrained(MODEL_PATH, trust_remote_code=True).float()
        return tokenizer, model
    
    tokenizer, model = get_model()
    
    if "history" not in st.session_state:
        st.session_state.history = []
    if "past_key_values" not in st.session_state:
        st.session_state.past_key_values = None
    
    max_length = st.sidebar.slider("max_length", 0, 32768, 8192, step=1)
    top_p = st.sidebar.slider("top_p", 0.0, 1.0, 0.8, step=0.01)
    temperature = st.sidebar.slider("temperature", 0.0, 1.0, 0.6, step=0.01)
    
    buttonClean = st.sidebar.button("清理会话历史", key="clean")
    if buttonClean:
        st.session_state.history = []
        st.session_state.past_key_values = None
        if torch.cuda.is_available():
            torch.cuda.empty_cache()
        st.rerun()
    
    for i, message in enumerate(st.session_state.history):
        if message["role"] == "user":
            with st.chat_message(name="user", avatar="user"):
                st.markdown(message["content"])
        else:
            with st.chat_message(name="assistant", avatar="assistant"):
                st.markdown(message["content"])
    
    with st.chat_message(name="user", avatar="user"):
        input_placeholder = st.empty()
    with st.chat_message(name="assistant", avatar="assistant"):
        message_placeholder = st.empty()
    
    prompt_text = st.chat_input("请输入您的问题")
    if prompt_text:
        input_placeholder.markdown(prompt_text)
        history = st.session_state.history
        past_key_values = st.session_state.past_key_values
        for response, history, past_key_values in model.stream_chat(
                tokenizer,
                prompt_text,
                history,
                past_key_values=past_key_values,
                max_length=max_length,
                top_p=top_p,
                temperature=temperature,
                return_past_key_values=True,
        ):
            message_placeholder.markdown(response)
        st.session_state.history = history
        st.session_state.past_key_values = past_key_values

运行时是使用streamlit运行web的方式，在终端输入命令如下

streamlit run app.py

![在这里插入图片描述][fec4c31b8ee74bc99a9721f21646db46.png_pic_center]  
运行界面如图，首次运行时间会比较久

![在这里插入图片描述][1347e1bf03ef432da70b2a1ce7434f58.png_pic_center]

大功告成

![在这里插入图片描述][5b1f977d647848f192a1367718b663ca.png_pic_center]

**希望本文对大家有帮助，上文若有不妥之处，欢迎指正**

**分享决定高度，学习拉开差距**

[ChatGLM-6B]: https://github.com/THUDM/ChatGLM3
[87415ce173ad4b2eb7bf276e3e9ce78a.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/e034393dab1342eea902c049758fb28d.png
[ChatGLM-6B-Models]: https://www.modelscope.cn/models/ZhipuAI/chatglm3-6b/summary
[85db69ee4f474630b699dc6038c5d06e.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/6d19ed141b8848dfacb8db77db4a01a0.png
[59315cad52c148bb9770fdde9e185a73.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/cb3fa314de194ccc86357d3cff35cd3b.png
[tdm-gcc]: https://jmeubank.github.io/tdm-gcc/
[f2825377444a49c79a6804007b1cfb8f.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/a9d63fcd3801496298133f46e6dab5fe.png
[58ada25236e64858bca441b9b936dffe.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/17abe101af584e4ab6b008417433153b.png
[40f9c5f318b44f2da9f6e966d4d50b12.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/60ee47fe709f4579b719c5101daefad9.png
[7fee85081923406199efddbae45f0f32.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/a1b462dd3c97406db1d084a72154e59c.png
[0e253478841b4270b06351a67e1c74ae.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/6324811386864a20884746073db83850.png
[d56e261fb5504d718cb1ddc89029e575.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/6d6e65e27cf24dd0ab0f2fc52da1b131.png
[4d8e016336504bd5afcc38a278204b38.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/48d27bdaa20f4a0583fddf5db3944ffe.png
[7466161dde0a4f49b6197d985a0617ed.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/0d6b391a7e9d4ed29a4b56010f7d6906.png
[dac02c5bbc8b42db9e8322a67a1047a9.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/22fb9ef10cd2458c9c8e65ffe03e46a5.png
[aa394d9557904e2580469c89b9db888a.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/7f6e5dd5e85c48268579157941950cb9.png
[Python_Anaconda_VS code_Win 10]: https://blog.csdn.net/qq_42078934/article/details/108973215
[ccb7b2ec0aa040b99723ccbac6615493.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/edc2ef8b1e6d43b3a17f6cab9c079a3d.png
[471b8346176a4d42926e1dd28f3bb0c4.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/d9a828edecf144779b440b32f3cac8b0.png
[e96e92a6a93146e2be32c53bf7c4ca58.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/3ecf6c7bd0904276b511ad2f985a3627.png
[7afe025b9b9245e49098658d4630a315.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/8770677802734e77a9b75b1b789f7ed8.png
[bdafd472283b4d84aa20c5c92afcb13c.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/edf00e8e3517487fa948efc017f9f60f.png
[64a4918f13374012a567127b89b57306.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/18d5f4e763d2450e84e59dad0a38dcdd.png
[31cd1ac29dcc431f99565d2af259f527.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/3fbe0eec96a54c2f9f5b682d1add3eed.png
[fec4c31b8ee74bc99a9721f21646db46.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/a563e777702346b396d6902e8e2a469a.png
[1347e1bf03ef432da70b2a1ce7434f58.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/cd203f0815f5443091d82530822aff80.png
[5b1f977d647848f192a1367718b663ca.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/08/4a921837b1014f049fc48f485e9e86c8.png