搭建与使用fastdfs分布式文件服务器

Dear 丶 2022-03-06 12:10 290阅读 0赞

**一、环境**  
系统：Centos7.3  
fastdfs：到 [https://github.com/happyfish100][https_github.com_happyfish100] 下载，都下载最新的：  
fastdfs-master：下载地址：[https://github.com/happyfish100/fastdfs.git][https_github.com_happyfish100_fastdfs.git]  
libfastcommon-master：下载地址：[https://github.com/happyfish100/libfastcommon.git][https_github.com_happyfish100_libfastcommon.git]  
fastdfs-nginx-module-master：下载地址：[https://github.com/happyfish100/fastdfs-nginx-module.git][https_github.com_happyfish100_fastdfs-nginx-module.git]  
避免版本问题  
Linux终端工具：xshell、xftp  
Linux ip：192.168.1.186 外网IP  
**二、准备**  
安装zip、unzip与相关依赖

yum install -y unzip zip

安装perl

yum -y install perl

安装gcc

yum install gcc-c++

**三、安装fastdfs**  
3.1、解压libfastcommon-master.zip

[root@root fastdfs-master]# cd /opt
    
    [root@root opt]# unzip -o libfastcommon-master.zip -d /usr/local

安装libfastcommon

[root@root opt]# cd /usr/local/libfastcommon-master/
    
    [root@root libfastcommon-master]# ./make.sh
    
    [root@root libfastcommon-master]# ./make.sh install

3.2、解压fastdfs-master.zip

[root@root opt]# unzip -o fastdfs-master.zip -d /usr/local

安装fastdfs

[root@root opt]# cd /usr/local/fastdfs-master
    
    [root@root fastdfs-master]# ./make.sh
    
    [root@root fastdfs-master]# ./make.sh install

将fastdfs安装目录下的conf下的文件拷贝到/etc/fdfs/下

[root@root fastdfs-master]# cp -r conf/* /etc/fdfs/

自此fastdfs安装完成了，接下来配置trackerd和storaged，并启动它们。

fastdfs执行命令

[root@root fdfs]# ll /usr/bin/fdfs*

配置并启动trackerd、不改也可以，但是要保证/home/yuqing/fastdfs路径存在

[root@root fdfs]# cd /etc/fdfs/
    
    [root@root fdfs]# vi tracker.conf
    
    将base_path=/home/yuqing/fastdfs改成base_path=/data/fastdfs

3.3、创建trackerd数据、日志目录、 启动trackerd

[root@root fdfs]# mkdir -p /data/fastdfs
     
    [root@root fdfs]# /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart

其实也可以查看日志：/data/fastdfs/logs/trackerd.log，来判断trackerd是否正常启动起来。  
3.4、配置并启动storaged/ 修改storage.conf

[root@root fdfs]# cd /etc/fdfs/
    
    [root@root fdfs]# vi storage.conf
    
    base_path=/home/yuqing/fastdfs改为：base_path=/data/fastdfs
    
    store_path0=/home/yuqing/fastdfs改为：store_path0=/data/fastdfs/storage
    
    tracker_server=192.168.209.121:22122改为 tracker_server=192.168.1.168:22122，这个ip改成自己的

创建storaged数据、日志目录、 启动storaged

[root@root fdfs]# mkdir -p /data/fastdfs/storage
     
     [root@root fdfs]# /usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart

其实也可以查看日志：/data/fastdfs/logs/storaged.log来判断storaged是否正常启动起来。

四、上传图片测试  
从https://github.com/happyfish100/fastdfs-client-java下载源码，我下载的是zip包，解压后目录如下图：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70]

maven本地安装：mvn clean install
    当然也可以用ant构建：ant clean package

五、项目代码：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 1]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 2]  
fdfs\_client\_mine.conf

connect_timeout =2
    network_timeout =30
    charset =UTF-8
    http.tracker_http_port =8080  #这个是trackerd的http_server_prot=8080端口
    http.anti_steal_token =no
    http.secret_key =FastDFS1234567890
    tracker_server =192.168.1.168:22122

FastdfsClientTest.java

package com.citydo.fastdfs;
    
    import org.csource.common.NameValuePair;
    import org.csource.fastdfs.*;
    import org.junit.Test;
    import org.springframework.cache.annotation.Cacheable;
    
    import java.io.FileNotFoundException;
    import java.io.IOException;
    import java.util.logging.Logger;
    
    public class FastdfsClientTest {
    
        //客户端配置文件
        public String conf_filename = "fdfs_client_mine.conf";
        //本地文件，要上传的文件
        public String local_filename = "D:\\下载\\JVM.png";
    
        //访问地址：http://192.168.1.168:8888/group1/M00/00/00/wKgBU1yO-gGAVuoFABJ7r6REMUw182.png
    
        //上传文件
        @Test
        public void testUpload() {
    
            for(int i=0;i<5;i++){
    
                try {
                    ClientGlobal.init(conf_filename);
                    TrackerClient tracker = new TrackerClient();
                    TrackerServer trackerServer = tracker.getConnection();
                    StorageServer storageServer = null;
    
                    StorageClient storageClient = new StorageClient(trackerServer, storageServer);
                    NameValuePair nvp [] = new NameValuePair[]{
                            new NameValuePair("item_id", "100010"),
                            new NameValuePair("width", "80"),
                            new NameValuePair("height", "90")
                    };
                    String fileIds[] = storageClient.upload_file(local_filename, null,
                            nvp);
    
                    System.out.println(fileIds.length);
                    System.out.println("组名：" + fileIds[0]);
                    System.out.println("路径: " + fileIds[1]);
    
                } catch (FileNotFoundException e) {
                    e.printStackTrace();
                } catch (IOException e) {
                    e.printStackTrace();
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
        }
    
    }

六、FastDFS 和nginx整合  
6.1、fastdfs-nginx-module安装

解压
    
    [root@root 00]# cd /opt
    
    [root@root opt]# unzip -o fastdfs-nginx-module-master.zip -d /usr/local
    
    拷贝配置文件
    
    [root@root opt]# cd /usr/local/fastdfs-nginx-module-master/src
    
    [root@root src]# cp mod_fastdfs.conf /etc/fdfs/
    
    编辑配置文件
    
    [root@root src]# cd /etc/fdfs/
    
    [root@root fdfs]# vi mod_fastdfs.conf
    
    base_path=/tmp改成：base_path=/data/fastdfs
    
    tracker_server=tracker:22122改成：tracker_server=192.168.1.168:22122
    
    url_have_group_name = false改成：url_have_group_name = true；#url中包含group名称
    
    store_path0=/home/yuqing/fastdfs改成：store_path0=/data/fastdfs/storage

6.2、nginx安装

nginx依赖包安装
    
    [root@root fdfs]# cd /opt
    
    [root@root opt]# yum -y install zlib zlib-devel openssl openssl--devel pcre pcre-devel
    
    解压nginx
    
    [root@root opt]# tar -zxvf nginx-1.13.12.tar.gz
    
    安装nginx并添加fastdfs模块
    
    [root@root opt]# cd nginx-1.13.12
    
    [root@root nginx-1.13.12]# ./configure --prefix=/usr/local/nginx --add-module=/usr/local/fastdfs-nginx-module-master/src
    
    [root@root nginx-1.13.12]# make
    
    [root@root nginx-1.13.12]# make install
    
    检查nginx模块
    
    [root@root nginx-1.13.12]# cd /usr/local/nginx/sbin/
    
    [root@root sbin]# ./nginx -V

已经把fastdfs模块添加进去了。

七、配置nginx、启动nginx

vim /etc/fdfs/mod_fastdfs.conf
    #需要修改的内容如下
    tracker_server=192.168.1.168:22122  #tracker服务器IP和端口
    url_have_group_name=true
    store_path0=/home/dfs
    #配置nginx.config
    vim /usr/local/nginx/conf/nginx.conf
    #添加如下配置
    server {
        listen       8888;    ## 该端口为storage.conf中的http.server_port相同
        server_name  localhost;
        location ~/group[0-9]/M00/ {
            ngx_fastdfs_module;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
        root   html;
        }
    }
    #测试下载，用外部浏览器访问刚才已传过的nginx安装包,引用返回的ID
    http://192.168.1.168:8888/group1/M00/00/00/wKgAQ1pysxmAaqhAAA76tz-dVgg.tar.gz
    #弹出下载单机部署全部跑通

[root@root conf]# cd /usr/local/nginx/sbin/
    
    [root@root sbin]# ./nginx -c /usr/local/nginx/conf/nginx.conf

八、访问路径  
[http://192.168.1.168/group1/M00/00/00/wKgBz1salX-ATR4PAABHO7x65CM553.jpg][http_192.168.1.168_group1_M00_00_00_wKgBz1salX-ATR4PAABHO7x65CM553.jpg]  
九、配置详解  
tracker配置

#服务器ip为 192.168.1.168
    #我建议用ftp下载下来这些文件 本地修改
    vim /etc/fdfs/tracker.conf
    #需要修改的内容如下
    port=22122  # tracker服务器端口（默认22122,一般不修改）
    base_path=/home/dfs  # 存储日志和数据的根目录

storage配置

vim /etc/fdfs/storage.conf
    #需要修改的内容如下
    port=23000  # storage服务端口（默认23000,一般不修改）
    base_path=/home/dfs  # 数据和日志文件存储根目录
    store_path0=/home/dfs  # 第一个存储目录
    tracker_server=192.168.1.168:22122  # tracker服务器IP和端口
    http.server_port=8888  # http访问文件的端口(默认8888,看情况修改,和nginx中保持一致)

client测试

vim /etc/fdfs/client.conf
    #需要修改的内容如下
    base_path=/home/dfs
    tracker_server=192.168.1.168:22122    #tracker服务器IP和端口
    #保存后测试,返回ID表示成功 如：group1/M00/00/00/xx.tar.gz
    fdfs_upload_file /etc/fdfs/client.conf /usr/local/src/nginx-1.15.4.tar.gz

如果是阿里云服务器开放23000、22122、8888、8080至少四个端口。  
如果是虚拟机一定关闭防火墙。

参考：[https://github.com/happyfish100/fastdfs/wiki\#tracker][https_github.com_happyfish100_fastdfs_wiki_tracker]  
参考：[https://www.cnblogs.com/youzhibing/p/9160690.html][https_www.cnblogs.com_youzhibing_p_9160690.html]

[https_github.com_happyfish100]: https://github.com/happyfish100
[https_github.com_happyfish100_fastdfs.git]: https://github.com/happyfish100/fastdfs.git
[https_github.com_happyfish100_libfastcommon.git]: https://github.com/happyfish100/libfastcommon.git
[https_github.com_happyfish100_fastdfs-nginx-module.git]: https://github.com/happyfish100/fastdfs-nginx-module.git
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70]: /images/20220303/f7ea5e54a7134d1c869d12c278c5988e.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 1]: /images/20220303/c5fdf94237834de2a1c937bf35d7da72.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMyNDQ3MzAx_size_16_color_FFFFFF_t_70 2]: /images/20220303/1211a622e3e641e0a781d3346fa3272e.png
[http_192.168.1.168_group1_M00_00_00_wKgBz1salX-ATR4PAABHO7x65CM553.jpg]: http://192.168.1.168/group1/M00/00/00/wKgBz1salX-ATR4PAABHO7x65CM553.jpg
[https_github.com_happyfish100_fastdfs_wiki_tracker]: https://github.com/happyfish100/fastdfs/wiki#tracker
[https_www.cnblogs.com_youzhibing_p_9160690.html]: https://www.cnblogs.com/youzhibing/p/9160690.html