linux下使用bandersnatch搭建Python pypi本地源

偏执的太偏执、 2022-07-15 05:20 284阅读 0赞

最近需要在离线环境下安装python的包找了几个发现都不太适用最后选择了**[bandersnatch][]**这个也是官方推荐使用的，bandersnatch会定时抓取官方镜像，安装简单一次安装之后你就不用管了而且运行也非常稳定。感觉被欺骗了。看了很多教程别人说只有50G左右，我留了500G的空间，最后查资料才发现2013年是50G，2014年120G，我现在同步的是326G。同步了6天有点大，截止到2016年11月18日 大小为402G,最近几天没有增长

我是在docker下搭建的,如果在实体机器上搭建流程一样。  
可以直接pull docker pull becivells/pypimirror  
贴出我的Dockerfile

1.  bandersnatch.conf

[mirror]
    directory = /opt/pypi/
    master = https://pypi.python.org
    timeout = 10
    workers = 3
    hash-index = false
    stop-on-error = false
    delete-packages = true
    [statistics]
    access-log-pattern = /opt/tengine/log/*.pypi.python.org*access*

1.  nginx.conf

worker_processes  1;
    
    events {
        worker_connections  1024;
    }
    
    
    
    http {
        include       mime.types;
        default_type  application/octet-stream;
        autoindex on;  
        autoindex_exact_size on;  
        autoindex_localtime on; 
        sendfile        on;
        keepalive_timeout  65;
        server {
            listen       80;
            server_name  localhost;
            location / {
                root   /opt/pypi/web/;
            }
    
            #
            error_page   500 502 503 504  /50x.html;
            location = /50x.html {
                root   html;
            }
    
        }
    
    }
    daemon off;

1.  supervisord.conf

[supervisord]
    nodaemon=true
    [program:tengine]
    command=/opt/tengine/sbin/nginx
    user = root
    autostart = true
    autorestart=true
    redirect_stderr=true
    stdout_logfile=/dev/stdout
    stdout_logfile_maxbytes=0
    
    [program:bandersnatch]
    command=/usr/local/bin/bandersnatch mirror
    user = root
    autostart = true
    redirect_stderr=true
    stdout_logfile=/dev/stdout
    stdout_logfile_maxbytes=0

1.  Dockerfile

FROM ubuntu:latest
    MAINTAINER  python <becivells@gmail.com> 
    RUN apt-get -y update && apt-get install -y  python-pip  wget supervisor && pip install bandersnatch 
    RUN apt-get install -y zlib1g-dev gcc make libpcre3 libpcre3-dev openssl  libssl-dev
    
    RUN mkdir -p /opt/pypi/ &&\
        mkdir -p /var/log/supervisor &&\
        mkdir -p /var/tmp/nginx/client/ &&\
        mkdir -p /var/tmp/nginx/proxy/ &&\
        mkdir -p /var/tmp/nginx/fcgi/ &&\
        mkdir -p /var/tmp/nginx/uwsgi/ &&\
        mkdir -p /var/tmp/nginx/scgi/  
    
    ENV TENGINE_VERSION tengine-2.1.0
    
    RUN cd /tmp/ && wget  http://tengine.taobao.org/download/${TENGINE_VERSION}.tar.gz &&\
        tar -zxvf ${TENGINE_VERSION}.tar.gz -C /tmp/ && chmod -R 777 /tmp/${TENGINE_VERSION}
    
    #./configure
    RUN cd /tmp/${TENGINE_VERSION} && ./configure \
      --prefix=/opt/${ TENGINE_VERSION}/ \   --error-log-path=/var/log/nginx/error.log \   --http-log-path=/var/log/nginx/access.log \   --pid-path=/var/run/nginx/nginx.pid \   --lock-path=/var/lock/nginx.lock \   --with-http_ssl_module \   --with-http_flv_module \   --with-http_stub_status_module \   --with-http_gzip_static_module \   --http-client-body-temp-path=/var/tmp/nginx/client/ \   --http-proxy-temp-path=/var/tmp/nginx/proxy/ \   --http-fastcgi-temp-path=/var/tmp/nginx/fcgi/ \   --http-uwsgi-temp-path=/var/tmp/nginx/uwsgi \   --http-scgi-temp-path=/var/tmp/nginx/scgi \   --with-pcre &&\       cd /tmp/${TENGINE_VERSION} && make && make install && \
          rm -rf /tmp/* && apt-get autoclean && \
          ln -s /opt/${TENGINE_VERSION}/ /opt/tengine
    
    
    COPY supervisord.conf  /etc/supervisor/conf.d/supervisord.conf
    COPY bandersnatch.conf /etc/bandersnatch.conf
    COPY nginx.conf /opt/tengine/conf/nginx.conf
    
    EXPOSE 80
    VOLUME ["/opt/pypi/"]
    CMD ["/usr/bin/supervisord"]

将这几个文件放到同一目录下  
![create][]  
创建  
![create][create 1]

docker run build -t pymirror .

运行  
![run][]

docker run -d -p 80:80 -v /主机要存放的目录:/opt/pypi pymirror

2017-01-01更新  
增加了同步时间  
可以直接`docker pull becivells/bandersnatch`  
运行

docker run -d -p 80:80  -e "TIMESLEEP=600" -v /docker/pypi/:/opt/pypi/  becivells/bandersnatch

使用方法因为http不被建议使用，我们也没有证书可以在home目录下创建.pip文件夹

linux下在用户目录下创建.pip文件夹，在.pip文件夹中创建文件pip.conf

mkdir ~/.pip
    touch ~/.pip/pip.conf
    
    # root @ Y400 in ~ [23:43:33] 
    $ cat .pip/pip.conf 
    [global]
    index-url = http://ip/simple
    extra-index-url=https://pypi.mirrors.ustc.edu.cn/simple
    [install]
    trusted-host = ip

windows下  
在用户目录下例如C:\\Users\\user-admin\\ 创建pip文件夹创建一个文件pip.ini

[global]
    index-url = http://ip/simple
    extra-index-url=https://pypi.mirrors.ustc.edu.cn/simple
    [install]
    trusted-host = ip

或者  
在window的文件夹窗口输入 ： %APPDATA%  
创建文件夹pip在pip文件加下创建文件pip.ini

[global]
    index-url = http://ip/simple
    extra-index-url=https://pypi.mirrors.ustc.edu.cn/simple
    [install]
    trusted-host = ip

或者临时使用

pip install -i http://ip/simple --trusted-host 192.168.10.90 Django

国内pypi源

阿里云 http://mirrors.aliyun.com/pypi/simple/
    中国科技大学https://pypi.mirrors.ustc.edu.cn/simple/
    豆瓣(douban) http://pypi.douban.com/simple/
    清华大学 https://pypi.tuna.tsinghua.edu.cn/simple/

[bandersnatch]: https://pypi.python.org/pypi/bandersnatch
[create]: /images/20220715/a1387e66524645fe91b85ce4d964a2b3.png
[create 1]: /images/20220715/d831c79af42f4494a75d7af2540b742d.png
[run]: /images/20220715/c437acb2a2c14816a7d639178791120d.png