HAProxy

怼烎@ 2022-07-13 03:52 181阅读 0赞

## 1 简介 ##

HAProxy 是单线程，事件驱动架构。  
haproxy是一款非常的专业的全7层的反向代理负载均衡器，采用的是epoll机制，可以实现4层和7层的负载均衡，4层使用的是tcp模式可以模拟lvs，7层使用的是http模式可以模拟nginx，nginx和haproxy的处理速度都远不及lvs，因为他们是工作在用户空间的，而lvs是工作在内核空间的

在四层（tcp）实现负载均衡的软件：
    lvs------>重量级
    nginx------>轻量级，带缓存功能，正则表达式较灵活
    haproxy------>模拟四层转发，较灵活
    在七层（http）实现反向代理的软件：
    haproxy------>天生技能，全面支持七层代理，会话保持，标记，路径转移；
    nginx------>只在http协议和mail协议上功能比较好，性能与haproxy差不多；
    apache------>功能较差

## 2 haproxy的工作模型图 ##

![这里写图片描述][SouthEast]

当用户并发请求达到一定的数量时，使用haproxy进行负载均衡有明显的优势；而且haproxy还可以根据用户的cookies，根据调度算法，将用户一直定向分配到以前访问过的后端服务器上；为了提高网站访问速度，一般在haproxy的后端都要配置缓存服务器，可以是静态页面内容的缓存，也可以是动态网页内容的缓存，生产环境中有必要添加mysql的缓存。  
用户访问网站域名时，DNS解析到外网接口haproxy服务器上，haproxy将请求直接转发（tcp）至后方服务器，或者先分析用户请求，然后以客户端身份向后端服务器发出同样的请求（http），获得后方服务器返回的内容后重新封装，响应给客户端，此时haproxy实现一手端两家，中间翻译官的角色。

## 3 haproxy与各负载均衡器的区别？ ##

与nginx：同样工作在用户空间，nginx是一款轻量级，能实现缓存、webserver、邮件、负载均衡等功能，但nginx的许多功能都需要第三方的模块，而haproxy的转发能力比nginx有更强更灵活的定制性，可以运用splice实现0复制的转发，并且有更直观的图形化管理界面，不过通用性不如nginx，并无缓存功能  
与varnish：varnish是一款web缓存系统，  
与lvs：lvs是工作在内核空间上直接转发的，无缓存功能

衡量一个负载均衡器的性能：  
会话率，会话接收的速率，类似于并发数QPS,每秒的NEW状态的查询请求数，可以通过http的header来标记，实时有效并发，最能体现负载均衡性能的最主要的指标，通常预期应该至少在10倍以上的并发能力  
会话并发能力，类似活动连接数，即ESTABLISHED状态的会话，由于使用了keep-alive保持连接数以及链接复用等机制，因此活动连接数通常是并发数的很多倍，1:20  
数据处理速度，减去与客户端，与服务端建立连接和传输的时间，中间那部分时间的数据处理的速度  
调度算法  
roundrobin 动态，加权轮询，所谓动态就是可以实时生效，不用重启服务，但是连接数受限，最多支持4128  
static-rr 静态轮询，需重启服务  
leastconn 动态，根据后端主机的负载数量进行调度  
source 类似源地址hash，可以指定hash-type ，有map-based(取膜法，静态), consistent(一致性哈希，动态)  
uri 类似于DH算法，目标地址哈希，可以指定hash-type ，有map-based(取膜法，静态), consistent(一致性哈希，动态)  
hdr():根据请求报文中指定的header(User-agent,referer,hostname,cookie)进行调度，把指定的header的值做hash计算；可根据header首部来进行调度，非常强大，比如根据User-Agent浏览器类型来进行调度，可以指定hash-type ，有map-based(取膜法，静态), consistent(一致性哈希，动态)

## 4 Companion products and alternatives\*\* ##

HAProxy integrates fairly well with certain products listed below, which is why  
they are mentionned here even if not directly related to HAProxy.  
4.1. Apache HTTP server  
Apache is the de-facto standard HTTP server. It’s a very complete and modular  
project supporting both file serving and dynamic contents. It can serve as a  
frontend for some application servers. In can even proxy requests and cache  
responses. In all of these use cases, a front load balancer is commonly needed.  
Apache can work in various modes, certain being heavier than other ones. Certain  
modules still require the heavier pre-forked model and will prevent Apache from  
scaling well with a high number of connections. In this case HAProxy can provide  
a tremendous help by enforcing the per-server connection limits to a safe value  
and will significantly speed up the server and preserve its resources that will  
be better used by the application.

Apache can extract the client’s address from the X-Forwarded-For header by using  
the “mod\_rpaf” extension. HAProxy will automatically feed this header when  
“option forwardfor” is specified in its configuration. HAProxy may also offer a  
nice protection to Apache when exposed to the internet, where it will better  
resist to a wide number of types of DoS.  
4.2. NGINX  
NGINX is the second de-facto standard HTTP server. Just like Apache, it covers a  
wide range of features. NGINX is built on a similar model as HAProxy so it has  
no problem dealing with tens of thousands of concurrent connections. When used  
as a gateway to some applications (eg: using the included PHP FPM), it can often  
be beneficial to set up some frontend connection limiting to reduce the load  
on the PHP application. HAProxy will clearly be useful there both as a regular  
load balancer and as the traffic regulator to speed up PHP by decongestionning  
it. Also since both products use very little CPU thanks to their event-driven  
architecture, it’s often easy to install both of them on the same system. NGINX  
implements HAProxy’s PROXY protocol, thus it is easy for HAProxy to pass the  
client’s connection information to NGINX so that the application gets all the  
relevant information. Some benchmarks have also shown that for large static  
file serving, implementing consistent hash on HAProxy in front of NGINX can be  
beneficial by optimizing the OS’ cache hit ratio, which is basically multiplied  
by the number of server nodes.  
4.3. Varnish  
Varnish is a smart caching reverse-proxy, probably best described as a web  
application accelerator. Varnish doesn’t implement SSL/TLS and wants to dedicate  
all of its CPU cycles to what it does best. Varnish also implements HAProxy’s  
PROXY protocol so that HAProxy can very easily be deployed in front of Varnish  
as an SSL offloader as well as a load balancer and pass it all relevant client  
information. Also, Varnish naturally supports decompression from the cache when  
a server has provided a compressed object, but doesn’t compress however. HAProxy  
can then be used to compress outgoing data when backend servers do not implement  
compression, though it’s rarely a good idea to compress on the load balancer  
unless the traffic is low.

When building large caching farms across multiple nodes, HAProxy can make use of  
consistent URL hashing to intelligently distribute the load to the caching nodes  
and avoid cache duplication, resulting in a total cache size which is the sum of  
all caching nodes.  
4.4. Alternatives  
Linux Virtual Server (LVS or IPVS) is the layer 4 load balancer included within  
the Linux kernel. It works at the packet level and handles TCP and UDP. In most  
cases it’s more a complement than an alternative since it doesn’t have layer 7  
knowledge at all.

Pound is another well-known load balancer. It’s much simpler and has much less  
features than HAProxy but for many very basic setups both can be used. Its  
author has always focused on code auditability first and wants to maintain the  
set of features low. Its thread-based architecture scales less well with high  
connection counts, but it’s a good product.

Pen is a quite light load balancer. It supports SSL, maintains persistence using  
a fixed-size table of its clients’ IP addresses. It supports a packet-oriented  
mode allowing it to support direct server return and UDP to some extents. It is  
meant for small loads (the persistence table only has 2048 entries).

NGINX can do some load balancing to some extents, though it’s clearly not its  
primary function. Production traffic is used to detect server failures, the  
load balancing algorithms are more limited, and the stickiness is very limited.  
But it can make sense in some simple deployment scenarios where it is already  
present. The good thing is that since it integrates very well with HAProxy,  
there’s nothing wrong with adding HAProxy later when its limits have been faced.

Varnish also does some load balancing of its backend servers and does support  
real health checks. It doesn’t implement stickiness however, so just like with  
NGINX, as long as stickiness is not needed that can be enough to start with.  
And similarly, since HAProxy and Varnish integrate so well together, it’s easy  
to add it later into the mix to complement the feature set.

## 5 安装 ##

在ubuntu下安装

root@haproxy:/opt# apt-cache search haproxy
    haproxy - fast and reliable load balancing reverse proxy
    root@haproxy:/opt# apt-get install haproxy

下载安装包：[http://www.haproxy.org/download/1.7/src/haproxy-1.7.1.tar.gz][http_www.haproxy.org_download_1.7_src_haproxy-1.7.1.tar.gz]

## 6 实例 ##

在openstack环境中，部署lbaas后，会部署haproxy

root@network:~# ps -ef | grep haproxy
    nobody   13378     1  0 Jul13 ?        00:04:57 haproxy -f /var/lib/neutron/lbaas/v2/628b6622-154b-4342-8be1-cd645dbb601e/haproxy.conf -p /var/lib/neutron/lbaas/v2/628b6622-154b-4342-8be1-cd645dbb601e/haproxy.pid -sf 13371

查看配置文件：/var/lib/neutron/lbaas/v2/628b6622-154b-4342-8be1-cd645dbb601e/haproxy.conf

global #全局配置，基本不需要修改
            log /dev/log    local0
            log /dev/log    local1 notice
            chroot /var/lib/haproxy
            stats socket /run/haproxy/admin.sock mode 660 level admin
            stats timeout 30s
            user haproxy
            group haproxy
            daemon
    
            # Default SSL material locations
            ca-base /etc/ssl/certs
            crt-base /etc/ssl/private
    
            # Default ciphers to use on SSL-enabled listening sockets.
            # For more information, see ciphers(1SSL).
            ssl-default-bind-ciphers kEECDH+aRSA+AES:kRSA+AES:+AES256:RC4-SHA:!kEDH:!LOW:!EXP:!MD5:!aNULL:!eNULL
    
    defaults
            log     global
            mode    http
            option  httplog
            option  dontlognull
            timeout connect 5000
            timeout client  50000
            timeout server  50000
            errorfile 400 /etc/haproxy/errors/400.http
            errorfile 403 /etc/haproxy/errors/403.http
            errorfile 408 /etc/haproxy/errors/408.http
            errorfile 500 /etc/haproxy/errors/500.http
            errorfile 502 /etc/haproxy/errors/502.http
            errorfile 503 /etc/haproxy/errors/503.http
            errorfile 504 /etc/haproxy/errors/504.http
    
    frontend localnodes #where HAProxy listens to connections
        bind *:80              #HAProxy 在所有网卡的 80 端口上监听 HTTP 请求
        mode http              #监听 HTTP 请求，这时它作为一个七层负载均衡器
        default_backend nodes  #所使用的后端服务器
    
    backend nodes #Where HAPoxy sends incoming connections
        mode http          #转发 HTTP 包给后端服务器
        balance roundrobin #分发算法
        option forwardfor  #在 HTTP 头中添加 X-Forwarded-For 头，使得后端服务器可以获取原始请求的来源地址
        http-request set-header X-Forwarded-Port %[dst_port] #在 HTTP 头中添加 X-Forwarded-Port 头从而使得后端服务器可以知道原始的 HTTP Port
        http-request add-header X-Forwarded-Proto https if { ssl_fc } #在使用 SSL 时添加 X-Forwarded-Proto头
        option httpchk HEAD / HTTP/1.1\r\nHost:localhost #使用 health check 来检查后端服务器的可达性
        server web01 127.0.0.1:9000 check #后端服务器。”check“表示做 health check。
        server web02 127.0.0.1:9001 check
        server web03 127.0.0.1:9002 check
    
    listen stats *:1936 #用于监控 HAProxy
        stats enable
        stats uri /
        stats hide-version
        stats auth someuser:password

单独部署haproxy时，默认配置：

root@haproxy:/opt# cat /etc/haproxy/haproxy.cfg 
    global
        log /dev/log    local0
        log /dev/log    local1 notice
        chroot /var/lib/haproxy
        user haproxy
        group haproxy
        daemon
    
    defaults
        log global
        mode    http
        option  dontlognull
            contimeout 5000
            clitimeout 50000
            srvtimeout 50000
        errorfile 400 /etc/haproxy/errors/400.http
        errorfile 403 /etc/haproxy/errors/403.http
        errorfile 408 /etc/haproxy/errors/408.http
        errorfile 500 /etc/haproxy/errors/500.http
        errorfile 502 /etc/haproxy/errors/502.http
        errorfile 503 /etc/haproxy/errors/503.http
        errorfile 504 /etc/haproxy/errors/504.http
    frontend loadbalancer
        bind *:80
        default_backend loadbalancer-servers
    backend loadbalancer-servers
        balance leastconn
        server loadbalancer-server-2 192.168.16.207:80
        server loadbalancer-server-2 192.168.16.232:80

过程：  
[http://blog.csdn.net/dengyuelin/article/details/50778886][http_blog.csdn.net_dengyuelin_article_details_50778886]  
[http://www.linuxidc.com/Linux/2015-01/112487.htm][http_www.linuxidc.com_Linux_2015-01_112487.htm]  
负载均衡器虚拟化

其它：  
HAProxy is a TCP proxy, not a router.

# netstat -ltnp #

Active Internet connections (only servers)  
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name  
tcp 0 0 0.0.0.0:22 0.0.0.0:\* LISTEN 1629/sshd  
tcp 0 0 0.0.0.0:80 0.0.0.0:\* LISTEN 2847/haproxy  
tcp 0 0 0.0.0.0:443 0.0.0.0:\* LISTEN 2847/haproxy

root@haproxy:/opt/guestagent/guestagent/haproxy# netstat -antpl
    Active Internet connections (servers and established)
    Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
    tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1295/haproxy    
    tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1295/haproxy    
    tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1295/haproxy    
    tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1295/haproxy    
    tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1295/haproxy    
    tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      998/sshd        
    tcp        0      0 0.0.0.0:8088            0.0.0.0:*               LISTEN      1295/haproxy

参考：  
1 [http://www.linuxidc.com/Linux/2015-06/118968.htm][http_www.linuxidc.com_Linux_2015-06_118968.htm]  
2 [http://cbonte.github.io/haproxy-dconv/1.7/intro.html][http_cbonte.github.io_haproxy-dconv_1.7_intro.html]

[SouthEast]: /images/20220713/ed392b2b904140db867bba9301639bd3.png
[http_www.haproxy.org_download_1.7_src_haproxy-1.7.1.tar.gz]: http://www.haproxy.org/download/1.7/src/haproxy-1.7.1.tar.gz
[http_blog.csdn.net_dengyuelin_article_details_50778886]: http://blog.csdn.net/dengyuelin/article/details/50778886
[http_www.linuxidc.com_Linux_2015-01_112487.htm]: http://www.linuxidc.com/Linux/2015-01/112487.htm
[http_www.linuxidc.com_Linux_2015-06_118968.htm]: http://www.linuxidc.com/Linux/2015-06/118968.htm
[http_cbonte.github.io_haproxy-dconv_1.7_intro.html]: http://cbonte.github.io/haproxy-dconv/1.7/intro.html