Golang http之server源码详解
http 是典型的 C/S 架构,客户端向服务端发送请求(request),服务端做出应答(response)。HTTP server–简而言之就是一个支持http协议的服务,http是一个相对简单的请求—响应的协议,通常是运行在TCP连接之上, 通过客户端发送请求到服务端,获取服务端的响应。
我们先看一个简单的http server例子
package main
import (
"io"
"net/http"
"log"
)
// hello world, the web server
func HelloServer(w http.ResponseWriter, req *http.Request) {
io.WriteString(w, "hello, world!\n")
}
func main() {
//路由注册
http.HandleFunc("/hello", HelloServer)
//开启一个http服务
log.Fatal(http.ListenAndServe(":8086", nil))
}
当服务器启动后,浏览器访问:http://127.0.0.1:8086/hello
网页上会显示:
以上就是http 服务的一个简单demo,下面我们主要从源码来分析http server
源码分析
一、 常用概念
1.Handler接口
type Handler interface {
ServeHTTP(ResponseWriter, *Request)
}
作用:Handler封装了处理客户端的请求逻辑及创建返回响应的逻辑,一个http server需要实现Handler接口。
2.Server结构体
//Server定义一个HTTP 服务
type Server struct {
//监听的TCP地址
Addr string // TCP address to listen on, ":http" if empty
//要调用的处理函数,如果为nil,则默认用http.DefaultServeMux
Handler Handler // handler to invoke, http.DefaultServeMux if nil
// TLSConfig optionally provides a TLS configuration for use
// by ServeTLS and ListenAndServeTLS. Note that this value is
// cloned by ServeTLS and ListenAndServeTLS, so it's not
// possible to modify the configuration with methods like
// tls.Config.SetSessionTicketKeys. To use
// SetSessionTicketKeys, use Server.Serve with a TLS Listener
// instead.
/*
TLSConfig可以选择提供TLS 配置 用于使用ServeTLS 和 ListenAndServeTLS
*/
TLSConfig *tls.Config
// ReadTimeout is the maximum duration for reading the entire
// request, including the body.
//
// Because ReadTimeout does not let Handlers make per-request
// decisions on each request body's acceptable deadline or
// upload rate, most users will prefer to use
// ReadHeaderTimeout. It is valid to use them both.
//ReadTimeout读取请求的最长时间,包括正文
ReadTimeout time.Duration
// ReadHeaderTimeout is the amount of time allowed to read
// request headers. The connection's read deadline is reset
// after reading the headers and the Handler can decide what
// is considered too slow for the body.
//ReadHeaderTimeout是允许读取请求头的最长时间
//读完请求头后会重置超时间,并且Handler可以判断太慢的body
ReadHeaderTimeout time.Duration
// WriteTimeout is the maximum duration before timing out
// writes of the response. It is reset whenever a new
// request's header is read. Like ReadTimeout, it does not
// let Handlers make decisions on a per-request basis.
//WriteTimeout是写入响应之前的最大持续时间。 只要读取新请求的标头,它就会重置。 与ReadTimeout一样,它不允许处理程序根据请求做出决策。
WriteTimeout time.Duration
// IdleTimeout is the maximum amount of time to wait for the
// next request when keep-alives are enabled. If IdleTimeout
// is zero, the value of ReadTimeout is used. If both are
// zero, ReadHeaderTimeout is used.
//当开启keep-alives时,IdleTimeout是等待下个请求的最长时间,如果IdleTimeout是0,则使用ReadTimeout,如果ReadTimeout也是0,则使用ReadHeaderTimeout
IdleTimeout time.Duration
// MaxHeaderBytes controls the maximum number of bytes the
// server will read parsing the request header's keys and
// values, including the request line. It does not limit the
// size of the request body.
// If zero, DefaultMaxHeaderBytes is used.
//请求头的最大字节数,如果是0,则使用默认值DefaultMaxHeaderBytes
MaxHeaderBytes int
// TLSNextProto optionally specifies a function to take over
// ownership of the provided TLS connection when an NPN/ALPN
// protocol upgrade has occurred. The map key is the protocol
// name negotiated. The Handler argument should be used to
// handle HTTP requests and will initialize the Request's TLS
// and RemoteAddr if not already set. The connection is
// automatically closed when the function returns.
// If TLSNextProto is not nil, HTTP/2 support is not enabled
// automatically.
/*
TLSNextProto可选地指定在发生NPN / ALPN协议升级时接管所提供的TLS连接的所有权的函数。 映射键是协商的协议名称。
*/
TLSNextProto map[string]func(*Server, *tls.Conn, Handler)
// ConnState specifies an optional callback function that is
// called when a client connection changes state. See the
// ConnState type and associated constants for details.
/*
ConnState指定在客户端连接更改状态时调用的可选回调函数。
*/
ConnState func(net.Conn, ConnState)
// ErrorLog specifies an optional logger for errors accepting
// connections, unexpected behavior from handlers, and
// underlying FileSystem errors.
// If nil, logging is done via the log package's standard logger.
ErrorLog *log.Logger
disableKeepAlives int32 // accessed atomically.原子访问
inShutdown int32 // accessed atomically (non-zero means we're in Shutdown) 原子访问,非零意味着已关闭
nextProtoOnce sync.Once // guards setupHTTP2_* init
nextProtoErr error // result of http2.ConfigureServer if used 如果使用http2.ConfigureServer的结果
mu sync.Mutex
listeners map[net.Listener]struct{} //记录所有监听net.Listener信息
activeConn map[*conn]struct{} //记录所有处于active状态的连接
doneChan chan struct{}
onShutdown []func()
}
Server定义了一个HTTP服务,里面包含了监听的TCP地址Addr,处理客户端请求的Handler,以及其他相关的配置信息
3.conn结构体
// A conn represents the server side of an HTTP connection.
//conn表示HTTP连接的服务器端。
type conn struct {
// server is the server on which the connection arrived.
// Immutable; never nil.
//server是连接到达的服务器,不能改变,不能为空。
server *Server
// cancelCtx cancels the connection-level context.
//用于取消连接的上下文
cancelCtx context.CancelFunc
// rwc is the underlying network connection.
// This is never wrapped by other types and is the value given out
// to CloseNotifier callers. It is usually of type *net.TCPConn or
// *tls.Conn.
//rwc是网络底层连接,不能被其他类型包装
//常用类型是 *net.TCPConn *tls.Conn
rwc net.Conn
// remoteAddr is rwc.RemoteAddr().String(). It is not populated synchronously
// inside the Listener's Accept goroutine, as some implementations block.
// It is populated immediately inside the (*conn).serve goroutine.
// This is the value of a Handler's (*Request).RemoteAddr.
//rwc.RemoteAddr().String() 客户端连接地址
/*
它不会在Listener的Accept goroutine中同步填充,因为某些实现会阻塞。 它立即填充在(* conn).serve goroutine中。
这是Handler's(* Request).RemoteAddr的值。
*/
remoteAddr string
// tlsState is the TLS connection state when using TLS.
// nil means not TLS.
//如果使用的是TLS,则表示TLS连接状态,nil表示没有TLS
tlsState *tls.ConnectionState
// werr is set to the first write error to rwc.
// It is set via checkConnErrorWriter{w}, where bufw writes.
//werr设置为rwc的第一个写入错误。 它通过checkConnErrorWriter {w}设置,其中bufw写入。
werr error
// r is bufr's read source. It's a wrapper around rwc that provides
// io.LimitedReader-style limiting (while reading request headers)
// and functionality to support CloseNotifier. See *connReader docs.
//是数据源用于bufr读取数据,它包装了rwc,提供o.LimitedReader-style
r *connReader
// bufr reads from r.
//从r中读取数据
bufr *bufio.Reader
// bufw writes to checkConnErrorWriter{c}, which populates werr on error.
//给 checkConnErrorWriter{c} 写
bufw *bufio.Writer
// lastMethod is the method of the most recent request
// on this connection, if any.
lastMethod string
curReq atomic.Value // 记录Request个数
curState atomic.Value // 记录连接状态 ConnState
// mu guards hijackedv
mu sync.Mutex
// hijackedv is whether this connection has been hijacked
// by a Handler with the Hijacker interface.
// It is guarded by mu.
//判断Handler中是否实现Hijacker接口
hijackedv bool
}
conn表示HTTP连接,它将底层的连接如:*net.TCPConn和 *tls.Conn进行封装
4.response结构体
type response struct {
conn *conn //网络连接
req *Request // 客户端的请求信息
reqBody io.ReadCloser
cancelCtx context.CancelFunc // when ServeHTTP exits 当ServeHTTP退出时调用
wroteHeader bool // reply header has been (logically) written header是否已经写入
wroteContinue bool // 100 Continue response was written 100 Continue 响应已写入
wants10KeepAlive bool // HTTP/1.0 w/ Connection "keep-alive" 是否保持长连接
wantsClose bool // HTTP request has Connection "close" 客户端想要断开连接
w *bufio.Writer // buffers output in chunks to chunkWriter 将缓冲区块输出到chunkWriter
cw chunkWriter
// handlerHeader is the Header that Handlers get access to,
// which may be retained and mutated even after WriteHeader.
// handlerHeader is copied into cw.header at WriteHeader
// time, and privately mutated thereafter.
handlerHeader Header
calledHeader bool // handler accessed handlerHeader via Header
written int64 // number of bytes written in body 写入到body中的字节数
contentLength int64 // explicitly-declared Content-Length; or -1 声明的Content-Length
status int // status code passed to WriteHeader 传递给WriteHeader的状态码
// close connection after this reply. set on request and
// updated after response from handler if there's a
// "Connection: keep-alive" response header and a
// Content-Length.
closeAfterReply bool //判断在响应后是否关闭连接
// requestBodyLimitHit is set by requestTooLarge when
// maxBytesReader hits its max size. It is checked in
// WriteHeader, to make sure we don't consume the
// remaining request body to try to advance to the next HTTP
// request. Instead, when this is set, we stop reading
// subsequent requests on this connection and stop reading
// input from it.
requestBodyLimitHit bool
// trailers are the headers to be sent after the handler
// finishes writing the body. This field is initialized from
// the Trailer response header when the response header is
// written.
trailers []string
handlerDone atomicBool // set true when the handler exits
// Buffers for Date, Content-Length, and status code
dateBuf [len(TimeFormat)]byte
clenBuf [10]byte
statusBuf [3]byte
// closeNotifyCh is the channel returned by CloseNotify.
// TODO(bradfitz): this is currently (for Go 1.8) always
// non-nil. Make this lazily-created again as it used to be?
closeNotifyCh chan bool
didCloseNotify int32 // atomic (only 0->1 winner should send)
}
#
二、http服务创建过程
我们根据上面提到过的demo来分析http服务创建的过程
// hello world, the web server
func HelloServer(w http.ResponseWriter, req *http.Request) {
io.WriteString(w, "hello, world!\n")
}
func main() {
//路由注册
http.HandleFunc("/hello", HelloServer)
//开启一个http服务
log.Fatal(http.ListenAndServe(":8086", nil))
}
大致流程如下:
第一步:路由注册
// HandleFunc registers the handler function for the given pattern
// in the DefaultServeMux.
// The documentation for ServeMux explains how patterns are matched.
func HandleFunc(pattern string, handler func(ResponseWriter, *Request)) {
DefaultServeMux.HandleFunc(pattern, handler)
}
这里使用http.HandleFunc是使用http包自带的DefaultServeMux来进行服务的路由注册与管理。DefaultServeMux是全局变量,DefaultServeMux的定义如下:
// DefaultServeMux is the default ServeMux used by Serve.
var DefaultServeMux = &defaultServeMux
var defaultServeMux ServeMux
/*
HandleFunc根据给定的pattern注册handler函数
路由注册
*/
func (mux *ServeMux) HandleFunc(pattern string, handler func(ResponseWriter, *Request)) {
mux.Handle(pattern, HandlerFunc(handler))
}
/*
Handle 根据给定的pattern注册 handler,如果pattern已经注册过,则panic
*/
func (mux *ServeMux) Handle(pattern string, handler Handler) {
mux.mu.Lock()
defer mux.mu.Unlock()
/*
如果pattern为"",或者handler为nil,或者该pattern已经注册,则panic
*/
if pattern == "" {
panic("http: invalid pattern")
}
if handler == nil {
panic("http: nil handler")
}
if _, exist := mux.m[pattern]; exist {
panic("http: multiple registrations for " + pattern)
}
if mux.m == nil { //如果m为nil,则初始化
mux.m = make(map[string]muxEntry)
}
//将pattern和handler注册到m中
mux.m[pattern] = muxEntry{h: handler, pattern: pattern}
if pattern[0] != '/' { //pattern第一个字符为'/',则hosts为true。说明该pattern包含主机名
mux.hosts = true
}
}
从以上代码可以看到路由的注册最后注册到了ServeMux结构体的m字段中,m是map格式储存了pattern和Handler的对应关系。ServeMux结构体如下:
type ServeMux struct {
mu sync.RWMutex
m map[string]muxEntry //path和muxEntry的对应关系
hosts bool // pattern是否包含主机名
}
type muxEntry struct {
h Handler
pattern string
}
ServeMux是HTTP请求的多路复用路由器,负责接收http handler的注册和路由解析。将所有的路径(pattern)与对应的处理函数映射存入到map表中。它将每个传入请求的URL与已注册模式列表进行匹配,并获取与URL最匹配的handler。
二、开启服务,监听客户端连接
func ListenAndServe(addr string, handler Handler) error {
server := &Server{Addr: addr, Handler: handler}
return server.ListenAndServe()
}
ListenAndServe 监听TCP的网络地址然后调用Serve的handler来处理连接上的请求,接收到的连接默认是启用TCP的keep-alives,也就是保持长连接,如果Handler为nil,则默认使用DefaultServeMux。
根据addr是监听的tcp地址和Handler创建服务server,然后调用ListenAndServe方法,ListenAndServe方法如下:
func (srv *Server) ListenAndServe() error {
addr := srv.Addr
if addr == "" {
addr = ":http"
}
ln, err := net.Listen("tcp", addr)
if err != nil {
return err
}
return srv.Serve(tcpKeepAliveListener{ln.(*net.TCPListener)})
}
监听TCP网络地址srv.Addr,然后调用Serve来处理传入连接上的请求。 接受的连接配置为启用TCP长连接。 如果srv.Addr为空,则使用":http"。
func (srv *Server) Serve(l net.Listener) error {
defer l.Close()
//如果testHookServerServe不为nil,则调用
if fn := testHookServerServe; fn != nil {
fn(srv, l)
}
var tempDelay time.Duration // how long to sleep on accept failure 当接收失败休眠多久
//设置HTTP2的相关信息
if err := srv.setupHTTP2_Serve(); err != nil {
return err
}
//添加到srv的listeners中,listeners记录所有的监听器
srv.trackListener(l, true)
defer srv.trackListener(l, false)
//初始化上下文,其父级是emptyCtx,用于保存key为ServerContextKey,value是对应的Server服务
baseCtx := context.Background() // base is always background, per Issue 16220
ctx := context.WithValue(baseCtx, ServerContextKey, srv)
//监听新的tcp连接
for {
rw, e := l.Accept() //rw监听到连接的tcp,rw是net.Conn
if e != nil {
select {
case <-srv.getDoneChan(): //Server已关闭
return ErrServerClosed
default:
}
if ne, ok := e.(net.Error); ok && ne.Temporary() { //如果是接收失败的错误
if tempDelay == 0 { //接收等待时间为0,则给于默认值
tempDelay = 5 * time.Millisecond
} else { //接收等待时间扩大两倍
tempDelay *= 2
}
if max := 1 * time.Second; tempDelay > max { //最大等待时间为1s,如果大于1s,则赋值为1s
tempDelay = max
}
srv.logf("http: Accept error: %v; retrying in %v", e, tempDelay)
time.Sleep(tempDelay)
continue
}
return e
}
tempDelay = 0 //接收失败等待时间置为0
c := srv.newConn(rw) //将net.Conn封装成http.conn
c.setState(c.rwc, StateNew) // 设置连接状态
go c.serve(ctx) //每一个新的连接,开启一个协程执行serve函数
}
}
Serve方法 接收Listener l 上传入的连接,为每个连接开启一个服的goroutine,新开启的goroutine执行serve方法。也就是读取客户端的请求数据,然后根据URL获取对应的Handler,Handler就是处理请求并响应客户端的请求。
func (c *conn) serve(ctx context.Context) {
c.remoteAddr = c.rwc.RemoteAddr().String() //获取远程地址
ctx = context.WithValue(ctx, LocalAddrContextKey, c.rwc.LocalAddr()) //创建一个子Context,父Context为ctx
defer func() {
if err := recover(); err != nil && err != ErrAbortHandler { //处理panic
const size = 64 << 10
buf := make([]byte, size)
buf = buf[:runtime.Stack(buf, false)]
c.server.logf("http: panic serving %v: %v\n%s", c.remoteAddr, err, buf)
}
if !c.hijacked() { //没有hijacked挂住,则关闭连接,改变ConnState
c.close()
c.setState(c.rwc, StateClosed)
}
}()
//处理tls的连接
if tlsConn, ok := c.rwc.(*tls.Conn); ok { //如果该连接是tls.Conn ,常见连接是net.TcpConn 和 tls.Conn
if d := c.server.ReadTimeout; d != 0 { //设置读超时时间
c.rwc.SetReadDeadline(time.Now().Add(d))
}
if d := c.server.WriteTimeout; d != 0 { //设置写超时时间
c.rwc.SetWriteDeadline(time.Now().Add(d))
}
if err := tlsConn.Handshake(); err != nil { //运行客户端或服务器握手协议,如果有错误则返回
c.server.logf("http: TLS handshake error from %s: %v", c.rwc.RemoteAddr(), err)
return
}
c.tlsState = new(tls.ConnectionState)
*c.tlsState = tlsConn.ConnectionState()
if proto := c.tlsState.NegotiatedProtocol; validNPN(proto) { //协商下一个协议
if fn := c.server.TLSNextProto[proto]; fn != nil {
h := initNPNRequest{tlsConn, serverHandler{c.server}} //初始化NPN请求
fn(c.server, tlsConn, h)
}
return
}
}
// HTTP/1.x from here on.
//创建上下文
ctx, cancelCtx := context.WithCancel(ctx)
c.cancelCtx = cancelCtx
defer cancelCtx()
//初始化connReader
c.r = &connReader{conn: c}
//初始化bufioReader用于读数据
c.bufr = newBufioReader(c.r)
//初始化bufw用于写数据
c.bufw = newBufioWriterSize(checkConnErrorWriter{c}, 4<<10)
for {
//读取Request,并封装成Response
w, err := c.readRequest(ctx)
if c.r.remain != c.server.initialReadLimitSize() {
// If we read any bytes off the wire, we're active.
c.setState(c.rwc, StateActive)
}
if err != nil { //错误处理
const errorHeaders = "\r\nContent-Type: text/plain; charset=utf-8\r\nConnection: close\r\n\r\n"
if err == errTooLarge {
// Their HTTP client may or may not be
// able to read this if we're
// responding to them and hanging up
// while they're still writing their
// request. Undefined behavior.
const publicErr = "431 Request Header Fields Too Large"
fmt.Fprintf(c.rwc, "HTTP/1.1 "+publicErr+errorHeaders+publicErr)
c.closeWriteAndWait()
return
}
if isCommonNetReadError(err) {
return // don't reply
}
publicErr := "400 Bad Request"
if v, ok := err.(badRequestError); ok {
publicErr = publicErr + ": " + string(v)
}
fmt.Fprintf(c.rwc, "HTTP/1.1 "+publicErr+errorHeaders+publicErr)
return
}
// Expect 100 Continue support
req := w.req
if req.expectsContinue() { //如果用户的请求期望 100-continue
if req.ProtoAtLeast(1, 1) && req.ContentLength != 0 {
// Wrap the Body reader with one that replies on the connection
//将请求的body进行封装
req.Body = &expectContinueReader{readCloser: req.Body, resp: w}
}
} else if req.Header.get("Expect") != "" { //如果Header的Expect的值不是100-continue则返回错误码
w.sendExpectationFailed()
return
}
c.curReq.Store(w) //储存response,并且response中有request
if requestBodyRemains(req.Body) {
registerOnHitEOF(req.Body, w.conn.r.startBackgroundRead)
} else {
if w.conn.bufr.Buffered() > 0 {
w.conn.r.closeNotifyFromPipelinedRequest()
}
w.conn.r.startBackgroundRead()
}
// HTTP cannot have multiple simultaneous active requests.[*]
// Until the server replies to this request, it can't read another,
// so we might as well run the handler in this goroutine.
// [*] Not strictly true: HTTP pipelining. We could let them all process
// in parallel even if their responses need to be serialized.
// But we're not going to implement HTTP pipelining because it
// was never deployed in the wild and the answer is HTTP/2.
//调用自己实现的Handler,用于处理Request,并且返回Response
serverHandler{c.server}.ServeHTTP(w, w.req)
w.cancelCtx() //调用上下文的取消函数
if c.hijacked() { //如果请求被挂住,则返回
return
}
w.finishRequest() //处理一些状态
if !w.shouldReuseConnection() {
if w.requestBodyLimitHit || w.closedRequestBodyEarly() {
c.closeWriteAndWait()
}
return
}
c.setState(c.rwc, StateIdle) //改变请求状态
c.curReq.Store((*response)(nil))
if !w.conn.server.doKeepAlives() { //过了keep alive时间
// We're in shutdown mode. We might've replied
// to the user without "Connection: close" and
// they might think they can send another
// request, but such is life with HTTP/1.1.
return
}
if d := c.server.idleTimeout(); d != 0 {
c.rwc.SetReadDeadline(time.Now().Add(d))
if _, err := c.bufr.Peek(4); err != nil {
return
}
}
c.rwc.SetReadDeadline(time.Time{}) //重置ReadDeadline
}
}
serve用于循环读取用户的请求数据Request,并封装成Response,然后调用调用自己传入的Handler,如果未指定Handler则使用默认的DefaultServeMux。
serverHandler{c.server}.ServeHTTP(w, w.req)
func (sh serverHandler) ServeHTTP(rw ResponseWriter, req *Request) {
handler := sh.srv.Handler
if handler == nil { //如果没有实现handler则用DefaultServeMux
handler = DefaultServeMux
}
if req.RequestURI == "*" && req.Method == "OPTIONS" {
handler = globalOptionsHandler{}
}
handler.ServeHTTP(rw, req)
}
调用handler的ServeHTTP来处理客户端的请求,并响应。
我们看一下默认的Handler DefaultServeMux的ServeHTTP
func (mux *ServeMux) ServeHTTP(w ResponseWriter, r *Request) {
/*
处理RequestURI为*的情况
*/
if r.RequestURI == "*" {
if r.ProtoAtLeast(1, 1) {
w.Header().Set("Connection", "close")
}
w.WriteHeader(StatusBadRequest)
return
}
//根据请求找到匹配的handler
h, _ := mux.Handler(r)
h.ServeHTTP(w, r)
}
看到Handler函数根据用户请求去匹配对应的Handler,如以上demo,根据pattern /hello,可以找到注册到路由的handler即HelloServer,由HelloServer处理请求,并写入response。
还没有评论,来说两句吧...