Nginx源代码分析之I/O细节（十一）

快来打我* 2022-08-03 05:11 197阅读 0赞

至于每个平台和模型里面具体I/O的细节，我们简单分析一下，先看看发送的具体实现，我们先以iocp模型来进行具体分析。

在Upstream部分，最后提到真正的发送函数是一个send\_chain指针，对于不同的系统，指向不同的调用函数，对于win平台，其指向的是ngx\_overlapped\_wsasend\_chain，此函数比较复杂，我们看看等同的调用ngx\_overlapped\_wsasend，首先要确定，当我们调用ngx\_http\_upstream\_send\_request的时候，要么是在ngx\_http\_upstream\_send\_request\_handler里面，要么是在ngx\_http\_upstream\_connect函数里面，实际上都是connect成功之后的进行的调用，因此，ngx\_connection\_t->write->ready =1，ngx\_connection\_t->write->complete = 0，这样，会执行下面的代码：

if (!wev->complete) \{  
/\* post the overlapped WSASend() \*/  
/\*  
\* WSABUFs must be 4-byte aligned otherwise  
\* WSASend() will return undocumented WSAEINVAL error.  
\*/  
wsabuf.buf = (char \*) buf;  
wsabuf.len = size;  
sent = 0;  
ovlp = (LPWSAOVERLAPPED) &c->write->ovlp;  
ngx\_memzero(ovlp, sizeof(WSAOVERLAPPED));  
n = WSASend(c->fd, &wsabuf, 1, &sent, 0, ovlp, NULL);  
ngx\_log\_debug4(NGX\_LOG\_DEBUG\_EVENT, c->log, 0,  
"WSASend: fd:%d, %d, %ul of %uz", c->fd, n, sent, size);  
wev->complete = 0;  
if (n == 0) \{  
if (ngx\_event\_flags & NGX\_USE\_IOCP\_EVENT) \{  
/\*  
\* if a socket was bound with I/O completion port then  
\* GetQueuedCompletionStatus() would anyway return its status  
\* despite that WSASend() was already complete  
\*/  
wev->active = 1;  
return NGX\_AGAIN;  
\}  
if (sent < size) \{  
wev->ready = 0;  
\}  
c->sent += sent;  
return sent;  
\}  
err = ngx\_socket\_errno;  
if (err == WSA\_IO\_PENDING) \{  
ngx\_log\_debug0(NGX\_LOG\_DEBUG\_EVENT, c->log, err,  
"WSASend() posted");  
wev->active = 1;  
return NGX\_AGAIN;  
\}  
wev->error = 1;  
ngx\_connection\_error(c, err, "WSASend() failed");  
return NGX\_ERROR;  
\}

如果WSASend的返回值是0说明发送数据立即成功了，这时候把complete=0，然后返回发送成功的长度sent即可。如果没有立即成功，返回值值又是WSA\_IO\_PENDING，则返回NGX\_AGAIN，让线程执行下一个I/O。

如果ngx\_connection\_t->write->complete = 1，说明ngx\_iocp\_process\_events里面的NGX\_IOCP\_IO被触发，当然套接口存在一个已经就绪的写数据，这时候需要调用WSAGetOverlappedResult来获取完成的操作的数据长度，即sent，如果sent < size，说明只完成了部分，还需要继续等待后续数据，因此ngx\_connection\_t->write->ready 置为0。

我们看看发送成功之后，接着ngx\_http\_upstream\_process\_header是如何接收数据，在经过一些初始化之后，是一个无限循环来读取数据

for ( ;; ) \{

n = c->recv(c, u->buffer.last, u->buffer.end - u->buffer.last);  
if (n == NGX\_AGAIN) \{  
\#if 0  
ngx\_add\_timer(rev, u->read\_timeout);  
\#endif  
if (ngx\_handle\_read\_event(c->read, 0) != NGX\_OK) \{  
ngx\_http\_upstream\_finalize\_request(r, u,NGX\_HTTP\_INTERNAL\_SERVER\_ERROR);  
return;  
\}

return;  
\}  
  
  
if (n == 0) \{  
ngx\_log\_error(NGX\_LOG\_ERR, c->log, 0,  
"upstream prematurely closed connection");  
\}  
  
if (n == NGX\_ERROR || n == 0) \{  
ngx\_http\_upstream\_next(r, u, NGX\_HTTP\_UPSTREAM\_FT\_ERROR);  
return;  
\}  
u->buffer.last += n;  
\#if 0  
u->valid\_header\_in = 0;  
u->peer.cached = 0;  
\#endif  
  
  
rc = u->process\_header(r);  
if (rc == NGX\_AGAIN) \{  
if (u->buffer.last == u->buffer.end) \{  
ngx\_log\_error(NGX\_LOG\_ERR, c->log, 0,  
"upstream sent too big header");  
  
ngx\_http\_upstream\_next(r, u,  
NGX\_HTTP\_UPSTREAM\_FT\_INVALID\_HEADER);  
return;  
\}  
continue;  
\}  
break;  
\}

如果是win的iocp模型其中c->recv对应的是ngx\_overlapped\_wsarecv，ngx\_overlapped\_wsarecv必须跟ngx\_iocp\_process\_events联系起来分析在NGX\_IOCP\_CONNECT完成的情况下，ev->ready = 1，ev->complete=0，于是直接执行下面的代码

ovlp = (LPWSAOVERLAPPED) &rev->ovlp;

ngx\_memzero(ovlp, sizeof(WSAOVERLAPPED));  
wsabuf\[0\].buf = (char \*) buf;  
wsabuf\[0\].len = size;  
flags = 0;  
bytes = 0;we  
rc = WSARecv(c->fd, wsabuf, 1, &bytes, &flags, ovlp, NULL);

发起一个读请求。

如果是NGX\_IOCP\_IO，即上面的普通读写请求， ev->complete = 1;ev->ready = 1;就会执行WSAGetOverlappedResult，并返回完成I/O的数据长度。

在看看epoll模型，由于epoll是异步通知，而非直接的异步I/O，因此，处理过程相对简单的多，发送的时候，参考ngx\_unix\_send的代码，

n = send(c->fd, buf, size, 0);

如果立即成功，则返回n，如果失败， wev->ready = 0;并读取错误代码errno，然后返回NGX\_AGAIN，让进程执行下一个操作。

如果是读操作，参考ngx\_unix\_recv的代码

n = recv(c->fd, buf, size, 0);

如果n>0，说明读取立即成功，返回读取的长度。

如果n为0，执行

if (n == 0) \{  
  
  
/\*  
\* on FreeBSD recv() may return 0 on closed socket  
\* even if kqueue reported about available data  
\*/  
  
  
rev->eof = 1;  
rev->available = 0;  
\}

说明socket已经被动关闭，因此设置eof标志，也返回长度0给调用者。

如果n<0，并且err == NGX\_EAGAIN，则返回NGX\_AGAIN，说明读操作处于等待状态。让进程执行后面的读写操作。