一個linux下簡單的純C++實現Http請求類（GET，POST，上傳，下載）

目錄

• 一個linux下簡單的純C++實現Http請求類（GET，POST，上傳，下載）
  • Http協議簡述
  • HttpRequest類設計
    • 請求部分
    • 接收部分
    • 關於上傳和下載
    • Cpp實現
    • 關於源碼中的Logger
  • 使用示例

---

最近寫了點關於Http上傳下載文件相關的，於是今天整理下代碼。

Http協議簡述
HttpRequest類設計
使用示例

Http協議簡述

協議：網絡協議的簡稱，網絡協議是通信計算機雙方必須共同遵從的一組約定。如怎么樣建立連接、怎么樣互相識別等。只有遵守這個約定，計算機之間才能相互通信交流，就好比兩台計算機交互的語言.

HTTP協議：超文本傳輸協議（HyperText Transfer Protocol)是互聯網上應用最為廣泛的一種網絡協議。主要被用於在Web瀏覽器和網站服務器之間傳遞信息。 HTTP 是基於 TCP/IP 協議的應用層協議。默認使用80端口。最新版本是HTTP 2.0，目前是用最廣泛的是HTTP 1.1。

HTTP協議工作於客戶端-服務端架構為上。瀏覽器作為HTTP客戶端通過URL向HTTP服務端即WEB服務器發送所有請求。Web服務器根據接收到的請求后，向客戶端發送響應信息。

請求方法：
HTTP/1.1協議中共定義了八種方法（有時也叫“動作”）來表明Request-URI指定的資源的不同操作方式：
OPTIONS - 返回服務器針對特定資源所支持的HTTP請求方法。也可以利用向Web服務器發送'*'的請求來測試服務器的功能性。
HEAD- 向服務器索要與GET請求相一致的響應，只不過響應體將不會被返回。這一方法可以在不必傳輸整個響應內容的情況下，就可以獲取包含在響應消息頭中的元信息。該方法常用於測試超鏈接的有效性，是否可以訪問，以及最近是否更新。
GET - 向特定的資源發出請求。注意：GET方法不應當被用於產生“副作用”的操作中，例如在web app.中。其中一個原因是GET可能會被網絡蜘蛛等隨意訪問。
POST - 向指定資源提交數據進行處理請求（例如提交表單或者上傳文件）。數據被包含在請求體中。POST請求可能會導致新的資源的建立和/或已有資源的修改。
PUT - 向指定資源位置上傳其最新內容。
DELETE - 請求服務器刪除Request-URI所標識的資源。
TRACE- 回顯服務器收到的請求，主要用於測試或診斷。
CONNECT - HTTP/1.1協議中預留給能夠將連接改為管道方式的代理服務器。
PATCH - 用來將局部修改應用於某一資源，添加於規范RFC5789。
方法名稱是區分大小寫的。當某個請求所針對的資源不支持對應的請求方法的時候，服務器應當返回狀態碼405（Method Not Allowed）；當服務器不認識或者不支持對應的請求方法的時候，應當返回狀態碼501（Not Implemented）。
HTTP服務器至少應該實現GET和HEAD方法，其他方法都是可選的。此外，除了上述方法，特定的HTTP服務器還能夠擴展自定義的方法。

GET和POST請求的區別
GET請求
GET /books/?sex=man&name=Professional HTTP/1.1
Host: www.wrox.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.6)
Gecko/20050225 Firefox/1.0.1
Connection: Keep-Alive
注意最后有一行空行

POST請求
POST / HTTP/1.1
Host: www.wrox.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.6)
Gecko/20050225 Firefox/1.0.1
Content-Type: application/x-www-form-urlencoded
Content-Length: 40
Connection: Keep-Alive

name=Professional%20Ajax&publisher=Wiley

1、GET提交：請求的數據會附在URL之后（就是把數據放置在HTTP協議頭中），以?分割URL和傳輸數據，多個參數用&連接；例 如：login.action?name=hyddd&password=idontknow&verify=%E4%BD%A0 %E5%A5%BD。如果數據是英文字母/數字，原樣發送，如果是空格，轉換為+，如果是中文/其他字符，則直接把字符串用BASE64加密，得出如： %E4%BD%A0%E5%A5%BD，其中％XX中的XX為該符號以16進制表示的ASCII。

2、POST提交：把提交的數據放置在是HTTP包的包體中。上文示例中紅色字體標明的就是實際的傳輸數據

HttpRequest類設計

請求部分

HttpRequest::HttpRequest() 通過傳入Url構造HttpUrl類分離url域名及uri.
HttpRequest::connect() 通過gethostbyname()獲取域名ip，與80端口組成遠端地址建立鏈接.
HttpRequest::setRequestMethod() 設置請求方法,目前只添加了Get和Post請求.
HttpRequest::setRequestProperty() 設置屬性.
HttpRequest::setRequestBody() 設置content.
HttpRequest::send() 將設置的請求流發送出去.

接收部分

HttpRequest::handRead() 處理服務器應答頭.
HttpRequest::getResponseCode() handRead()后可以獲取到應答code
HttpRequest::getResponseProperty() handRead()后可以獲取到對應的應答屬性
HttpRequest::getResponseContent() handRead()后可以獲取到應答content

#ifndef _HTTP_REQUEST_HH
#define _HTTP_REQUEST_HH

#include <netdb.h>
#include <arpa/inet.h>
#include <algorithm>
#include <vector>
#include <string>
#include <assert.h>
#include <string.h>
#include <sstream>
#include <map>
#include "Logger.hh"

const size_t kBufferSize = 4096;

/// A buffer class modeled after org.jboss.netty.buffer.ChannelBuffer
///
/// @code
/// +-------------------+------------------+------------------+
/// | prependable bytes |  readable bytes  |  writable bytes  |
/// |                   |     (CONTENT)    |                  |
/// +-------------------+------------------+------------------+
/// |                   |                  |                  |
/// 0      <=      readerIndex   <=   writerIndex    <=     size
/// @endcode
class Buffer
{
public:

  static const size_t kCheapPrepend = 8;
  static const size_t kInitialSize = 4096;
//public:

	explicit Buffer(size_t initialSize = kInitialSize)
	: m_buffer(kCheapPrepend + initialSize),
	  m_readerIndex(kCheapPrepend),
	  m_writerIndex(kCheapPrepend)
	  {
	    assert(readableBytes() == 0);
        assert(writableBytes() == initialSize);
        assert(prependableBytes() == kCheapPrepend);
	  }

  size_t readableBytes() const
  { return m_writerIndex - m_readerIndex; }

  size_t writableBytes() const
  { return m_buffer.size() - m_writerIndex; }

  size_t prependableBytes() const
  { return m_readerIndex; }

  const char* peek() const
  { return begin() + m_readerIndex; }

  char* beginWrite()
  { return begin() + m_writerIndex; }

  void hasWritten(size_t len)
  {
    assert(len <= writableBytes());
    m_writerIndex += len;
  }

  void unwrite(size_t len)
  {
    assert(len <= readableBytes());
    m_writerIndex -= len;
  }

  // retrieve returns void, to prevent
  // string str(retrieve(readableBytes()), readableBytes());
  // the evaluation of two functions are unspecified
  void retrieve(size_t len)
  {
    assert(len <= readableBytes());
    if (len < readableBytes())
    {
      m_readerIndex += len;
    }
    else
    {
      retrieveAll();
    }
  }

  void retrieveAll()
  {
  	m_readerIndex = kCheapPrepend;
  	m_writerIndex = kCheapPrepend;
  }

private:
  char* begin()
  {return &*m_buffer.begin(); }

  const char* begin() const
  {return &*m_buffer.begin(); }

private:
	std::vector<char> m_buffer;
	size_t m_readerIndex;
	size_t m_writerIndex;
};


namespace sockets
{

/// Creates a non-blocking socket file descriptor,
/// abort if any error.
int createSocket(sa_family_t family);

int  connect(int sockfd, const struct sockaddr* addr);
ssize_t read(int sockfd, void *buf, size_t count);
ssize_t readv(int sockfd, const struct iovec *iov, int iovcnt);
ssize_t write(int sockfd, const void *buf, size_t count);
void close(int sockfd);

void fromIpPort(const char* ip, uint16_t port,
                struct sockaddr_in* addr);

int getSocketError(int sockfd);
void delaySecond(int sec);
//const struct sockaddr* sockaddr_cast(const struct sockaddr_in* addr)
//const struct sockaddr_in* sockaddr_in_cast(const struct sockaddr* addr);

}

class InetAddress
{
 public:
  /// Constructs an endpoint with given ip and port.
  /// @c ip should be "1.2.3.4"
  InetAddress(std::string ip, uint16_t port);

  /// Constructs an endpoint with given struct @c sockaddr_in
  /// Mostly used when accepting new connections
  explicit InetAddress(const struct sockaddr_in& addr)
    : m_addr(addr)
  { }

  sa_family_t family() const { return m_addr.sin_family; }
  //std::string toIp() const;
  //std::string toIpPort() const;

  const struct sockaddr* getSockAddr() const { return (struct sockaddr*)(&m_addr); }

  uint32_t ipNetEndian() const;

  // resolve hostname to IP address, not changing port or sin_family
  // return true on success.
  // thread safe
  // static bool resolve(StringArg hostname, StringArg* ip);
  // static std::vector<InetAddress> resolveAll(const char* hostname, uint16_t port = 0);

 private:
    struct sockaddr_in m_addr;

};


class HttpUrl
{
public:
	HttpUrl(std::string& httpUrl)
	:m_httpUrl(httpUrl),
	 m_smatch(detachHttpUrl())
	{
		LOG_DEBUG << "URL : " << m_httpUrl;
	}
	~HttpUrl(){};

	enum HttpUrlMatch
	{
		URL = 0,
		HOST = 1,
		URI = 2
	};

	std::vector<std::string> detachHttpUrl() const
	{
	  std::vector<std::string> v;
	  std::string::size_type pos1, pos2;
	  pos2 = m_httpUrl.find('/');
	  assert(std::string::npos != pos2);
	  pos1 = pos2 + 2;
	  pos2 = m_httpUrl.find('/', pos1);
	  assert(std::string::npos != pos2);
	  v.push_back(m_httpUrl);
	  v.push_back(m_httpUrl.substr(pos1, pos2 - pos1));
	  v.push_back(m_httpUrl.substr(pos2 + 1));
	  LOG_DEBUG << "detachHttpUrl() url :" << v[0];
	  LOG_DEBUG << "detachHttpUrl() host :" << v[1];
	  LOG_DEBUG << "detachHttpUrl() uri :" << v[2];
	  return v;
	}

	bool HttpUrlToIp(const std::string& host, char* ip) const
	{
		struct hostent* phost = NULL;

		phost = gethostbyname(host.c_str());
	    if (NULL == phost)
	    {
	    	LOG_ERROR << "HttpUrlToIp(): gethostbyname error : " << errno << " : "<< strerror(errno);
	    	return false;
	    	//LOG_SYSERR << "urlToIp(): gethostbyname error";
	    }

	    inet_ntop(phost->h_addrtype,  phost->h_addr, ip, 17);

	    return true;
	}

	std::string domain() const
	{
		return getHttpUrlSubSeg(HOST);
	}

	std::string getHttpUrlSubSeg(HttpUrlMatch sub = HOST) const{ return m_smatch[sub]; }

private:
	std::string m_httpUrl;
	std::vector<std::string> m_smatch;
};


class HttpRequest
{
public:
	enum HttpRequestMethod{
		GET = 0,
		POST
	};
	HttpRequest(std::string httpUrl);
	~HttpRequest();

	void connect();
	//void TEST(const std::string path,const std::string content);
	void setRequestMethod(const std::string &method);
	void setRequestProperty(const std::string &key, const std::string &value);
	void setRequestBody(const std::string &content);

	//void clear() { clearStream(); m_buffer.retrieveAll(); }
	void clearStream() {m_stream.str("");}
	std::string strStream() const { return m_stream.str(); };

	int getResponseCode() const {
		assert(m_haveHandleHead);
		return m_code;
	}

	std::string getResponseProperty(const std::string& key) const {
		assert(m_haveHandleHead);
		return m_ackProperty.at(key);
	}

	std::string getResponseContent() {
		assert(m_haveHandleHead);
		return std::string(m_buffer.peek(), m_buffer.readableBytes());
	}

	void handleRead();
	void uploadFile(const std::string& file, const std::string& contentEnd);
	void downloadFile(const std::string& file);

	void send(){
		sockets::write(m_sockfd, strStream().c_str(), strStream().size());
	}

	void close(){ sockets::close(m_sockfd); }

private:
    void SplitString(const std::string& s, std::vector<std::string>& v, const std::string& c);

  Buffer m_buffer;
	HttpUrl m_httpUrl;
	std::stringstream m_stream;
	int m_code;
	int m_sockfd;
	bool m_haveHandleHead;
	std::map<std::string, std::string> m_ackProperty;
};

#endif

關於上傳和下載

HttpRequest::downloadFile(const std::string& file);
下載直接在 handRead() 處理完應答頭后 ， 調用downloadFile() 存儲在本地.
HttpRequest::uploadFile(const std::string& file, const std::string& contentEnd)
上傳部分復雜一點，需要根據kBoundary 設置邊界,如下.

	off_t fileSize = FileSize(uploadFile);
	LOG_DEBUG << "fileSize : " << fileSize;
	std::stringstream content;
	content << "--" + kBoundary << "\r\n";
	content << "Content-Disposition: form-data; name=\"upload\"; filename=\"test.file\"\r\n";
	content << "Content-Type: text/plain\r\n\r\n";

	std::string contentEnd = "\r\n--" + kBoundary + "--\r\n";

	HttpClient upload("http://xxxxx/upload");
	upload.setRequestMethod("POST");
	upload.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + kBoundary);
	upload.setRequestProperty("Cache-Control", "no-cache");
	upload.setRequestProperty("Content-Length", std::to_string(fileSize + content.str().size() + contentEnd.size()));
	upload.setRequestProperty("Connection", "close\r\n");
	upload.setRequestBody(content.str());

Cpp實現

#include <string>
#include <iostream>
#include <fcntl.h>
#include <errno.h>
#include <sys/uio.h>  // readv
#include <stdint.h>
#include <endian.h>
#include <unistd.h>
#include <map>
#include <fstream>

#include "HttpRequest.hh"

const std::map<std::string, int>::value_type init_value[] =
{
	std::map<std::string, int>::value_type( "GET", HttpRequest::GET),

	std::map<std::string, int>::value_type( "POST", HttpRequest::POST)
};

const static std::map<std::string, int> kRequestMethodMap(init_value, init_value + (sizeof init_value / sizeof init_value[0]));

static inline uint16_t hostToNetwork16(uint16_t host16)
{
  return htobe16(host16);
}


int sockets::createSocket(sa_family_t family){
  // Call "socket()" to create a (family) socket of the specified type.
  // But also set it to have the 'close on exec' property (if we can)

	int sock;

	//CLOEXEC，即當調用exec（）函數成功后，文件描述符會自動關閉。
	//在以往的內核版本（2.6.23以前）中，需要調用 fcntl(fd, F_SETFD, FD_CLOEXEC) 來設置這個屬性。
	//而新版本（2.6.23開始）中，可以在調用open函數的時候，通過 flags 參數設置 CLOEXEC 功能，
#ifdef SOCK_CLOEXEC
  sock = socket(family, SOCK_STREAM|SOCK_CLOEXEC, 0);
  if (sock != -1 || errno != EINVAL) return sock;
  // An "errno" of EINVAL likely means that the system wasn't happy with the SOCK_CLOEXEC; fall through and try again without it:
#endif

  sock = socket(family, SOCK_STREAM, 0);

#ifdef FD_CLOEXEC
  if (sock != -1) fcntl(sock, F_SETFD, FD_CLOEXEC);
#endif
  return sock;
}

int sockets::connect(int sockfd, const struct sockaddr* addr)
{
  return ::connect(sockfd, addr, sizeof(struct sockaddr));
}

void sockets::fromIpPort(const char* ip, uint16_t port,
                         struct sockaddr_in* addr)
{
  addr->sin_family = AF_INET;
  addr->sin_port = hostToNetwork16(port);
  if (::inet_pton(AF_INET, ip, &addr->sin_addr) <= 0)
  {
    LOG_SYSERR << "sockets::fromIpPort";
  }
}

ssize_t sockets::read(int sockfd, void *buf, size_t count)
{
  return ::read(sockfd, buf, count);
}

ssize_t sockets::readv(int sockfd, const struct iovec *iov, int iovcnt)
{
  return ::readv(sockfd, iov, iovcnt);
}

ssize_t sockets::write(int sockfd, const void *buf, size_t count)
{
  return ::write(sockfd, buf, count);
}

void sockets::close(int sockfd)
{
  if (::close(sockfd) < 0)
  {
    LOG_SYSERR << "sockets::close";
  }
}

void sockets::delaySecond(int sec)
{
  struct  timeval tv;
  tv.tv_sec = sec;
  tv.tv_usec = 0;
  select(0, NULL, NULL, NULL, &tv);
}



InetAddress::InetAddress(std::string ip, uint16_t port)
{
  ::bzero(&m_addr, sizeof m_addr);
  sockets::fromIpPort(ip.c_str(), port, &m_addr);
}

uint32_t InetAddress::ipNetEndian() const
{
  assert(family() == AF_INET);
  return m_addr.sin_addr.s_addr;
}


HttpRequest::HttpRequest(std::string httpUrl)
  :m_httpUrl(httpUrl)
{

}

HttpRequest::~HttpRequest()
{

}

void HttpRequest::connect()
{
  char ip[32] = {0};
  while(true)
  {
    struct hostent* phost = NULL;

    phost = gethostbyname(m_httpUrl.domain().c_str());
    if (NULL == phost)
    {
      LOG_ERROR << "HttpUrlToIp(): gethostbyname error : " << errno << " : "<< strerror(errno) << " continue.";
      sockets::delaySecond(1);
      continue;
    }

    inet_ntop(phost->h_addrtype,  phost->h_addr, ip, sizeof ip);

    LOG_DEBUG << "HttpRequest::Connector() gethostbyname Successful";

    InetAddress serverAddr = InetAddress(ip, 80);

    m_sockfd = sockets::createSocket(serverAddr.family());
    if(m_sockfd < 0) LOG_SYSERR << "HttpRequest::connect() : createSocket error";
    int ret = sockets::connect(m_sockfd, serverAddr.getSockAddr());
    LOG_DEBUG << "sockfd : " << m_sockfd << "sockets::connect ret : " << ret ;
    int savedErrno = (ret == 0) ? 0 : errno;

    switch (savedErrno)
    {
      case 0:
      case EINPROGRESS:
      case EINTR:
      case EISCONN:
        LOG_INFO << "HttpRequest::connect() sockfd : " << m_sockfd << " Successful";
        break;
      default :
        LOG_ERROR << "Connect Error ";
        sockets::delaySecond(1);
        continue;
    }

    break;
  }

  LOG_DEBUG << "HttpRequest::Connector() end";
}

void HttpRequest::setRequestMethod(const std::string &method)
{
	switch(kRequestMethodMap.at(method))
	{
		case HttpRequest::GET :
			m_stream << "GET " << "/" << m_httpUrl.getHttpUrlSubSeg(HttpUrl::URI) << " HTTP/1.1\r\n";
			LOG_DEBUG << m_stream.str().c_str();
			break;
		case HttpRequest::POST :
			m_stream << "POST "  << "/" << m_httpUrl.getHttpUrlSubSeg(HttpUrl::URI) << " HTTP/1.1\r\n";
			LOG_DEBUG << m_stream.str().c_str();
			break;
		default :
			LOG_ERROR << "No such Method : " << method.c_str();
			break;
	}

	m_stream << "Host: " << m_httpUrl.getHttpUrlSubSeg(HttpUrl::HOST) << "\r\n";
}


void HttpRequest::setRequestProperty(const std::string &key, const std::string &value)
{
	m_stream << key << ": " << value << "\r\n";
}

void HttpRequest::setRequestBody(const std::string &content)
{
	m_stream << content;
}

void HttpRequest::handleRead()
{
	assert(!m_haveHandleHead);
	ssize_t nread = 0;
	ssize_t writtenBytes = 0;

	nread = sockets::read(m_sockfd, m_buffer.beginWrite(), kBufferSize);
	if(nread < 0) LOG_SYSFATAL << "sockets::read";
	m_buffer.hasWritten(nread);
	LOG_TRACE << "sockets::read(): nread: " << nread << " remain: " << m_buffer.writableBytes();
	size_t remain = kBufferSize - nread;
	while(remain > 0)
	{
		size_t n = sockets::read(m_sockfd, m_buffer.beginWrite(), remain);
		if(n < 0) LOG_SYSFATAL << "sockets::read";
		m_buffer.hasWritten(n);
		if(0 == n)
		{
			LOG_DEBUG << "sockets::read finish";
			break;
		}
		remain = remain - n;
	}
	//std::cout << m_buffer.peek();

	//for(int i = 0; i < nread; i++) printf("%02x%c",(unsigned char)buffer[i],i==nread - 1 ?'\n':' ');
	//LOG_DEBUG << "handleRead Recv Response : \n" << m_buffer.peek();
	int headsize = 0;
	std::string line;
	std::stringstream ss(m_buffer.peek());
	std::vector<std::string> v;
	getline(ss, line);
	//LOG_DEBUG << line;
	headsize += line.size() + 1;
	SplitString(line, v, " ");
	//for(int i = 0; i < v.size(); i++) std::cout << v[i] << std::endl;
	m_code = std::stoi(v[1]);
	if(v[1] != "200")
	{
	  LOG_ERROR << "Error Http Server Response : " << v[1].c_str();
	}

	do{
		getline(ss, line);
		headsize += line.size() + 1;  // + 1('\n')
		if(!line.empty()) line.erase(line.end()-1); // remove '/r'
		//LOG_DEBUG << line;
		v.clear();
		SplitString(line, v, ":");
		if(v.size() == 2){
			m_ackProperty[v[0]] = v[1].erase(0,v[1].find_first_not_of(" "));
		}
	}while(!line.empty());

	LOG_DEBUG << "Http Head Size is " << headsize;
	std::string res(m_buffer.peek(), headsize);
	LOG_DEBUG << "Http Response :\n" << res;
	m_buffer.retrieve(headsize);

	m_haveHandleHead = true;

}

void HttpRequest::uploadFile(const std::string& file, const std::string& contentEnd)
{

	FILE* fp = fopen(file.c_str(), "rb");
	if(fp == NULL)
	{
		LOG_SYSFATAL << "fopen() File :" << file.c_str() << " Errno";
	}

	bool isEnd = false;
	ssize_t writtenBytes = 0;

	assert(m_buffer.writableBytes() == Buffer::kInitialSize);

	while(!isEnd)
	{
		ssize_t nread = fread(m_buffer.beginWrite(), 1, kBufferSize, fp);
		m_buffer.hasWritten(nread);
		while(m_buffer.writableBytes() > 0)
		{
			LOG_TRACE << "file read(): nread: " << nread << " remain: " << m_buffer.writableBytes();
			size_t n = fread(m_buffer.beginWrite(), 1, m_buffer.writableBytes(), fp);
			m_buffer.hasWritten(n);
			if(0 == n)
			{	int err = ferror(fp);
				if(err)
				{
					fprintf(stderr, "fread failed : %s\n", strerror(err));
				}
				LOG_DEBUG << "sockets::read finish";
				isEnd = true;
				break;
			}
		}

		ssize_t nwrite = sockets::write(m_sockfd, m_buffer.peek(), m_buffer.readableBytes());
		if(nwrite < 0) LOG_SYSFATAL << "sockets::write";
		writtenBytes += nwrite;
		LOG_TRACE << "sockets::write nread " << m_buffer.readableBytes() << " nwrite " << nwrite << " writtenBytes " << writtenBytes;
		m_buffer.retrieve(nwrite);
	}

	fclose(fp);

	m_buffer.retrieveAll();

	ssize_t n = sockets::write(m_sockfd, contentEnd.c_str(), contentEnd.size());
	if(n < 0) LOG_SYSFATAL << "sockets::write";
}

void HttpRequest::downloadFile(const std::string& file)
{
	assert(m_haveHandleHead);

	bool isEnd = false;
	ssize_t nread = 0;
	ssize_t writtenBytes = 0;
	bool haveHandleHead = false;
	bool isDownFile = false;

	std::ofstream output(file, std::ios::binary);
	if (!output.is_open()){ // 檢查文件是否成功打開
		LOG_SYSFATAL << "open file error" << file;
	}

	output.write(m_buffer.peek(), m_buffer.readableBytes());
	writtenBytes += m_buffer.readableBytes();
	m_buffer.retrieve(m_buffer.readableBytes());

	LOG_DEBUG << "Content-Length : " << getResponseProperty("Content-Length");

	while(!isEnd)
	{
		nread = sockets::read(m_sockfd, m_buffer.beginWrite(), kBufferSize);
		if(nread < 0) LOG_SYSFATAL << "sockets::read";
		m_buffer.hasWritten(nread);
		LOG_TRACE << "sockets::read(): nread: " << nread << " remain: " << m_buffer.writableBytes() << " writtenBytes: " << writtenBytes;
		size_t remain = kBufferSize - nread;
		while(remain > 0)
		{
			size_t n = sockets::read(m_sockfd, m_buffer.beginWrite(), remain);
			if(n < 0) LOG_SYSFATAL << "sockets::read";
			m_buffer.hasWritten(n);
			if(0 == n)
			{
				LOG_DEBUG << "sockets::read finish";
				isEnd = true;
				break;
			}
			remain = remain - n;
		}

		output.write(m_buffer.peek(), m_buffer.readableBytes());
		writtenBytes += m_buffer.readableBytes();
		m_buffer.retrieve(m_buffer.readableBytes());
	}
	LOG_DEBUG << " writtenBytes " << writtenBytes;

	output.close();
	sockets::close(m_sockfd);
}

void HttpRequest::SplitString(const std::string& s, std::vector<std::string>& v, const std::string& c)
{
  std::string::size_type pos1, pos2;
  pos2 = s.find(c);
  pos1 = 0;
  while(std::string::npos != pos2)
  {
    v.push_back(s.substr(pos1, pos2-pos1));

    pos1 = pos2 + c.size();
    pos2 = s.find(c, pos1);
  }
  if(pos1 != s.length())
    v.push_back(s.substr(pos1));
}

關於源碼中的Logger

Logger部分使用的前面文章的Logger類，可通過編輯器批量更改，注釋掉,或復制第一篇異步日志的代碼直接使用.

使用示例

下載圖片

#include "HttpRequest.hh"

int main()
{

  HttpRequest ImageReq("http://img.zcool.cn/community/01ddc256eb71586ac7257d209712b7.jpg@1280w_1l_2o_100sh.jpg");
  ImageReq.setRequestMethod("GET");
  ImageReq.setRequestProperty("Cache-Control", "no-cache");
  ImageReq.setRequestProperty("Content-Type", "application/octet-stream");
  ImageReq.setRequestProperty("Connection", "close\r\n");

  ImageReq.connect();
  ImageReq.send();
  ImageReq.handleRead();
  ImageReq.downloadFile("./test.jpg");

  return 0;
}

2018-10-23 20:24:07.257167 [DEBUG] [HttpRequest.hh:194] [detachHttpUrl] detachHttpUrl() url :http://img.zcool.cn/community/01ddc256eb71586ac7257d209712b7.jpg@1280w_1l_2o_100sh.jpg
2018-10-23 20:24:07.257411 [DEBUG] [HttpRequest.hh:195] [detachHttpUrl] detachHttpUrl() host :img.zcool.cn
2018-10-23 20:24:07.257561 [DEBUG] [HttpRequest.hh:196] [detachHttpUrl] detachHttpUrl() uri :community/01ddc256eb71586ac7257d209712b7.jpg@1280w_1l_2o_100sh.jpg
2018-10-23 20:24:07.257661 [DEBUG] [HttpRequest.hh:171] [HttpUrl] URL : http://img.zcool.cn/community/01ddc256eb71586ac7257d209712b7.jpg@1280w_1l_2o_100sh.jpg
2018-10-23 20:24:07.257734 [DEBUG] [HttpRequest.cpp:178] [setRequestMethod] GET /community/01ddc256eb71586ac7257d209712b7.jpg@1280w_1l_2o_100sh.jpg HTTP/1.1

2018-10-23 20:24:08.192206 [DEBUG] [HttpRequest.cpp:142] [connect] HttpRequest::Connector() gethostbyname Successful
2018-10-23 20:24:08.232474 [DEBUG] [HttpRequest.cpp:149] [connect] sockfd : 3sockets::connect ret : 0
2018-10-23 20:24:08.233236 [INFO ] [HttpRequest.cpp:158] HttpRequest::connect() sockfd : 3 Successful
2018-10-23 20:24:08.233453 [DEBUG] [HttpRequest.cpp:169] [connect] HttpRequest::Connector() end
2018-10-23 20:24:08.257666 [TRACE] [HttpRequest.cpp:212] [handleRead] sockets::read(): nread: 4096 remain: 0
2018-10-23 20:24:08.258490 [DEBUG] [HttpRequest.cpp:257] [handleRead] Http Head Size is 798
2018-10-23 20:24:08.259139 [DEBUG] [HttpRequest.cpp:259] [handleRead] Http Response :
HTTP/1.1 200 OK
Server: Tengine
Content-Type: image/jpeg
Content-Length: 155984
Connection: close
Date: Tue, 23 Oct 2018 10:05:56 GMT
x-oss-request-id: 5BCEF284EA872628A4BA7B32
ETag: "3D2EABCF4B0809B38A154C5087E5ECD4"
Last-Modified: Fri, 18 Mar 2016 03:09:12 GMT
x-oss-object-type: Normal
x-oss-hash-crc64ecma: 17069884749112719525
x-oss-storage-class: Standard
Expires: Thu, 18 Mar 2021 03:09:12 GMT
Cache-Control: max-age=86400
Via: cache10.l2cm9[0,304-0,H], cache35.l2cm9[37,0], cache1.cn683[0,200-0,H], cache1.cn683[2,0]
Age: 8479
Ali-Swift-Global-Savetime: 1538030855
X-Cache: HIT TCP_HIT dirn:6:15817583
X-Swift-SaveTime: Tue, 23 Oct 2018 10:18:38 GMT
X-Swift-CacheTime: 85638
Access-Control-Allow-Origin: *
Timing-Allow-Origin: *
EagleId: 71db884115402976359511426e


2018-10-23 20:24:08.263882 [DEBUG] [HttpRequest.cpp:335] [downloadFile] Content-Length : 155984
2018-10-23 20:24:08.264447 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 3298
2018-10-23 20:24:08.264569 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 7394
2018-10-23 20:24:08.264978 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 11490
2018-10-23 20:24:08.265032 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 15586
2018-10-23 20:24:08.265073 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 19682
2018-10-23 20:24:08.265112 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 23778
2018-10-23 20:24:08.265151 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 288 remain: 3808 writtenBytes: 27874
2018-10-23 20:24:08.277661 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 3432 remain: 664 writtenBytes: 31970
2018-10-23 20:24:08.278022 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 784 remain: 3312 writtenBytes: 36066
2018-10-23 20:24:08.278264 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 2480 remain: 1616 writtenBytes: 40162
2018-10-23 20:24:08.278580 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 1280 remain: 2816 writtenBytes: 44258
2018-10-23 20:24:08.278913 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 1528 remain: 2568 writtenBytes: 48354
2018-10-23 20:24:08.279254 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 1776 remain: 2320 writtenBytes: 52450
2018-10-23 20:24:08.279402 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 576 remain: 3520 writtenBytes: 56546
2018-10-23 20:24:08.279920 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 824 remain: 3272 writtenBytes: 60642
2018-10-23 20:24:08.280905 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 64738
2018-10-23 20:24:08.280948 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 2768 remain: 1328 writtenBytes: 68834
2018-10-23 20:24:08.281217 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 3016 remain: 1080 writtenBytes: 72930
2018-10-23 20:24:08.281506 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 3264 remain: 832 writtenBytes: 77026
2018-10-23 20:24:08.281948 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 2064 remain: 2032 writtenBytes: 81122
2018-10-23 20:24:08.297685 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 3760 remain: 336 writtenBytes: 85218
2018-10-23 20:24:08.298170 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4008 remain: 88 writtenBytes: 89314
2018-10-23 20:24:08.298362 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 2808 remain: 1288 writtenBytes: 93410
2018-10-23 20:24:08.298919 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 3056 remain: 1040 writtenBytes: 97506
2018-10-23 20:24:08.299576 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 101602
2018-10-23 20:24:08.299641 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 656 remain: 3440 writtenBytes: 105698
2018-10-23 20:24:08.300041 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 2352 remain: 1744 writtenBytes: 109794
2018-10-23 20:24:08.300479 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 2600 remain: 1496 writtenBytes: 113890
2018-10-23 20:24:08.301340 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 2848 remain: 1248 writtenBytes: 117986
2018-10-23 20:24:08.303384 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 122082
2018-10-23 20:24:08.303646 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 1920 remain: 2176 writtenBytes: 126178
2018-10-23 20:24:08.303926 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 130274
2018-10-23 20:24:08.304004 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 134370
2018-10-23 20:24:08.304376 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 138466
2018-10-23 20:24:08.304540 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 4096 remain: 0 writtenBytes: 142562
2018-10-23 20:24:08.304737 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 768 remain: 3328 writtenBytes: 146658
2018-10-23 20:24:08.318544 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 2464 remain: 1632 writtenBytes: 150754
2018-10-23 20:24:08.320273 [TRACE] [HttpRequest.cpp:342] [downloadFile] sockets::read(): nread: 1134 remain: 2962 writtenBytes: 154850
2018-10-23 20:24:08.320374 [DEBUG] [HttpRequest.cpp:351] [downloadFile] sockets::read finish
2018-10-23 20:24:08.320460 [DEBUG] [HttpRequest.cpp:362] [downloadFile]  writtenBytes 155984

---