winform多線程分塊下載文件

使用HttpWebRequst.分塊下載思路:

(為什么用它?原因在於: request.AddRange(startPos, endPos); 可以設置下載的起始位置)

1.先計算每個線程下載塊的平均值,

     //計算每條線程下載的數據長度
      this.block = (this.fileSize % this.threads.Length) == 0 ? this.fileSize / this.threads.Length : this.fileSize / this.threads.Length + 1;
                  

2.計算每個線程的起始位置

 int startPos = (int)(block * (threadId - 1) + downLength);//開始位置
 int endPos = (int)(block * threadId - 1);//結束位置

3.當前下載塊失敗后.重新下載當前塊內容,並設置當前失敗標記,以便重新啟動該線程

this.downLength = -1;

具體實現

創建一個接口.獲取當前下載文件的總和

public  interface  IDownloadProgressListener
    {
         void OnDownloadSize(long size);
    }

實現上面的接口,並創建一個委托.告知調用者當前下載的總和

   public class DownloadProgressListener : IDownloadProgressListener
    {
        public delegate void dlgSendMsg(DownMsg msg);
        public dlgSendMsg doSendMsg = null;
        public void OnDownloadSize(long size)
        {
            DownMsg msg = new DownMsg();
            msg.speed = (float)(size - Form1.presize); //下載速度
            msg.size = size; //下載總量
            Form1.presize = size;
            msg.tag = 1; 
            if (doSendMsg != null) doSendMsg(msg);//通知具體調用者下載進度
        }
    }
    public class DownMsg
    {
        public int tag { get; set; }
        public long size { get; set; }
        public float speed { get; set; }
    }

每個線程調用當前ThreadRun()方法

 public class DownloadThread
    {
        private string saveFilePath;
        private string  downUrl;
        private long block;
        private int threadId = -1;
        private long downLength;
        private bool finish = false;
        private FileDownloader downloader;

        public DownloadThread(FileDownloader downloader, string downUrl, string saveFile, long block, long downLength, int threadId)
        {
            this.downUrl = downUrl;
            this.saveFilePath = saveFile;
            this.block = block;
            this.downloader = downloader;
            this.threadId = threadId;
            this.downLength = downLength;
        }


        public void ThreadRun()
        {
            //task
            Thread td = new Thread(new ThreadStart(() =>
            {
                 if (downLength < block)//未下載完成
                {
                    try
                    {
                        int startPos = (int)(block * (threadId - 1) + downLength);//開始位置
                        int endPos = (int)(block * threadId - 1);//結束位置
                        Console.WriteLine("Thread " + this.threadId + " start download from position " + startPos + "  and endwith " + endPos);
                        HttpWebRequest request = (HttpWebRequest)WebRequest.Create(downUrl);
                        request.Referer = downUrl.ToString();
                        request.Method = "GET";
                        request.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; SV1; .NET CLR 2.0.1124)";
                        request.AllowAutoRedirect = false;
                        request.ContentType = "application/octet-stream";
                        request.Accept = "image/gif, image/jpeg, image/pjpeg, image/pjpeg, application/x-shockwave-flash, application/xaml+xml, application/vnd.ms-xpsdocument, application/x-ms-xbap, application/x-ms-application, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*";
                        request.Timeout = 10 * 1000;
                        request.AllowAutoRedirect = true;
                        request.AddRange(startPos, endPos);
                        //Console.WriteLine(request.Headers.ToString()); //輸出構建的http 表頭
                        HttpWebResponse response = (HttpWebResponse)request.GetResponse();
                        WebResponse wb = request.GetResponse();
                        using (Stream _stream = wb.GetResponseStream())
                        {
                            byte[] buffer = new byte[1024 * 50]; //緩沖區大小
                            long offset = -1;
                            using (Stream threadfile = new FileStream(this.saveFilePath, FileMode.Open, FileAccess.ReadWrite, FileShare.ReadWrite)) //設置文件以共享方式讀寫,否則會出現當前文件被另一個文件使用.
                            {
                                threadfile.Seek(startPos, SeekOrigin.Begin); //移動文件位置
                                while ((offset = _stream.Read(buffer, 0, buffer.Length)) != 0)
                                {
                                    //offset 實際下載流大小
                                    downloader.append(offset); //更新已經下載當前總文件大小
                                    threadfile.Write(buffer, 0, (int)offset);
                                    downLength += offset;  //設置當前線程已下載位置
                                    downloader.update(this.threadId, downLength);
                                }
                                threadfile.Close(); //using 用完后可以自動釋放..手動釋放一遍.木有問題的(其實是多余的)
                                _stream.Close();
                                Console.WriteLine("Thread " + this.threadId + " download finish");
                                this.finish = true;
                            }
                        }
                    }
                    catch (Exception e)
                    {
                        this.downLength = -1;
                        Console.WriteLine("Thread " + this.threadId + ":" + e.Message);
                    }
                } 
            }));
            td.IsBackground = true;
            td.Start();
        }
        /// <summary>
        /// 下載是否完成
        /// </summary>
        /// <returns></returns>
        public bool isFinish()
        {
            return finish;
        }
        /// <summary> 
        ///  已經下載的內容大小  
        /// </summary>
        /// <returns>如果返回值為-1,代表下載失敗</returns>
        public long getDownLength()
        {
            return downLength;
        } 
    }

獲取下載文件大小,並分配每個線程的任務

    public class FileDownloader
    {
        /// <summary>
        /// 已下載文件長度
        /// </summary>
        private long downloadSize = 0;
        /// <summary>
        /// 原始文件長度
        /// </summary>
        private long fileSize = 0;
        /// <summary>
        /// 線程數
        /// </summary>
        private DownloadThread[] threads;
        /// <summary>
        /// 本地保存文件
        /// </summary>
        private string saveFile;
        /// <summary>
        /// 緩存各線程下載的長度
        /// </summary>
        public Dictionary<int, long> data = new Dictionary<int, long>();
        /// <summary>
        /// 每條線程下載的長度
        /// </summary>
        private long block;
        /// <summary>
        /// 下載路徑
        /// </summary>
        private String downloadUrl;
        /// <summary>
        ///  獲取線程數
        /// </summary>
        /// <returns> 獲取線程數</returns>
        public int getThreadSize()
        {
            return threads.Length;
        }
        /// <summary>
        ///   獲取文件大小
        /// </summary>
        /// <returns>獲取文件大小</returns>
        public long getFileSize()
        {
            return fileSize;
        }
        /// <summary>
        /// 累計已下載大小
        /// </summary>
        /// <param name="size">累計已下載大小</param>
        public void append(long size)
        {
            lock (this)  //鎖定同步..............線程開多了竟然沒有同步起來.文件下載已經完畢了,下載總數量卻不等於文件實際大小,找了半天原來這里錯誤的
            {
                downloadSize += size;
            }

        }
        /// <summary>
        /// 更新指定線程最后下載的位置
        /// </summary>
        /// <param name="threadId">threadId 線程id</param>
        /// <param name="pos">最后下載的位置</param>
        public void update(int threadId, long pos)
        {
            if (data.ContainsKey(threadId))
            {
                this.data[threadId] = pos;
            }
            else
            {
                this.data.Add(threadId, pos);
            }
        }

        /// <summary>
        /// 構建下載准備,獲取文件大小
        /// </summary>
        /// <param name="downloadUrl">下載路徑</param>
        /// <param name="fileSaveDir"> 文件保存目錄</param>
        /// <param name="threadNum">下載線程數</param>
        public FileDownloader(string downloadUrl, string fileSaveDir, int threadNum)
        {
            try
            {
                //構建http 請求
                this.downloadUrl = downloadUrl;
                if (!Directory.Exists(fileSaveDir)) Directory.CreateDirectory(fileSaveDir);
                this.threads = new DownloadThread[threadNum];
                HttpWebRequest request = (HttpWebRequest)WebRequest.Create(downloadUrl);
                request.Referer = downloadUrl.ToString();
                request.Method = "GET";
                request.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; SV1; .NET CLR 2.0.1124)";
                request.ContentType = "application/octet-stream";
                request.Accept = "image/gif, image/jpeg, image/pjpeg, image/pjpeg, application/x-shockwave-flash, application/xaml+xml, application/vnd.ms-xpsdocument, application/x-ms-xbap, application/x-ms-application, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*";
                request.Timeout = 20 * 1000;
                request.AllowAutoRedirect = true;

                using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
                {
                    if (response.StatusCode == HttpStatusCode.OK)
                    {

                        this.fileSize = response.ContentLength;//根據響應獲取文件大小
                        if (this.fileSize <= 0) throw new Exception("獲取文件大小失敗");
                        string filename = Uri.UnescapeDataString(Path.GetFileName(downloadUrl));//獲取文件名稱 uri 解碼中文字符
                        if (filename.Length == 0) throw new Exception("獲取文件名失敗");
                        this.saveFile = Path.Combine(fileSaveDir, filename); //構建保存文件 
                        //計算每條線程下載的數據長度
                        this.block = (this.fileSize % this.threads.Length) == 0 ? this.fileSize / this.threads.Length : this.fileSize / this.threads.Length + 1;
                    }
                    else
                    {
                        throw new Exception("服務器返回狀態失敗,StatusCode:" + response.StatusCode);
                    }
                }
            }
            catch (Exception e)
            {
                Console.WriteLine(e.Message);
                throw new Exception("無法連接下載地址");
            }
        }

        /// <summary>
        /// 開始下載文件
        /// </summary>
        /// <param name="listener">監聽下載數量的變化,如果不需要了解實時下載的數量,可以設置為null</param>
        /// <returns>已下載文件大小</returns>
        public long download(IDownloadProgressListener listener)
        {
            try
            {
                using (FileStream fstream = new FileStream(this.saveFile, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite))
                {
                    if (this.fileSize > 0) fstream.SetLength(this.fileSize);
                    fstream.Close();
                }
                if (this.data.Count() != this.threads.Length)
                {
                    this.data.Clear();
                    for (int i = 0; i < this.threads.Length; i++)
                    {
                        this.data.Add(i + 1, 0);//初始化每條線程已經下載的數據長度為0
                    }
                }
                for (int i = 0; i < this.threads.Length; i++)
                {//開啟線程進行下載
                    long downLength = this.data[i + 1];
                    if (downLength < this.block && this.downloadSize < this.fileSize)
                    {//判斷線程是否已經完成下載,否則繼續下載    +

                        // Console.WriteLine("threads" + i.ToString() + ",下載塊" + this.block.ToString() + "    " + this.data[i + 1].ToString() + "              " + downloadSize.ToString());
                        this.threads[i] = new DownloadThread(this, downloadUrl, this.saveFile, this.block, this.data[i + 1], i + 1);
                        this.threads[i].ThreadRun();

                    }
                    else
                    {
                        this.threads[i] = null;
                    }
                }
                bool notFinish = true;//下載未完成
                while (notFinish)
                {// 循環判斷所有線程是否完成下載
                    Thread.Sleep(900);
                    notFinish = false;//假定全部線程下載完成
                    for (int i = 0; i < this.threads.Length; i++)
                    {
                        if (this.threads[i] != null && !this.threads[i].isFinish())
                        {//如果發現線程未完成下載
                            notFinish = true;//設置標志為下載沒有完成
                            if (this.threads[i].getDownLength() == -1)
                            {//如果下載失敗,再重新下載
                                this.threads[i] = new DownloadThread(this, downloadUrl, this.saveFile, this.block, this.data[i + 1], i + 1);
                                this.threads[i].ThreadRun();
                            }
                        }
                    }
                    if (listener != null)
                    {
                        listener.OnDownloadSize(this.downloadSize);//通知目前已經下載完成的數據長度
                        Console.WriteLine(this.downloadSize);
                    }
                }
            }
            catch (Exception e)
            {
                Console.WriteLine(e.Message);
                throw new Exception("下載文件錯誤");
            }
            return this.downloadSize;
        }
    }

主窗體調用下載

public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }
        Stopwatch stop;
        public static long presize = 0;
        private void btnDown_Click(object sender, EventArgs e)
        {
            stop = new Stopwatch();
            stop.Start(); //計時
            string path = Uri.EscapeUriString(txtUrl.Text.Trim());
            string dir = @"G:\test"; 
            bar.Value = 0;
            btnDown.Enabled = false;
            Task tsk = new Task(() =>
            {
                download(path, dir);
            });
            tsk.Start();
            lbPercent.Visible = true;
            lbSpeed.Visible = true;
        }
        private void download(string path, string dir)
        {

            try
            {
                FileDownloader loader = new FileDownloader(path, dir, (int)NumThreads.Value);
                loader.data.Clear();

                this.Invoke(new MethodInvoker(() =>
                {
                    bar.Maximum = (int)loader.getFileSize();
                }));
                DownloadProgressListener linstenter = new DownloadProgressListener();
                linstenter.doSendMsg = new DownloadProgressListener.dlgSendMsg(SendMsgHander);
                loader.download(linstenter);
            }
            catch (Exception ex)
            {
                DownMsg msg = new DownMsg();
                msg.tag = -1;
                SendMsgHander(msg);
                Console.WriteLine(ex.Message);
            }
        }
        private void SendMsgHander(DownMsg msg)
        {
            switch (msg.tag)
            {
                case 1:
                    this.Invoke(new MethodInvoker(() =>
                    {
                        bar.Value = (int)msg.size;
                        float count = (float)bar.Value / (float)bar.Maximum;
                        lbPercent.Text = ((int)(count * 100)).ToString() + "%";
                        lbSpeed.Text = ((int)msg.speed / 1024).ToString() + "K";
                        Console.WriteLine(msg.size + "               " + msg.speed);
                        if (bar.Maximum == msg.size)
                        {
                            stop.Stop();
                            string str = "使用線程數 " + NumThreads.Value.ToString() + "   耗時:" + stop.ElapsedMilliseconds.ToString();
                            Console.WriteLine(str);
                            lbSpeed.Visible = false;
                            btnDown.Enabled = true;
                            MessageBox.Show(str, "下載完畢"); 
                        }
                    }));
                    break;
                case -1:
                    MessageBox.Show("下載失敗");
                    break;
            }
        }

        private void Form1_Load(object sender, EventArgs e)
        { 
            lbPercent.Visible = false;
            lbSpeed.Visible = false;
        }


    }

窗體界面: 下載一個132M的某軟件為例

 

下載中.....

下載完畢后

 

整個下載完畢,並可以執行下載文件.說明分塊下載成功.

ps:看到最后一張圖沒有人認為我在打廣告吧.那只能呵呵呵呵呵呵呵呵呵了

 

 編碼調試時,遇見的問題

1.下載中文路徑處理(已處理).

2.計算總下載字節數同步問題(調試了好久才發覺是同步問題)(已處理)

3.並不是開的線程越多,下載的速度越快.(在debug的時候.會拋出大量的請求超時) ,3-5個差不多了

ps:偽原創啦.原創代碼是java的.

下載地址: 源代碼 

vs2010  framework 4.0