SQL Server FileStream

　　以往我們對文件管理有兩種方法：

1. 數據庫只保存文件的路徑，具體的文件保存在文件服務器(NFS)上，使用時，編程實現從文件服務器讀取文件；
2. 將文件直接以varbinary(max)或image數據類型保存在數據庫中。

　　上面兩種文件存放方式都有問題：第一種方法因為會訪問磁盤，故受I/O影響性能不是很好，而且不能很好的進行文件備份；第二種方法雖然解決了文件備份（數據庫的備份）問題，但是由於字段的字節數太大，對數據庫本身也會造成影響，性能也很低下。

　　微軟在SQL Server 2008推出了一種新的方式 - FileStream，它不是一種新的數據類型，而是一種技術，它使SQL Server數據庫引擎和NTFS文件系統成為了一個整體，它結合了上面兩種方式的優點：FileStream使用NT系統來緩存文件數據，而對文件數據的操作可使用Transact-SQL語句對其進行插入、更新、查詢、搜索和備份。

　　https://msdn.microsoft.com/en-us/library/gg471497(v=sql.110).aspx

　　FILESTREAM integrates the SQL Server Database Engine with an NTFS file system by storing varbinary(max) binary large object (BLOB) data as files on the file system. Transact-SQL statements can insert, update, query, search, and back up FILESTREAM data. Win32 file system interfaces provide streaming access to the data.

　　FILESTREAM uses the NT system cache for caching file data. This helps reduce any effect that FILESTREAM data might have on Database Engine performance. The SQL Server buffer pool is not used; therefore, this memory is available for query processing.

 

一、FileStream配置

1. 配置SQL Server安裝實例：Start -> All Programs -> Microsoft SQL Server 2008 R2 -> Configuration Tools -> SQL Server Configuration Manager

　　右擊屬性，切換到FILESTREAM標簽，勾選如下配置

　　2. 打開SQL Server，並配置如下

 　　以上也可以通過如下腳本執行：

Exec sp_configure filesteam_access_level, 2
RECONFIGURE

　　最后重啟SQL Server Service

二、實例展示

　　創建FileStream類型文件/組

　　FILESTREAM data must be stored in FILESTREAM filegroups. A FILESTREAM filegroup is a special filegroup that contains file system directories instead of the files themselves. These file system directories are called data containers. Data containers are the interface between Database Engine storage and file system storage.

--Create filestreamgroup 
ALTER DATABASE [Archive]
ADD FILEGROUP [FileStreamGroup] CONTAINS FILESTREAM 
GO

--Create filestream and association with filestreamgroup above
ALTER DATABASE [Archive]
ADD FILE ( NAME = N'FileStream', FILENAME = N'D:\Company\Data\SQL Server\FileStream') TO FILEGROUP [FileStreamGroup]
GO

　　filestream.hdr 文件是 FILESTREAM 容器的頭文件。filestream.hdr 文件是重要的系統文件。它包含 FILESTREAM 標頭信息。請勿刪除或修改此文件。

 

　　創建測試表（注意：如果表包含FILESTREAM列，則每一行都必須具有唯一的行ID）

--Create table
CREATE TABLE Archive.dbo.Attachment (
    [ID] [UNIQUEIDENTIFIER] ROWGUIDCOL NOT NULL PRIMARY KEY,
    [FileName] NVARCHAR(100) NULL,
    [CreateUser] NVARCHAR(100) NULL,
    [CreateDatetime] DATETIME NULL,
    [Content] VARBINARY(MAX) FILESTREAM NULL  
)
FILESTREAM_ON [FileStreamGroup] 

　　插入一些測試數據

--Insert some records
INSERT INTO Attachment VALUES 
(NEWID(),'File Name 1','shg.cpan', GETDATE(),NULL),
(NEWID(),'File Name 1','shg.cpan', GETDATE(),CAST('' AS VARBINARY(MAX))),
(NEWID(),'File Name 1','shg.cpan', GETDATE(),CAST('This is a attachment, which contains all introduction for filestream' AS VARBINARY(MAX)))

　　從前台插入一些數據

using (SqlConnection conn = new SqlConnection("server=10.7.15.172;database=Archive;uid=sa;pwd=1234;Connect Timeout=180"))
{
    conn.Open();
    using (SqlCommand cmd = conn.CreateCommand())
    {
        string id = Guid.NewGuid().ToString();
        cmd.CommandText = "INSERT INTO Attachment VALUES('" + id + "','File Name 2','shg.cpan','" + DateTime.Now + "',@content)";
        SqlParameter param = new SqlParameter("@content", SqlDbType.VarBinary, 1000000000);
        param.Value = File.ReadAllBytes(@"D:\Folder\131 u_ex151207.log");
        cmd.Parameters.Add(param);
        cmd.ExecuteNonQuery();
    }
    conn.Close();
}

　　檢索數據 

SELECT DATALENGTH(CONTENT)/(1024.0 * 1024.0) AS MB,* FROM ATTACHMENT

　　結果

　　文件系統

　　

　　上面的文件都是上傳的真實文件，只不過沒有后綴，如果重命名加上后綴，即可讀取，如最后一個是excel文件，加上.xls，即可用Excel軟件打開此文件。

 

　　下面我們再插入一條記錄

INSERT INTO Attachment VALUES 
(NEWID(),'Win32API','shg.cpan', GETDATE(),CAST('This is a attachment, which contains all introduction for filestream' AS VARBINARY(MAX)))

　　

　　文件名00000016-0000016e-000c是如何和數據關聯的呢

　　DBCC IND (Archive, Attachment, -1)

　　我們看一下PagePID為110的頁情況（PageType為1表明是數據頁）

DBCC TRACEON (3604);
DBCC PAGE (Archive, 1, 110, 3);
GO

　　看到了什么？CreateLSN即是我們在文件系統中看到的文件名00000016:0000016e:000c，這樣數據庫中的紀錄就和文件聯系起來了

 

三、使用 Win32 管理 FILESTREAM 數據

https://technet.microsoft.com/zh-cn/library/cc645940(v=sql.105).aspx

可以使用 Win32 在 FILESTREAM BLOB 中讀取和寫入數據。您需要執行以下步驟：

1. 讀取 FILESTREAM 文件路徑;
2. 讀取當前事務上下文;
3. 獲取 Win32 句柄，並使用該句柄在 FILESTREAM BLOB 中讀取和寫入數據

讀取 FILESTREAM 文件路徑

DECLARE @filePath varchar(max)

SELECT @filePath = Content.PathName()
FROM Archive.dbo.Attachment
WHERE FileName = 'Win32API'
PRINT @filepath

\\CHUNTING-PC\MSSQLSERVER\v1\Archive\dbo\Attachment\%!Content\583FFDB4-921B-4340-8247-130174488DC8

 

讀取當前事務上下文

DECLARE @txContext varbinary(max)

BEGIN TRANSACTION
SELECT @txContext = GET_FILESTREAM_TRANSACTION_CONTEXT()
PRINT @txContext
COMMIT

0x9D84E776FD943D419C99727C7AAA5B00

 

獲取 Win32 句柄，並使用該句柄在 FILESTREAM BLOB 中讀取和寫入數據

若要獲取 Win32 文件句柄，請調用 OpenSqlFilestream API。此 API 是從 sqlncli.dll 文件中導出的。可以將返回的句柄傳遞給以下任何 Win32 API：ReadFile、WriteFile、TransmitFile、SetFilePointer、SetEndOfFile 或 FlushFileBuffers。下面的示例說明了如何獲取 Win32 文件句柄並使用它在 FILESTREAM BLOB 中讀取和寫入數據。

using System;
using System.Collections.Generic;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using System.IO;
using System.Linq;
using System.Security.AccessControl;
using System.Text;

namespace FileStreamConsoleApp
{
    class Program
    {
        static void Main(string[] args)
        {
            SqlConnection sqlConnection = new SqlConnection("server=10.7.15.172;database=Archive;Connect Timeout=180;Integrated Security=true");

            SqlCommand sqlCommand = new SqlCommand();
            sqlCommand.Connection = sqlConnection;

            try
            {
                sqlConnection.Open();

                //The first task is to retrieve the file path of the SQL FILESTREAM BLOB that we want to access in the application.
                sqlCommand.CommandText = "SELECT Content.PathName() FROM Archive.dbo.Attachment WHERE FileName = 'Win32API'";

                String filePath = null;

                Object pathObj = sqlCommand.ExecuteScalar();
                if (DBNull.Value != pathObj)
                {
                    filePath = (string)pathObj;
                }
                else
                {
                    throw new System.Exception("Chart.PathName() failed to read the path name for the Chart column.");
                }

                //The next task is to obtain a transaction context. All FILESTREAM BLOB operations occur within a transaction context to maintain data consistency.

                //All SQL FILESTREAM BLOB access must occur in a transaction. MARS-enabled connections have specific rules for batch scoped transactions,
                //which the Transact-SQL BEGIN TRANSACTION statement violates. To avoid this issue, client applications should use appropriate API facilities for transaction management, 
                //management, such as the SqlTransaction class.

                SqlTransaction transaction = sqlConnection.BeginTransaction("mainTranaction");
                sqlCommand.Transaction = transaction;

                sqlCommand.CommandText = "SELECT GET_FILESTREAM_TRANSACTION_CONTEXT()";

                Object obj = sqlCommand.ExecuteScalar();
                byte[] txContext = (byte[])obj;


                //The next step is to obtain a handle that can be passed to the Win32 FILE APIs.
                SqlFileStream sqlFileStream = new SqlFileStream(filePath, txContext, FileAccess.ReadWrite);

                byte[] buffer = new byte[512];

                int numBytes = 0;

                //Write the string, "EKG data." to the FILESTREAM BLOB. In your application this string would be replaced with the binary data that you want to write.

                string someData = "EKG data.";
                Encoding unicode = Encoding.GetEncoding(0);

                sqlFileStream.Write(unicode.GetBytes(someData.ToCharArray()), 0, someData.Length);

                //Read the data from the FILESTREAM BLOB.

                sqlFileStream.Seek(0L, SeekOrigin.Begin);

                numBytes = sqlFileStream.Read(buffer, 0, buffer.Length);

                string readData = unicode.GetString(buffer);

                if (numBytes != 0)
                {
                    Console.WriteLine(readData);
                }
                //Because reading and writing are finished, FILESTREAM must be closed. This closes the c# FileStream class, 
                //but does not necessarily close the the underlying FILESTREAM handle. 
                sqlFileStream.Close();

                //The final step is to commit or roll back the read and write operations that were performed on the FILESTREAM BLOB.

                sqlCommand.Transaction.Commit();
            }
            catch (System.Exception ex)
            {
                Console.WriteLine(ex.ToString());
            }
            finally
            {
                sqlConnection.Close();
            }
            Console.ReadKey();
        }
    }
}

注意此處的連接方式，必須是integrated security，如果設置成如下

SqlConnection conn = new SqlConnection("server=10.7.15.172;database=Archive;uid=sa;pwd=1234;Connect Timeout=180")

會報錯提示權限問題

 

　　這是因為實際的文件保存在服務器的硬盤上，讀取時讀到的文件是"\\xxxx"，還是通過網上鄰居，所以要使用信任連接

Only the account under which the SQL Server service account runs is granted NTFS permissions to the FILESTREAM container. We recommend that no other account be granted permissions on the data container.

Note

SQL logins will not work with FILESTREAM containers. Only NTFS authentication will work with FILESTREAM containers.

另一篇文章 http://stackoverflow.com/questions/1398404/sql-server-filestream-access-denied

插入Attachment的完整C#語句

using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using System.IO;
using System.Transactions;

namespace FileStreamConsoleApp
{
    public class ArchiveAttachment
    {
        private const string ConnStr = "server=10.7.15.172;database=Archive;Connect Timeout=180;Integrated Security=true";

        public static void InsertAttachment(string Id, string fileName, string fileFullName, string createUser, DateTime createDatetime)
        {
            string InsertTSql = @"INSERT INTO Attachment(Id, FileName, CreateUser, CreateDatetime, Content) VALUES(@Id, @FileName, @CreateUser, @CreateDatetime, 0x); 
                                    SELECT Content.PathName(), GET_FILESTREAM_TRANSACTION_CONTEXT() FROM Attachment WHERE Id = @Id";

            string serverPath;
            byte[] serverTransaction;

            using (TransactionScope ts = new TransactionScope())
            {
                using (SqlConnection conn = new SqlConnection(ConnStr))
                {
                    conn.Open();

                    using (SqlCommand cmd = new SqlCommand(InsertTSql, conn))
                    {
                        cmd.Parameters.Add("@Id", SqlDbType.NVarChar).Value = Id;
                        cmd.Parameters.Add("@FileName", SqlDbType.NVarChar).Value = fileName;
                        cmd.Parameters.Add("@CreateUser", SqlDbType.NVarChar).Value = createUser;
                        cmd.Parameters.Add("@createDatetime", SqlDbType.DateTime).Value = createDatetime;
                        using (SqlDataReader sdr = cmd.ExecuteReader())
                        {
                            sdr.Read();
                            serverPath = sdr.GetSqlString(0).Value;
                            serverTransaction = sdr.GetSqlBinary(1).Value;
                            sdr.Close();
                        }
                    }
                    SaveAttachment(fileFullName, serverPath, serverTransaction);
                }
                ts.Complete();
            }
        }

        private static void SaveAttachment(string clientPath, string serverPath, byte[] serverTransaction)
        {
            const int BlockSize = 1024 * 512;

            using (FileStream source = new FileStream(clientPath, FileMode.Open, FileAccess.Read))
            {
                using (SqlFileStream dest = new SqlFileStream(serverPath, serverTransaction, FileAccess.Write))
                {
                    byte[] buffer = new byte[BlockSize];
                    int bytesRead;
                    while ((bytesRead = source.Read(buffer, 0, buffer.Length)) > 0)
                    {
                        dest.Write(buffer, 0, bytesRead);
                        dest.Flush();
                    }
                    dest.Close();
                }
                source.Close();
            }
        }
    }
}

 

 

四、使用場合

　　請注意以下事項：

　　並不是所有的文件存儲都適合使用FileStream，如果所存儲的文件對象平均大於1MB考慮使用FileStream，否則對於較小的文件對象，以varbinary(max)BLOB存儲在數據庫中通常會提供更為優異的流性能

https://msdn.microsoft.com/en-us/library/gg471497(v=sql.110).aspx

 

http://stackoverflow.com/questions/13420305/storing-files-in-sql-server#

http://research.microsoft.com/apps/pubs/default.aspx?id=64525

 

一些參考鏈接： 

使用FILESTREAM最佳實踐：https://technet.microsoft.com/zh-cn/library/dd206979(v=sql.105).aspx

設計和實現 FILESTREAM 存儲：https://technet.microsoft.com/zh-cn/library/bb895234%28v=sql.105%29.aspx?f=255&MSPPError=-2147217396

二進制大型對象 (Blob) 數據 (SQL Server)：https://technet.microsoft.com/zh-cn/library/bb895234(v=sql.110).aspx

Files and Filegroups Architecture：https://msdn.microsoft.com/en-us/library/ms179316.aspx?f=255&MSPPError=-2147217396

FILESTREAM Storage in SQL Server 2008：https://msdn.microsoft.com/en-us/library/hh461480.aspx?f=255&MSPPError=-2147217396