如何對SQL Server實例執行性能測試

介紹 (Introduction)

目標聽眾 (Intended audience)

This document is intended for application developers and database administrators who plan to evaluate performance for any computer hosting a SQL Server instance.

本文檔適用於計划評估托管SQL Server實例的任何計算機的性能的應用程序開發人員和數據庫管理員。

語境 (Context)

Let’s say a new physical server has been delivered and we have to test its performance in regards to its future destination, which is a database server. For newbies, there are a lot of questions that emerge:

假設已經交付了一個新的物理服務器，並且我們必須針對其未來的目的地（即數據庫服務器）測試其性能。 對於新手，會出現很多問題：

• What has to be tested?
  必須測試什么？
• What solutions can be used?
  可以使用什么解決方案？
• What information do they provide?
  他們提供什么信息？
• How to use them?
  如何使用它們？
• Is there a tool that does all the tests I need to get a good insight into server performance or is it better to use two or more tools in conjunction?
  是否有工具可以完成我所需的所有測試，以便深入了解服務器性能，還是結合使用兩個或多個工具更好？
• …

   

Most questions out there should lead us to following steps:

那里的大多數問題應引導我們執行以下步驟：

1. Definition of the scope of the test 
   定義測試范圍
2. Analysis and selection of the tools available to achieve test purposes
   分析和選擇可用於實現測試目的的工具
3. If not already done, creation of a performance report template
   如果尚未完成，請創建效果報告模板
4. Usage of this report template to build actual performance tests report for the server
   使用此報告模板為服務器構建實際性能測試報告
5. Actual performance test with results documentation in report
   實際性能測試，報告中包含結果文檔
6. Analysis of performance test results (compared to a baseline)
   性能測試結果分析（與基准相比）
7. Feedback on report template for continuous improvement
   對報告模板的反饋，以進行持續改進

 

There is a bunch of tools that we can use to do a performance test:

我們可以使用許多工具來進行性能測試：

• 1. PassMark PerformanceTest    
     
  2. CPU-z
  3. CrystalDisk Mark
  4. Phoronix Test Suite (linux下使用)
  5. … 
1. Sqlio.exe (已經廢了，可改用diskspd.exe)
2. HammerDb
3. OStress
4. …

In conjunction with these tools, it’s worth using a database monitoring tool so that we can collect and historize performance statistics. Here are some of the tools that can be used:

結合使用這些工具，值得使用數據庫監視工具，以便我們可以收集和歷史化性能統計數據。 以下是一些可以使用的工具：

• sp_BlitzFirst®
• Windows Performance Monitor (perfmon)
• Spotlight 聚光燈
• Idera Diagnostic Manager

Here are the choices made for the purpose of this article:

以下是為本文目的所做的選擇：

1. PassMark PerformanceTest to get a first overview of overall server performance 
   PassMark PerformanceTest獲得服務器總體性能的初步概覽
2. Benchmarking with HammerDb and collecting performance statistics with sp_BlitzFirst®.
   使用HammerDb進行基准測試，並使用sp_BlitzFirst®收集性能統計信息。

使用PassMark PerformanceTest獲取服務器性能概述 (Getting server performance overview with PassMark PerformanceTest)

The latest version for this tool is the 9^th and can be downloaded for testing (and purchase) on its dedicated web page.

該工具的最新版本是9 ^號 ，可以在其專用網頁上下載以進行測試（和購買）。

The installation is pretty straightforward, so it won’t be reviewed here.

安裝非常簡單，因此這里不再進行審查。

Once PerformanceTest is installed, we can launch it. We will get following window.

一旦安裝PerformanceTest，我們就可以啟動它。 我們將得到以下窗口。

As we can see, there are multiple options available. We will just click on “Run Benchmark” to run all the tests. We are prompted following message to which we can answer “Yes”.

我們可以看到，有多個可用選項。 我們只需單擊“運行基准”即可運行所有測試。 在提示您可以回答“是”的消息后，我們會被提示。

 

And the benchmark begins…

基准開始……

Once the tests are done, you will get a score that you can compare to others or between computers in order to evaluate performance.

測試完成后，您將獲得一個分數，可以與他人進行比較或在計算機之間進行比較，以評估性能。

 

Note

注意

In this example, the 3D Graphics Mark is grayed because I did the test on a virtual machine without 3D acceleration.

在此示例中，“ 3D圖形標記”顯示為灰色，因為我在沒有3D加速的虛擬機上進行了測試。

If we get to detailed test data, we can see that there is a preamble testing for database operations:

如果獲得詳細的測試數據，我們可以看到對數據庫操作進行了前導測試：

 

 

 

HammerDb的性能測試 (Performance testing with HammerDb)

HammerDb安裝過程 (HammerDb Installation Process)

Go to https://www.hammerdb.com/download.html to get back the latest version of HammerDB.

請訪問https://www.hammerdb.com/download.html以獲取HammerDB的最新版本。

The installation process is straightforward and won’t be discussed.

安裝過程非常簡單，不再討論。

SQL Server性能統計信息收集 (SQL Server Performance statistics collection)

集合存儲過程 (Collection stored procedure)

While HammerDb is running, we will collect performance statistics using the sp_BlitzFirst® stored procedure from the open source project called SQL Server First Responder Kit.

在HammerDb運行時，我們將使用sp_BlitzFirst®存儲過程從名為SQL Server First Responder Kit的開源項目中收集性能統計信息。

So, the first step is to get back latest version of this stored procedure and set this up to the target SQL Server instance we will be testing.

因此，第一步是獲取此存儲過程的最新版本，並將其設置為我們將要測試的目標SQL Server實例。

Preferably, this procedure should be installed in a database reserved to DBA usage, let’s call it [DBAMGMT].

最好將此過程安裝在為DBA使用保留的數據庫中，我們將其稱為[DBAMGMT]。

We can check the stored procedure actually works with following statement:

我們可以使用以下語句檢查存儲過程是否真正起作用：

1.  
   EXEC dbo.sp_BlitzFirst
2.  
          @Seconds                     = 5,
3.  
          @OutputDatabaseName          = 'DBAMGMT',
4.  
          @OutputSchemaName            = 'Monitoring',
5.  
          @OutputTableName             = 'BlitzFirstResults',
6.  
          @OutputTableNameFileStats    = 'BlitzFirstResults_FileStats',
7.  
          @OutputTableNamePerfmonStats = 'BlitzFirstResults_PerfmonStats',
8.  
          @OutputTableNameWaitStats    = 'BlitzFirstResults_WaitStats'
9.  
   ;

使用SQL Agent Job自動執行收集 (Automating collection with a SQL Agent Job)

This statement will be used to define a SQL Server Agent Job that will run every 10 seconds during the whole performance test.

該語句將用於定義一個SQL Server代理作業，該作業在整個性能測試期間每10秒運行一次。

Here is the code to create such a job:

這是創建此類工作的代碼：

1.  
   USE [msdb]
2.  
   GO
3.  
    
4.  
   /****** Object:  Job [BlitzFirst - Collection]    Script Date: 17-04-18 14:37:26 ******/
5.  
   IF  EXISTS (SELECT job_id FROM msdb.dbo.sysjobs_view WHERE name = N 'BlitzFirst - Collection')
6.  
   EXEC msdb.dbo.sp_delete_job @job_name = 'BlitzFirst - Collection', @delete_unused_schedule=1
7.  
   GO
8.  
    
9.  
   /****** Object:  Job [BlitzFirst - Collection]    Script Date: 17-04-18 14:37:26 ******/
10.  
    BEGIN TRANSACTION
11.  
    DECLARE @ReturnCode INT
12.  
    SELECT @ReturnCode = 0
13.  
    /****** Object:  JobCategory [[Uncategorized (Local)]]    Script Date: 17-04-18 14:37:26 ******/
14.  
    IF NOT EXISTS (SELECT name FROM msdb.dbo.syscategories WHERE name=N '[Uncategorized (Local)]' AND category_class=1)
15.  
    BEGIN
16.  
    EXEC @ReturnCode = msdb.dbo.sp_add_category @class=N'JOB', @type=N'LOCAL', @name=N'[Uncategorized (Local)]'
17.  
    IF (@ @ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
18.  
     
19.  
    END
20.  
     
21.  
    DECLARE @jobId BINARY(16)
22.  
    select @jobId = job_id from msdb.dbo.sysjobs where (name = N'BlitzFirst - Collection')
23.  
    if ( @jobId is NULL)
24.  
    BEGIN
25.  
    EXEC @ReturnCode =  msdb.dbo.sp_add_job @job_name=N'BlitzFirst - Collection',
26.  
    @enabled=0,
27.  
    @notify_level_eventlog=2,
28.  
    @notify_level_email=2,
29.  
    @notify_level_netsend=0,
30.  
    @notify_level_page=0,
31.  
    @delete_level=0,
32.  
    @description=N'No description available.',
33.  
    @category_name=N'[Uncategorized (Local)]',
34.  
    @owner_login_name=N'sa',
35.  
    @notify_email_operator_name=N'The DBA Team', @job_id = @jobId OUTPUT
36.  
    IF (@ @ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
37.  
     
38.  
    END
39.  
    /****** Object:  Step [Run Sp_BlitzFirst with statistics collection]    Script Date: 17-04-18 14:37:26 ******/
40.  
    IF NOT EXISTS (SELECT * FROM msdb.dbo.sysjobsteps WHERE job_id = @jobId and step_id = 1)
41.  
    EXEC @ReturnCode = msdb.dbo.sp_add_jobstep @job_id=@jobId, @step_name=N'Run Sp_BlitzFirst with statistics collection',
42.  
    @step_id=1,
43.  
    @cmdexec_success_code=0,
44.  
    @on_success_action=1,
45.  
    @on_success_step_id=0,
46.  
    @on_fail_action=2,
47.  
    @on_fail_step_id=0,
48.  
    @retry_attempts=0,
49.  
    @retry_interval=0,
50.  
    @os_run_priority=0, @subsystem=N'TSQL',
51.  
    @command=N'EXEC Monitoring.sp_BlitzFirst
52.  
           @Seconds                     = 5,
53.  
           @OutputDatabaseName          = ''DBAMGMT'',
54.  
           @OutputSchemaName            = ''dbo'',
55.  
           @OutputTableName             = ''BlitzFirstResults'',
56.  
           @OutputTableNameFileStats    = ''BlitzFirstResults_FileStats'',
57.  
           @OutputTableNamePerfmonStats = ''BlitzFirstResults_PerfmonStats'',
58.  
           @OutputTableNameWaitStats    = ''BlitzFirstResults_WaitStats''
59.  
    ;
60.  
    ',
61.  
    @database_name=N'DBAMGMT',
62.  
    @flags=4
63.  
    IF (@ @ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
64.  
    EXEC @ReturnCode = msdb.dbo.sp_update_job @job_id = @jobId, @start_step_id = 1
65.  
    IF (@ @ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
66.  
    EXEC @ReturnCode = msdb.dbo.sp_add_jobschedule @job_id=@jobId, @name=N'sc',
67.  
    @enabled=1,
68.  
    @freq_type=4,
69.  
    @freq_interval=1,
70.  
    @freq_subday_type=2,
71.  
    @freq_subday_interval=10,
72.  
    @freq_relative_interval=0,
73.  
    @freq_recurrence_factor=0,
74.  
    @active_start_date=20180416,
75.  
    @active_end_date=99991231,
76.  
    @active_start_time=0,
77.  
    @active_end_time=235959,
78.  
    @schedule_uid=N'624dd28c-8faf-4979-88a4-947977e9a834'
79.  
    IF (@ @ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
80.  
    EXEC @ReturnCode = msdb.dbo.sp_add_jobserver @job_id = @jobId, @server_name = N'(local)'
81.  
    IF (@ @ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
82.  
    COMMIT TRANSACTION
83.  
    GOTO EndSave
84.  
    QuitWithRollback:
85.  
        IF (@ @TRANCOUNT > 0) ROLLBACK TRANSACTION
86.  
    EndSave:
87.  
    GO

Note

注意

Previous script will drop an existing SQL Agent Job. It will also create the job disabled. It will be up to the tester to enable and disable the job for his tests.

先前的腳本將刪除現有SQL Agent Job。 它還將創建禁用的作業。 測試人員應根據自己的測試來啟用和禁用該作業。

One needs to set the database names in red according to its desire.

需要根據需要將數據庫名稱設置為紅色。

一些有用的查詢 (Some useful queries)

You will find below some useful queries against collection tables that will allow the generation of report charts. Parts in red have to be changed before execution.

您將在下面找到一些有關收集表的有用查詢，這些查詢將允許生成報告圖表。 紅色部分必須在執行前更改。

An excel worksheet model is available to generate charts for some statistics collected during benchmark test. Its name is “MDL-0020 – sp_BlitzFirst graphs”.

Excel工作表模型可用於為基准測試期間收集的一些統計信息生成圖表。 它的名稱是“ MDL-0020 – sp_BlitzFirst圖”。

網頁預期壽命 (Page Life Expectancy)

1.  
   select CheckDate,cntr_value as PageLifeExpectancy
2.  
   from dbo.BlitzFirstResults_PerfmonStats
3.  
   where counter_name ='Page life expectancy'
4.  
   AND CheckDate BETWEEN '2018-04-16 13:10:00' AND '2018-04-16 13:33:00'
5.  
   order by ID desc

批處理請求/秒 (Batch requests / sec)

1.  
   select
2.  
       CheckDate,
3.  
        REPLACE(
4.  
            CONVERT(VARCHAR(100),value_per_second),'.',','
5.  
       ) as BatchRequestsPerSec
6.  
   from dbo.BlitzFirstResults_PerfmonStats
7.  
   where counter_name ='Batch Requests/sec'
8.  
   AND CheckDate BETWEEN '2018-04-16 13:10:00' AND '2018-04-16 13:33:00'
9.  
   order by ID desc

服務器等待總計 (Server wait total)

1.  
   select
2.  
        CheckDate,value_delta as AverageWaitTimeMs
3.  
   from dbo.BlitzFirstResults_PerfmonStats
4.  
   where counter_name ='Average Wait Time (ms)'
5.  
   and instance_name = '_Total'
6.  
   AND CheckDate BETWEEN '2018-04-16 13:10:00' AND '2018-04-16 13:33:00'

服務器按類別等待 (Server wait by category)

1.  
   select
2.  
        CheckDate,instance_name,value_delta as AverageWaitTimeMs
3.  
   from dbo.BlitzFirstResults_PerfmonStats
4.  
   where counter_name ='Average Wait Time (ms)'
5.  
   and instance_name <> '_Total'
6.  
   AND CheckDate BETWEEN '2018-04-16 13:10:00' AND '2018-04-16 13:33:00'

測試准備 (Test preparation)

The HammerDb tools will create a database which will contain the schema and database objects used for benchmarking. This schema represents the activity of a firm with one or more warehouses.

HammerDb工具將創建一個數據庫，其中將包含用於基准測試的架構和數據庫對象。 該模式表示具有一個或多個倉庫的公司的活動。

If it’s the first time we run this tool against SQL Server, we ought to create some test databases with different numbers of warehouses.

如果這是我們首次針對SQL Server運行此工具，則應該創建一些具有不同倉庫數量的測試數據庫。

創建測試數據庫的步驟 (Steps to create a test database)

Let’s first review how to create a test database to use as a target for HammerDb.

讓我們首先回顧一下如何創建測試數據庫以用作HammerDb的目標。

To do so, we will first step create an empty database (with HammerDb) called [TPCC_x_WH] where x represents the number of warehouses we will create. The approximate (but unconfirmed) database size computation formula is 100Mb per warehouse.

為此，我們將第一步創建一個名為[TPCC_x_WH]的空數據庫（使用HammerDb），其中x表示我們將創建的倉庫數量。 每個倉庫的近似（但不確定）數據庫大小計算公式為100Mb。

Let’s start HammerDb.

讓我們啟動HammerDb。

 

 

 

We will double-click on SQL Server and select TPCC benchmarking option (for an OLTP simulation) then click on the “OK” button.

我們將雙擊SQL Server並選擇TPCC基准測試選項（用於OLTP模擬），然后單擊“確定”按鈕。

 

We will get a confirmation message:

我們將收到確認消息：

 

And the display is now for SQL Server. We can expand the “SQL Server > TPC-C “ tree node, go down to “Schema Build” and double click on Options

現在該顯示適用於SQL Server。 我們可以展開“ SQL Server> TPC-C”樹節點，轉到“ Schema Build”，然后雙擊“ Options”。

 

 

 

We will be prompted a dialog in which we can set the actual target computer and databases so as the credentials to be used for testing.

系統將提示您一個對話框，在其中可以設置實際的目標計算機和數據庫，以作為用於測試的憑據。

Here is an example of one warehouse:

這是一個倉庫的示例：

 

Once you hit “OK”, double-click on the “Build” leaf. A confirmation will be asked.

單擊“確定”后，雙擊“構建”葉子。 將要求確認。

 

Once, we clicked on the “Yes” button, we will have to wait for HammerDb to complete the creation of its testing schema:

一次，我們單擊“是”按鈕，我們將不得不等待HammerDb完成其測試架構的創建：

 

 

 

Once the creation is complete, you will see following message (green box).

創建完成后，您將看到以下消息（綠色框）。

 

 

 

Hit the red stop button to stop the thread used to create the database.

點擊紅色的停止按鈕以停止用於創建數據庫的線程。

Now the database is ready and as we won’t stop at this point for performance testing, it’s worth taking a backup of this database.

現在數據庫已經准備就緒，並且由於我們現在不會停止進行性能測試，因此值得備份該數據庫。

為基准創建的數據庫 (Databases to create for benchmark)

Four databases should be created and backed up on the lowest version of SQL Server with following a number of warehouses.

應該創建四個數據庫，並在具有多個倉庫的最低版本SQL Server上進行備份。

Number of warehouses	Experienced size on disk	Experienced backup size (compressed)
1	150 Mb	53 Mb
5	416 Mb	238 Mb
100	8100 Mb	4.5 Gb
500	39.75 Gb	22.6 Gb

倉庫數量	磁盤上的經驗大小	經驗豐富的備份大小（壓縮）
1個	150 Mb	53兆位
5	416 Mb	238 Mb
100	8100 Mb	4.5 Gb
500	39.75 Gb	22.6 Gb

Note

注意

It’s strongly recommended to take backups of these databases as the creation process is really time-consuming.

強烈建議對這些數據庫進行備份，因為創建過程確實很耗時。

測試定義 (Test Definition)

Here are the steps to follow to fire a test against a given [TPCC_x_DW] database. We will assume you let HammerDb open.

以下是針對給定的[TPCC_x_DW]數據庫進行測試的步驟。 我們假設您讓HammerDb打開。

測試用例腳本生成 (Test case script generation)

Expand “Driver Script” node and double-click on the “Options” leaf.

展開“驅動程序腳本”節點，然后雙擊“選項”葉。

 

 

 

Actions to be done:

要做的動作：

1. Edit server, database and credential settings

    

   編輯服務器，數據庫和憑據設置
2. Select “Timed Test Driver Script”

    

   選擇“定時測試驅動程序腳本”
3. Keep the default ramp-up settings.

    

   保留默認的加速設置。
4. Adjust the test duration to 10 minutes

    

   將測試時間調整為10分鍾
5. Keep “Use All warehouses” box unchecked

    

   取消選中“使用所有倉庫”框
6. documentation page for further details) 文檔頁面 ）
7. Click on the “OK” button

    

   點擊“確定”按鈕

The settings dialog should look as follows after step 6:

步驟6之后，設置對話框應如下所示：

 

Note:

注意事項 ：

We could also check the “Checkpoint when complete” option if we use the AutoPilot mode that we will discuss in following subsection.

如果我們使用將在以下小節中討論的AutoPilot模式，也可以選中“完成時檢查點”選項。

Now the options are set, we can generate the script that will be used by HammerDb. This is done by clicking on the “Load” leaf of tree hierarchy in the left panel:

現在設置了選項，我們可以生成將由HammerDb使用的腳本。 通過單擊左側面板中樹狀層次結構的“加載”葉子來完成此操作：

 

虛擬用戶創建 (Virtual users creation)

Now it’s time to tell HammerDb how many users will be using the database defined in “Driver Script” options.

現在是時候告訴HammerDb，將有多少用戶使用“驅動程序腳本”選項中定義的數據庫。

Virtual users refer to threads that will be created by HammerDb during the test. Their number can be configured in the “Virtual Users” node.

虛擬用戶引用測試期間HammerDb將創建的線程。 可以在“虛擬用戶”節點中配置其編號。

 

Most of the time, we will fire more than one test where only the number of virtual users vary. There are two ways to make it. The first one is to perform as many tests as virtual users count variations. The other one is to use the “Autopilot” mode of HammerDb.

在大多數情況下，我們將觸發不止一個測試，其中僅虛擬用戶數量有所不同。 有兩種方法可以實現。 第一個是執行與虛擬用戶計算變化數量一樣多的測試。 另一種是使用HammerDb的“自動駕駛”模式。

To enable the Autopilot mode, we will expand the “Autopilot” node and double-click on “Options”. This opens up a dialog in which we will select the “Autopilot enabled”:

要啟用自動駕駛模式，我們將展開“自動駕駛”節點，然后雙擊“選項”。 這將打開一個對話框，我們將在其中選擇“啟用自動駕駛”：

 

As we have a ramp-up of 2 minutes and test duration set to 10 minutes, we will set minutes per test value to 15 to be sure everything is ok.

因為我們有2分鍾的加速時間，並且測試持續時間設置為10分鍾，所以我們將每個測試值的分鍾設置為15，以確保一切正常。

We will also replace the default virtual user sequence by following:

我們還將通過以下方式替換默認的虛擬用戶序列：

3 5 9 17 25 37 49 65 97 129 197 257

3 5 9 17 25 37 49 65 97 129 197 257

Note

注意

Feel free to create your own virtual user’s number sequence based on the requirements for the application that will be using SQL Server.

根據將要使用SQL Server的應用程序的要求，隨意創建自己的虛擬用戶的號碼序列。

For instance, an alternative could be:

例如，一種替代方法可能是：

3 11 101 201 501 1001 1501 2001

3 11 101 201 501 1001 1501 2001

Anyway, with the first sequence, the dialog will look like this:

無論如何，對於第一個序列，對話框將如下所示：

 

We can click on the “OK” button to close the dialog.

我們可以單擊“確定”按鈕關閉對話框。

Before digging into the execution of AutoPilot mode, there is a step not to forget: the activation of the performance statistics collection job.

在深入了解執行AutoPilot模式之前，有一個不可忘記的步驟：激活性能統計信息收集作業。

This can be done with following T-SQL statement:

可以使用以下T-SQL語句完成此操作：

1.  
   EXEC msdb.dbo.sp_update_job  
2.  
        @job_name = N'BlitzFirst - Collection',  
3.  
        @enabled = 1 ;  
4.  
   GO

測試執行 (Test execution)

Now, everything is ready and we can double-click on the Autopilot button:

現在，一切就緒，我們可以雙擊“自動駕駛”按鈕：

 

It will open an “Autopilot” tab in the central pane:

它將在中央窗格中打開“自動駕駛”選項卡：

 

 

 

As soon as a test is finished, there will be a “TEST RESULT” line showing the observed number of transactions per minute and the observed number of order created per minute.

測試完成后，將出現“ TEST RESULT”（測試結果）行，顯示觀察到的每分鍾交易數和觀察到的每分鍾創建的訂單數。

 

 

 

We could keep track of these two numbers associated with the number of configured virtual users in order to create charts like this one:

我們可以跟蹤與已配置的虛擬用戶數量相關的這兩個數字，以創建類似這樣的圖表：

You could also create charts with SQL Server statistics like “Page Life Expectancy” or “Average Wait Time (ms)”. As we can find the time window during which a test occurred, it’s pretty easy to restrict data sets to just this time window.

您還可以使用SQL Server統計信息（例如“頁面預期壽命”或“平均等待時間（ms）”）創建圖表。 由於我們可以找到進行測試的時間范圍，因此很容易將數據集限制為該時間范圍。

翻譯自: https://www.sqlshack.com/how-to-perform-a-performance-test-against-a-sql-server-instance/

 

 附加信息：

1.   使用OStressSQL Server壓力測試

2.   About TPCC

3.  數據庫壓力測試工具 -- Hammerdb 使用說明