碼上歡樂
相關內容    簡體    繁體

MS SQL 監控數據/日志文件增長

本文轉載自   查看原文   2013-05-12 22:08   1440    數據庫技術(MS SQL)
    前幾天,在所有數據庫服務器部署了監控磁盤空間的存儲過程和作業后（MS SQL 監控磁盤空間告警），今天突然收到了兩封告警郵件，好吧，存儲規划是一方面，但是，是不是要分析一下是什么原因造成磁盤空間不足的呢？會不會是因為突然暴增的日志文件，抑或是系統業務猛增導致數據量暴增，還是歷史數據累計原因....分析總得有數據來支撐吧，但是現在只有那些數據文件的當前大小信息，沒有數據文件的歷史增長變化信息，所以，今天就想實現這么一個功能，每天（頻率可以調整）去收集一下數據文件的信息，放到一個表里面，這樣方便我們分析數據文件的增長演變例程，甚至你可以將數據文件的增長幅度和業務變化關聯起來分析....
 
那么接下來就是我的設計思路和實現代碼，目前只是簡單實現，以后將繼續優化,豐富一些功能。
 
首先我們創建一個表DiskCapacityHistory，用來保存數據庫文件的歷史增長變化信息： 
USE  msdb;
GO
 
IF  EXISTS (SELECT 1 FROM dbo.sysobjects WHERE id = OBJECT_ID(N'') AND xtype='U')
    DROP TABLE DiskCapacityHistory;
GO

CREATE TABLE dbo.DiskCapacityHistory
(
    [Date_CD]            INT                     ,
    [DataBaseID]         INT                     ,
    [FileID]             INT                     ,
    [DataBaseName]       sysname                 ,
    [LogicalName]        VARCHAR(32)             ,
    [FileTypeDesc]       NVARCHAR(60)            ,
    [PhysicalName]       NVARCHAR(260)         　,
    [StateDesc]          NVARCHAR(60)         　 ,
    [MaxSize]            NVARCHAR(32)            ,
    [GrowthType]         NVARCHAR(8)             ,
    [IsReadOnly]         INT                     ,
    [IsPercentGrowth]    SMALLINT                ,
    [Size]               FLOAT                   ,
    [Growth_MOM_RAT]     FLOAT                   ,
    [Growth_YOY_RAT]     FLOAT                   ,
    CONSTRAINT PK_DiskCapacityHistory PRIMARY KEY(Date_CD, DataBaseID, FileID)     
);
View Code 
EXEC sys.sp_addextendedproperty @name = N'MS_Description'
    , @value = '日期編碼'
    , @level0type = N'SCHEMA'
    , @level0name = N'dbo'
    , @level1type = N'TABLE'
    , @level1name = N'DiskCapacityHistory'
    , @level2type = N'COLUMN'
    , @level2name = N'Date_CD';
    
EXEC sys.sp_addextendedproperty @name = N'MS_Description'
    , @value = '數據庫標識'
    , @level0type = N'SCHEMA'
    , @level0name = N'dbo'
    , @level1type = N'TABLE'
    , @level1name = N'DiskCapacityHistory'
    , @level2type = N'COLUMN'
    , @level2name = N'DataBaseID';
    
EXEC sys.sp_addextendedproperty @name = N'MS_Description'
    , @value = '文件標識'
    , @level0type = N'SCHEMA'
    , @level0name = N'dbo'
    , @level1type = N'TABLE'
    , @level1name = N'DiskCapacityHistory'
    , @level2type = N'COLUMN'
    , @level2name = N'FileID';

EXEC sys.sp_addextendedproperty @name = N'MS_Description'
    , @value = '數據庫名稱'
    , @level0type = N'SCHEMA'
    , @level0name = N'dbo'
    , @level1type = N'TABLE'
    , @level1name = N'DiskCapacityHistory'
    , @level2type = N'COLUMN'
    , @level2name = N'DataBaseName';
 
 
 EXEC sys.sp_addextendedproperty @name = N'MS_Description'
    , @value = '數據庫邏輯名稱'
    , @level0type = N'SCHEMA'
    , @level0name = N'dbo'
    , @level1type = N'TABLE'
    , @level1name = N'DiskCapacityHistory'
    , @level2type = N'COLUMN'
    , @level2name = N'LogicalName';
    
 
EXEC sys.sp_addextendedproperty @name = N'MS_Description'
    , @value = '文件類型描述'
    , @level0type = N'SCHEMA'
    , @level0name = N'dbo'
    , @level1type = N'TABLE'
    , @level1name = N'DiskCapacityHistory'
    , @level2type = N'COLUMN'
    , @level2name = N'FileTypeDesc';
    

    
EXEC sys.sp_addextendedproperty @name = N'MS_Description'
    , @value = '物理數據庫文件'
    , @level0type = N'SCHEMA'
    , @level0name = N'dbo'
    , @level1type = N'TABLE'
    , @level1name = N'DiskCapacityHistory'
    , @level2type = N'COLUMN'
    , @level2name = N'PhysicalName';
    
EXEC sys.sp_addextendedproperty @name = N'MS_Description'
    , @value = '文件最大大小'
    , @level0type = N'SCHEMA'
    , @level0name = N'dbo'
    , @level1type = N'TABLE'
    , @level1name = N'DiskCapacityHistory'
    , @level2type = N'COLUMN'
    , @level2name = N'MaxSize';
    
EXEC sys.sp_addextendedproperty @name = N'MS_Description'
    , @value = '文件增長類型'
    , @level0type = N'SCHEMA'
    , @level0name = N'dbo'
    , @level1type = N'TABLE'
    , @level1name = N'DiskCapacityHistory'
    , @level2type = N'COLUMN'
    , @level2name = N'GrowthType';
 
EXEC sys.sp_addextendedproperty @name = N'MS_Description'
    , @value = '是否只讀類型'
    , @level0type = N'SCHEMA'
    , @level0name = N'dbo'
    , @level1type = N'TABLE'
    , @level1name = N'DiskCapacityHistory'
    , @level2type = N'COLUMN'
    , @level2name = N'IsReadOnly';
    
EXEC sys.sp_addextendedproperty @name = N'MS_Description'
    , @value = '是否按百分比增長'
    , @level0type = N'SCHEMA'
    , @level0name = N'dbo'
    , @level1type = N'TABLE'
    , @level1name = N'DiskCapacityHistory'
    , @level2type = N'COLUMN'
    , @level2name = N'IsPercentGrowth';

EXEC sys.sp_addextendedproperty @name = N'MS_Description'
    , @value = '數據文件大小(GB)'
    , @level0type = N'SCHEMA'
    , @level0name = N'dbo'
    , @level1type = N'TABLE'
    , @level1name = N'DiskCapacityHistory'
    , @level2type = N'COLUMN'
    , @level2name = N'Size';
    
EXEC sys.sp_addextendedproperty @name = N'MS_Description'
    , @value = '文件增長環比(%)'
    , @level0type = N'SCHEMA'
    , @level0name = N'dbo'
    , @level1type = N'TABLE'
    , @level1name = N'DiskCapacityHistory'
    , @level2type = N'COLUMN'
    , @level2name = N'Growth_MOM_RAT';
    
EXEC sys.sp_addextendedproperty @name = N'MS_Description'
    , @value = '文件增長同比(%)'
    , @level0type = N'SCHEMA'
    , @level0name = N'dbo'
    , @level1type = N'TABLE'
    , @level1name = N'DiskCapacityHistory'
    , @level2type = N'COLUMN'
    , @level2name = N'Growth_YOY_RAT';
    
GO

IF  OBJECT_ID(N'sp_diskcapacity_cal')  IS NOT NULL
    DROP PROCEDURE sp_diskcapacity_cal;
GO

 

接下來，我們創建存儲過程，負責來收集、統計這些數據庫的文件的相關信息。關於環比/同比,正常情況一般是:

環比:  (指標當前值 - 指標值(上個月同一天))/ 指標值(上個月同一天) 。

同比:  (指標當前值 - 指標值(去年月同一天))/ 指標值(去年月同一天) 。

其實如果關注每天的數據文件變化情況，這個代碼里面的環比、同比其實意義不大，其實我們可以這樣定義環比、同比：

環比： （指標當前值 - 指標值（昨天）)/指標值（昨天）。

同比： （指標當前值 - 指標值 （上個月））/指標值（上個月）

當然，你也可以把這四個指標都加上，對比參考，側重點不同而已。

存儲過程
  1.   IF  OBJECT_ID(N'sp_diskcapacity_cal')IS NOT NULL
  2.     DROP PROCEDURE sp_diskcapacity_cal;
  3. GO
  4.  
  5. CREATE PROCEDURE dbo.sp_diskcapacity_cal
  6. AS
  7. BEGIN
  8.    
  9.    INSERT INTO dbo.DiskCapacityHistory
  10.    (
  11.         [Date_CD]           ,
  12.         [DataBaseID]        ,
  13.         [FileID]            ,
  14.         [DataBaseName]      ,
  15.         [LogicalName]       ,
  16.         [FileTypeDesc]      ,
  17.         [PhysicalName]      ,
  18.         [StateDesc]         ,
  19.         [MaxSize]           ,
  20.         [GrowthType]        ,
  21.         [IsReadOnly]        ,
  22.         [IsPercentGrowth]   ,
  23.         [Size]                
  24.    )
  25.      SELECT CAST(REPLACE(CONVERT(varchar(10),GETDATE(),120),'-','') AS INT)
  26.                                                                             AS DateCD        ,
  27.             database_id                                                     AS DataBaseId    ,
  28.             file_id                                                         AS FileID        ,
  29.             DB_NAME(database_id)                                            AS DataBaseName  ,
  30.             name                                                            AS LogicalName   ,
  31.             type_desc                                                       AS FileTypeDesc  ,
  32.             physical_name                                                   AS PhysicalName  ,
  33.             state_desc                                                      AS StateDesc     ,
  34.             CASE WHEN max_size = 0 THEN N'不允許增長'
  35.                  WHEN max_size = -1 THEN N'自動增長'
  36.                  ELSE LTRIM(STR(max_size * 8.0 / 1024 / 1024, 14, 2)) + 'G'
  37.             END                                                             AS MaxSize       ,
  38.             CASE WHEN is_percent_growth = 1
  39.                  THEN RTRIM(CAST(Growth AS CHAR(10))) + '%'
  40.                  ELSE RTRIM(CAST(Growth AS CHAR(10))) + 'M'
  41.             END                                                             AS Growth        ,
  42.             Is_Read_Only AS IsReadOnly ,
  43.             Is_Percent_Growth AS IsPercentGrowth ,
  44.             CAST(size * 8.0 / 1024 / 1024 AS DECIMAL(8, 4))                 AS Size
  45.      FROM   sys.master_files;
  46.      
  47.      
  48.      MERGE INTO dbo.DiskCapacityHistory DM USING
  49.      (
  50.      SELECT M.Date_CD        ,
  51.             M.DataBaseID     ,
  52.             M.FileID         ,
  53.             CASE WHEN N.SIZE IS NULL OR N.SIZE = 0 THEN 0 ELSE
  54.                 (M.SIZE - N.SIZE)/N.SIZE END AS Growth_MOM_RAT
  55.      FROM dbo.DiskCapacityHistory M
  56.       LEFT JOIN dbo.DiskCapacityHistory  N ON
  57.               CAST(CAST(M.Date_CD AS CHAR(8)) AS DATE) = DATEADD(MONTH, 1, CAST(CAST(N.Date_CD AS CHAR(8)) AS DATE))
  58.           AND M.DataBaseID = N.DataBaseID AND M.FileID = N.FileID
  59.      WHERE M.Date_CD =  CAST(REPLACE(CONVERT(varchar(10),GETDATE(),120),'-','') AS INT)
  60.      ) TMP
  61.      ON
  62.      (
  63.             DM.Date_CD       = TMP.Date_CD     AND
  64.             DM.DatabaseId    = TMP.DataBaseId  AND
  65.             DM.FileId        = TMP.FileId
  66.      )
  67.      WHEN MATCHED THEN UPDATE SET
  68.         DM.Growth_MOM_RAT = TMP.Growth_MOM_RAT;
  69. END    
  70. GO

順便吐槽一下：由於前兩年一直使用ORACLE數據庫，很少接觸SQL SERVER，在實現上面功能的時候，我深深的體會到了ORACLE和SQL SERVER的巨大差距，如果用PL/SQL實現，那非常方便快捷，但是用T-SQL讓我遇到了幾個相當痛苦地方，下面順便記錄對比一下吧：

 
一：由於我采用INT來保存日期數據，那么需要在DATE類型和INT類型之間轉換，我們來對比一下兩者的差別吧：
 
 
1.1 DATE類型轉換為整型:
 
T-SQL：
    
SELECT CAST(REPLACE(CONVERT(varchar(10),GETDATE(),120),'-','') AS INT);
 
PL/SQL：
  
SELECT TO_CHAR(Date_CD, 'YYYYMMDD') FROM DUAL;
 
 
1.2 整型轉換為DATE類型（字段DATE_CD）
 
T-SQL：
   
    SELECT CAST(CAST(DATE_CD AS CHAR(8)) AS DATE) FROM TEST;
 
PL/SQL：
 
    SELECT TO_DATE(DATE_CD, 'YYYY-MM-DD') FROM TEST;
 
結論: 純屬個人感受,從上面的腳本的簡單性,方便性上,感覺ORACLE完勝SQL SERVER
 
 
二：計算數據文件增長同比、環比值
 
 
  1：SQL SERVER 2005 沒有MERGE語句功能，上面的腳本得改寫成
 
Code Snippet
  1. UPDATEdbo.DiskCapacityHistory
  2.  SET     GROWTH_MOM_RAT =( SELECTCASE WHEN N.SIZE IS NULL
  3.                                             OR N.SIZE = 0 THEN 0
  4.                                        ELSE ( dbo.DiskCapacityHistory.SIZE
  5.                                               - N.SIZE ) / N.SIZE
  6.                                   END AS Growth_MOM_RAT
  7.                          FROM     dbo.DiskCapacityHistory N
  8.                          WHERE    CAST(CAST(dbo.DiskCapacityHistory.Date_CD AS CHAR(8)) AS DATE) = DATEADD(MONTH,
  9.                                                             1,
  10.                                                             CAST(CAST(N.Date_CD AS CHAR(8)) AS DATE))
  11.                                   AND dbo.DiskCapacityHistory.DataBaseID = N.DataBaseID
  12.                                   AND dbo.DiskCapacityHistory.FileID = N.FileID
  13.                        )
  14.  WHEREdbo.DiskCapacityHistory.Date_CD = CAST(REPLACE(CONVERT(VARCHAR(10), GETDATE(), 120),
  15.                                                      '-', '') AS INT)
  16.  
  17. UPDATEdbo.DiskCapacityHistory
  18.  SET     GROWTH_YOY_RAT =( SELECTCASE WHEN N.SIZE IS NULL
  19.                                             OR N.SIZE = 0 THEN 0
  20.                                        ELSE ( dbo.DiskCapacityHistory.SIZE
  21.                                               - N.SIZE ) / N.SIZE
  22.                                   END AS Growth_YOY_RAT
  23.                          FROM     dbo.DiskCapacityHistory N
  24.                          WHERE    CAST(CAST(dbo.DiskCapacityHistory.Date_CD AS CHAR(8)) AS DATE) = DATEADD(MONTH,
  25.                                                             12,
  26.                                                             CAST(CAST(N.Date_CD AS CHAR(8)) AS DATE))
  27.                                   AND dbo.DiskCapacityHistory.DataBaseID = N.DataBaseID
  28.                                   AND dbo.DiskCapacityHistory.FileID = N.FileID
  29.                        )
  30.  WHEREdbo.DiskCapacityHistory.Date_CD = CAST(REPLACE(CONVERT(VARCHAR(10), GETDATE(), 120),
  31.                                                      '-', '') AS INT)
或
Code Snippet
  1. CREATE TABLE #DiskCapacityHistory
  2.     (
  3.       DATE_CD INT ,
  4.       DataBaseID INT ,
  5.       FileID INT ,
  6.       Growth_MOM_RAT FLOAT
  7.     ) ;
  8.  
  9.   INSERTINTO #DiskCapacityHistory
  10.         SELECT  M.DATE_CD ,
  11.                 M.DataBaseID ,
  12.                 M.FileID ,
  13.                 CASE WHEN N.SIZE IS NULL
  14.                           OR N.SIZE = 0 THEN 0
  15.                      ELSE ( M.SIZE - N.SIZE ) / N.SIZE
  16.                 END AS Growth_MOM_RAT
  17.         FROM    dbo.DiskCapacityHistory M ,
  18.                 dbo.DiskCapacityHistory N
  19.         WHERE   CAST(CAST(M.Date_CD AS CHAR(8)) AS DATE) = DATEADD(MONTH, 1,
  20.                                                               CAST(CAST(N.Date_CD AS CHAR(8)) AS DATE))
  21.                 AND M.DataBaseID = N.DataBaseID
  22.                 AND M.FileID = N.FileID
  23.                 AND M.Date_CD = CAST(REPLACE(CONVERT(VARCHAR(10), GETDATE()
  24.                                              - 1, 120), '-', '') AS INT)
  25.  
  26.   UPDATE dbo.DiskCapacityHistory
  27.      SET Growth_MOM_RAT = M.Growth_MOM_RAT
  28.     FROM #DiskCapacityHistory M
  29.    WHERE dbo.DiskCapacityHistory.DATE_CD = M.DATE_CD
  30.         AND dbo.DiskCapacityHistory.DataBaseID = M.DataBaseID
  31.         AND dbo.DiskCapacityHistory.FileID = M.FileID ;

 

  2: 幸好SQL 2008還把ORACLE的MERGE的功能給模仿了過來，但是T-SQL缺少ORACLE數據庫強大的分析函數LAG，如果有這個，我計算環比，同比就非常方便了，一個SQL就搞定了，下面是個例子，本想把ORACLE的SQL也做個例子展現，但是又要建表、造數，折騰起來比較麻煩。

Oracle Sample
  1. MERGE INTO DM.TM_WGGBO_IDCTOBUSVOLDTL_DAY DM
  2. USING    (
  3.              SELECT *
  4.                FROM (
  5.                        SELECT    DATE_CD,
  6.                                  CITY_ID,
  7.                                  IDC_NODE,
  8.                                  VOL_TYPE,
  9.                                  LAG(IDC_VOL_RAT   ) OVER(PARTITION BY CITY_ID,IDC_NODE,VOL_TYPE,SUBSTR(DATE_CD,7,2) ORDER BY SUBSTR(DATE_CD,0,6)) AS IDC_MOM_RAT                ,
  10.                                  LAG(IDC_VOL_RAT   ) OVER(PARTITION BY CITY_ID,IDC_NODE,VOL_TYPE,SUBSTR(DATE_CD,5,4) ORDER BY SUBSTR(DATE_CD,0,4)) AS IDC_YOY_RAT                 ,
  11.                                        
  12.                          FROM DM.TM_WGGBO_IDCTOBUSVOLDTL_DAY
  13.                        ) T
  14.                  WHERE EXISTS(SELECT 1 FROM ETL.T_IDCVOL_DAY_${ssid} WHERE DATE_CD = T.DATE_CD)
  15.           ) TEMP
  16.                         ON (
  17.                                 DM.DATE_CD     = TEMP.DATE_CD     AND
  18.                                 DM.CITY_ID     = TEMP.CITY_ID     AND
  19.                                 DM.IDC_NODE    = TEMP.IDC_NODE    AND
  20.                                 DM.VOL_TYPE    = TEMP.VOL_TYPE
  21.                                 )
  22. WHEN MATCHED THEN
  23.   UPDATE
  24.        SET DM.IDC_MOM_RAT    =       TEMP.IDC_MOM_RAT                    ,
  25.            DM.IDC_YOY_RAT    =       TEMP.IDC_YOY_RAT                     
  26. ;
  27. COMMIT;

 


免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。



猜您在找 SQL SERVER 監控數據文件增長情況 MS SQL 監控錯誤日志的告警信息 MS SQL 需要定期清理日志文件 SQL SEVER 數據庫日志（Log）文件增長過快的處理 MS Sql Server 數據庫或表修復(Log日志文件損壞的修復方法) SQL Server日志文件不斷增長處理方法 你還可以再詭異點嗎——SQL日志文件不斷增長 MS SQL監控數據庫的DDL操作 MS SQL 日志記錄管理 MS SQL 日志記錄管理
 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM