oracle按日期分組 (寫一個定時任務, 每天晚上運行, 統計某渠道近30天里每天的數據)

 

1. 問題描述:

 

 

 

 有一張表叫domains_report, 有 渠道號, 日期,登陸數,注冊數, 交易筆數, 交易金額, 退款筆數,退款金額 這幾個字段, 字段是分散在4張表中(每個表都有日期字段. 可以限定統計區間). 需求是 將統計的數據查詢出來, 然后插入到domains_report表里面去.

 

 

 

 

 

2. 業務思路:

 

1.先在數據庫編寫生成統計數據的查詢sql.

2.將寫好的查詢sql放到java程序里面

3.查詢sql , 得到list(近30天的數據) 結果.  先delete表domains_report表里近30天的數據, 遍歷list, 把數據 insert到domains_report表里去. 

4.設置java程序,每天凌晨啟動. 

補充:

• 對於第3點, 為什么要先刪除domains_report近30天的數據, 再插入domains_report近30天的數據?

          因為, 不刪除直接插入的話, 會報主鍵唯一錯誤. 因為日期report_date是唯一鍵.

          並且, delete和 insert 要放在一個事物里面. 要么都一起成功,要么一起失敗. 

• 在把查詢sql放到數據庫里面去的時候, 最好把一些參數, 都寫在配置文件里面 . 以及定時任務設置運行的時間, 也放在配置文件里面

 

 

 

3. 在統計查詢的sql過程中, 很快就發現2大問題:

1. 發現4個表一起查詢, 會有重復數據
2. 從8月1號, 到8月30號, 如果沒有數據, 那天什么都不顯示. 如這位網友遇到的問題一樣: https://blog.csdn.net/jie11447416/article/details/50887888

為了解決這2個問題, 參考網友的做法,可以解決.  建立一張日歷表, 查詢的時候, 關聯日歷表即可.  但是網友是mysql數據庫, 我是orcal, 於是函數和方法找了替代.

a . 建立一張日期表 calendar

create table CALENDAR
(
  datelist VARCHAR2(1024) not null
)
;
alter table CALENDAR
  add constraint PK_CALENDAR_ITEM primary key (DATELIST);

b. 生成日期表的數據, 把數據導入到 calendar日期表里面去

SELECT A.DATAS
  FROM (SELECT TO_CHAR(TO_DATE('20190101', 'yyyyMMdd') + ROWNUM - 1,
                       'yyyyMMdd') AS DATAS
          FROM DUAL
        CONNECT BY ROWNUM <=
                   TRUNC(TO_DATE('20500902', 'yyyyMMdd') -
                         TO_DATE('20190101', 'yyyyMMdd')) + 1) A

 

 

 

c. 在查詢的語句中, 要關聯 calendar日期表,就解決這個問題啦. 下面這個是查詢數據的完整sql

select a1.domains_id,
       a1.report_date,
       a3.login_count,
       a2.regin_count,
       a1.trans_count,
       a1.trans_sum,
       a4.refund_count,
       a4.refund_sum
  from (select t2.report_date,
               nvl(t1.trans_count, 0) trans_count,
               nvl(t1.trans_sum, 0) trans_sum,
               nvl(t1.domains_id, '04103000000001000') domains_id
          from (select to_char(to_date(t.trans_time, 'yyyy-MM-dd hh24:mi:ss'),
                               'yyyyMMdd') trans_time,
                       count(t.trans_nbr) trans_count,
                       sum(t.trans_amount) trans_sum,
                       t.domains_id
                  from trans_item t
                 where t.domains_id = '04103000000001000'
                   and t.stats in (8, 9)
                 group by to_char(to_date(t.trans_time,
                                          'yyyy-MM-dd hh24:mi:ss'),
                                  'yyyyMMdd'),
                          domains_id) t1
         right join (SELECT datelist as report_date, null, null, null
                      FROM calendar t
                     where SYSDATE - 30 <=
                           trunc(to_date(t.datelist, 'yyyyMMdd'))
                       and trunc(to_date(t.datelist, 'yyyyMMdd')) <= sysdate) t2
            on t1.trans_time = t2.report_date) a1,
       (select t2.dday, nvl(t1.regin_count, 0) regin_count
          from (select count(1) regin_count,
                       to_char(to_date(t.create_time, 'yyyy-MM-dd hh24:mi:ss'),
                               'yyyyMMdd') create_time
                  from domains_users t
                 where t.domains_id = '04103000000001000'
                 group by to_char(to_date(t.create_time,
                                          'yyyy-MM-dd hh24:mi:ss'),
                                  'yyyyMMdd')) t1
         right join (SELECT datelist as dday, null, null
                      FROM calendar t
                     where SYSDATE - 30 <=
                           trunc(to_date(t.datelist, 'yyyyMMdd'))
                       and trunc(to_date(t.datelist, 'yyyyMMdd')) <= sysdate) t2
            on t1.create_time = t2.dday) a2,
       (select t2.dday, nvl(t1.login_count, 0) login_count
          from (select count(1) login_count,
                       to_char(to_date(t.login_time, 'yyyy-MM-dd hh24:mi:ss'),
                               'yyyyMMdd') login_time
                  from login_log t
                 where t.domains_id = '04103000000001000'
                 group by to_char(to_date(t.login_time,
                                          'yyyy-MM-dd hh24:mi:ss'),
                                  'yyyyMMdd')) t1
         right join (SELECT datelist as dday, null
                      FROM calendar t
                     where SYSDATE - 30 <=
                           trunc(to_date(t.datelist, 'yyyyMMdd'))
                       and trunc(to_date(t.datelist, 'yyyyMMdd')) <= sysdate) t2
            on t1.login_time = t2.dday) a3,
       (select t2.dday,
               nvl(t1.refund_count, 0) refund_count,
               nvl(t1.refund_sum, 0) refund_sum
          from (select nvl(count(1), '0') refund_count,
                       sum(ti.refund_amount) refund_sum,
                       to_char(to_date(ti.refund_time,
                                       'yyyy-MM-dd hh24:mi:ss'),
                               'yyyyMMdd') refund_time
                  from refund_item ti, trans_item tt
                 where ti.trans_nbr = tt.trans_nbr
                   and tt.domains_id = '04103000000001000'
                 group by to_char(to_date(ti.refund_time,
                                          'yyyy-MM-dd hh24:mi:ss'),
                                  'yyyyMMdd')) t1
         right join (SELECT datelist as dday, null
                      FROM calendar t
                     where SYSDATE - 30 <=
                           trunc(to_date(t.datelist, 'yyyyMMdd'))
                       and trunc(to_date(t.datelist, 'yyyyMMdd')) <= sysdate) t2
            on t1.refund_time = t2.dday) a4
 where a1.report_date = a2.dday
   and a2.dday = a3.dday
   and a3.dday = a4.dday

運行出來的效果圖如下:

 

 

 

補充:

刪除近30天統計數據的sql , 這個時間 30代表30天,  也要放在配置文件里面.

delete from domains_report t where t.report_date in (  SELECT *   FROM calendar t  where SYSDATE - 30 <= trunc(to_date(t.datelist, 'yyyyMMdd')) and trunc(to_date(t.datelist, 'yyyyMMdd')) <= sysdate)

 

 

 

 

 

 

.

~~~~~~~~~~~~~~~~~靜下心來想想, 在寫這個定時程序中, 細節怎么實現. 技術怎么實現.   多試試. 一定可以的. 加油!~~~~~~~~~~~~~~~~~