mysql千萬級數據量查詢出所有重復的記錄

查詢重復的字段需要創建索引，多個條件則創建組合索引，各個條件的索引都存在則不必須創建組合索引

有些情況直接使用GROUP BY HAVING則能直接解決；但是有些情況下查詢緩慢，則需要使用下面其他的方法

以下是本人整理出來的查詢重復數據的SQL，適用於千萬級數據量的MYSQL數據庫。

1.適用於整數類型的

SELECT DISTINCT UId FROM vehicle WHERE 1=1 GROUP BY UId HAVING(COUNT(UId))>1;

 

2.分組排序

SELECT UId FROM 
(
  SELECT UId,LicenseNo,IF(@grp=a.UId,@rank:=@rank+1,@rank:=1) rank,@grp:=a.UId FROM 
   (SELECT UId,LicenseNo FROM vehicle WHERE 1=1 ORDER BY UId) AS a,(SELECT @grp:=0,@rank:=0) AS b
)AS t
WHERE t.rank=2;

 

3.適用於字符串類型,包含中文

SELECT DISTINCT(LicenseNo) FROM vehicle a WHERE 1 = 1 AND ((SELECT COUNT(LicenseNo) FROM vehicle WHERE LicenseNo = a.LicenseNo) > 1) ;

 

4.適用於字符串類型，不包含中文

SELECT DISTINCT (a.CarVIN) FROM vehicle a LEFT JOIN vehicle b ON a.CarVIN=b.CarVIN WHERE 1=1 AND a.id<>b.Id;

 

本文版權歸作者所有，歡迎轉載，但未經作者同意必須保留此段聲明，且在文章頁面明顯位置給出 原文鏈接