一、ICP( Index_Condition_Pushdown)
對 where 中過濾條件的處理，根據索引使用情況分成了三種：（何登成）index key, index filter, table filter
如果WHERE條件可以使用索引，MySQL 會把這部分過濾操作放到存儲引擎層，存儲引擎通過索引過濾，把滿足的行從表中讀取出。ICP能減少Server層訪問存儲引擎的次數和引擎層訪問基表的次數。
- session級別設置：set optimizer_switch="index_condition_pushdown=on
-
對於InnoDB表，ICP只適用於輔助索引
-
當使用ICP優化時，執行計划的Extra列顯示Using index condition提示
-
不支持主建索引的ICP（對於Innodb的聚集索引，完整的記錄已經被讀取到Innodb Buffer，此時使用ICP並不能降低IO操作）
-
當 SQL 使用覆蓋索引時但只檢索部分數據時，ICP 無法使用
-
ICP的加速效果取決於在存儲引擎內通過ICP篩選掉的數據的比例
index_condition_pushdown會大大減少行鎖的個數，如select for update， 因為行鎖是在引擎層的
例如：
現在的索引
show index from sm_performance_all; +--------------------+------------+-------------------------------+--------------+----------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+ | Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment | +--------------------+------------+-------------------------------+--------------+----------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+ | sm_performance_all | 0 | PRIMARY | 1 | id | A | 40527 | NULL | NULL | | BTREE | | | | sm_performance_all | 1 | FK_a9t29a4b2af1vfny1j2minc1x | 1 | company_id | A | 316 | NULL | NULL | YES | BTREE | | | | sm_performance_all | 1 | FK_n3ng4a5qju19fw8qy4uskp4g1 | 1 | bill_id | A | 21532 | NULL | NULL | YES | BTREE | | | | sm_performance_all | 1 | FK_eb13u3xwslt9t7wwuycg7vha6 | 1 | car_id | A | 16794 | NULL | NULL | YES | BTREE | | | | sm_performance_all | 1 | FK_2bfhskvklf6mdk557tc3yy3y1 | 1 | commission_entity_id | A | 177 | NULL | NULL | YES | BTREE | | | | sm_performance_all | 1 | FK_6fr5ib5iyjyu155dncmc48cwr | 1 | member_card_id | A | 34 | NULL | NULL | YES | BTREE | | | | sm_performance_all | 1 | FK_93p22vcog266wa82i44a6m18b | 1 | user_id | A | 483 | NULL | NULL | YES | BTREE | | | | sm_performance_all | 1 | FK_p6nc7l6ewnkcpm2y4o3wct81r | 1 | member_card_bill_id | A | 4 | NULL | NULL | YES | BTREE | | | | sm_performance_all | 1 | billId_userId_memberCarBillId | 1 | bill_id | A | 24194 | NULL | NULL | YES | BTREE | | | | sm_performance_all | 1 | billId_userId_memberCarBillId | 2 | user_id | A | 25688 | NULL | NULL | YES | BTREE | | | | sm_performance_all | 1 | billId_userId_memberCarBillId | 3 | member_card_bill_id | A | 25946 | NULL | NULL | YES | BTREE | | | +--------------------+------------+-------------------------------+--------------+----------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+ 11 rows in set (0.00 sec)
現在的語句執行情況
explain select * from sm_performance_all p where p.date_created>'2018-01-01' and p.date_created< '2018-02-01' and p.type=0; +----+-------------+-------+------------+------+---------------+------+---------+------+-------+----------+-------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+-------+------------+------+---------------+------+---------+------+-------+----------+-------------+ | 1 | SIMPLE | p | NULL | ALL | NULL | NULL | NULL | NULL | 40527 | 1.11 | Using where | +----+-------------+-------+------------+------+---------------+------+---------+------+-------+----------+-------------+ 1 row in set, 1 warning (0.00 sec)
添加索引后
ALTER TABLE sm_performance_all add index date_created_type(date_created, type );
explain select * from sm_performance_all p where p.date_created>'2018-01-01' and p.date_created< '2018-02-01' and p.type=0; +----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+-----------------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+-----------------------+ | 1 | SIMPLE | p | NULL | range | date_created_type | date_created_type | 6 | NULL | 1 | 10.00 | Using index condition | +----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+-----------------------+ 1 row in set, 1 warning (0.00 sec)
二、MRR(Multi-Range Read )
隨機 IO 轉化為順序 IO 以降低查詢過程中 IO 開銷的一種手段，這對IO-bound類型的SQL語句性能帶來極大的提升。
MRR can be used for InnoDB and MyISAM tables for index range scans and equi-join operations.
-
A portion of the index tuples are accumulated in a buffer.
-
The tuples in the buffer are sorted by their data row ID.
-
Data rows are accessed according to the sorted index tuple sequence.
上述的SQL語句需要根據輔助索引date_created_type進行查詢，但是由於要求得到的是表中所有的列，因此需要回表進行讀取。而這里就可能伴隨着大量的隨機I/O。這個過程如下圖所示：
而MRR的優化在於，並不是每次通過輔助索引就回表去取記錄，而是將其rowid給緩存起來，然后對rowid進行排序后，再去訪問記錄，這樣就能將隨機I/O轉化為順序I/O，從而大幅地提升性能。這個過程如下所示：
然而，在MySQL當前版本中，基於成本的算法過於保守，導致大部分情況下優化器都不會選擇MRR特性。為了確保優化器使用mrr特性，請執行下面的SQL語句：
set optimizer_switch='mrr=on,mrr_cost_based=off';
讀取全部字段時
explain select * from sm_performance_all p where p.date_created>'2018-01-01' and p.date_created< '2018-02-01' and p.type=0; +----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+----------------------------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+----------------------------------+ | 1 | SIMPLE | p | NULL | range | date_created_type | date_created_type | 6 | NULL | 1 | 10.00 | Using index condition; Using MRR | +----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+----------------------------------+ 1 row in set, 1 warning (0.00 sec)
只讀取部分字段時：
讀取外鍵
explain select car_id from sm_performance_all p where p.date_created>'2018-01-01' and p.date_created< '2018-02-01' and p.type=0; +----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+----------------------------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+----------------------------------+ | 1 | SIMPLE | p | NULL | range | date_created_type | date_created_type | 6 | NULL | 1 | 10.00 | Using index condition; Using MRR | +----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+----------------------------------+ 1 row in set, 1 warning (0.00 sec)
讀取主鍵
explain select id from sm_performance_all p where p.date_created>'2018-01-01' and p.date_created< '2018-02-01' and p.type=0; +----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+--------------------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+--------------------------+ | 1 | SIMPLE | p | NULL | range | date_created_type | date_created_type | 6 | NULL | 1 | 10.00 | Using where; Using index | +----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+--------------------------+ 1 row in set, 1 warning (0.00 sec)
For MRR, a storage engine uses the value of the read_rnd_buffer_size system variable as a guideline for how much memory it can allocate for its buffer.
默認256KB
show GLOBAL VARIABLES like '%buffer_size'; +-------------------------+----------+ | Variable_name | Value | +-------------------------+----------+ | bulk_insert_buffer_size | 8388608 | | innodb_log_buffer_size | 16777216 | | innodb_sort_buffer_size | 1048576 | | join_buffer_size | 262144 | | key_buffer_size | 8388608 | | myisam_sort_buffer_size | 8388608 | | preload_buffer_size | 32768 | | read_buffer_size | 131072 | | read_rnd_buffer_size | 262144 | | sort_buffer_size | 262144 | +-------------------------+----------+ 10 rows in set (0.00 sec)
三、表連接實現方式
3.1 Nested Loop Join
將驅動表/外部表的結果集作為循環基礎數據，然后循環該結果集，每次獲取一條數據作為下一個表的過濾條件查詢數據，然后合並結果，獲取結果集返回給客戶端。Nested-Loop一次只將一行傳入內層循環, 所以外層循環(的結果集)有多少行, 內存循環便要執行多少次，效率非常差。
EXPLAIN SELECT * from sm_performance_all p LEFT JOIN sm_bill b ON p.bill_id > b.car_id where p.company_id>1024; +----+-------------+-------+------------+-------+------------------------------+------------------------------+---------+------+--------+----------+------------------------------------------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+-------+------------+-------+------------------------------+------------------------------+---------+------+--------+----------+------------------------------------------------+ | 1 | SIMPLE | p | NULL | range | FK_a9t29a4b2af1vfny1j2minc1x | FK_a9t29a4b2af1vfny1j2minc1x | 9 | NULL | 20263 | 100.00 | Using index condition; Using MRR | | 1 | SIMPLE | b | NULL | ALL | car_id_idx | NULL | NULL | NULL | 738383 | 100.00 | Range checked for each record (index map: 0x2) | +----+-------------+-------+------------+-------+------------------------------+------------------------------+---------+------+--------+----------+------------------------------------------------+ 2 rows in set, 1 warning (0.00 sec)
3.2 Block Nested-Loop Join
將外層循環的行/結果集存入join buffer, 內層循環的每一行與整個buffer中的記錄做比較，從而減少內層循環的次數。主要用於當被join的表上無索引。
CREATE TABLE t1 (a int PRIMARY KEY, b int); CREATE TABLE t2 (a int PRIMARY KEY, b int); INSERT INTO t1 VALUES (1,2), (2,1), (3,2), (4,3), (5,6), (6,5), (7,8), (8,7), (9,10); INSERT INTO t2 VALUES (3,0), (4,1), (6,4), (7,5); EXPLAIN SELECT * FROM t1 LEFT JOIN t2 ON t1.a = t2.a WHERE t2.b <= t1.a AND t1.a <= t1.b; +----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+----------------------------------------------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+----------------------------------------------------+ | 1 | SIMPLE | t1 | NULL | ALL | PRIMARY | NULL | NULL | NULL | 9 | 33.33 | Using where | | 1 | SIMPLE | t2 | NULL | ALL | PRIMARY | NULL | NULL | NULL | 4 | 25.00 | Using where; Using join buffer (Block Nested Loop) | +----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+----------------------------------------------------+ 2 rows in set, 1 warning (0.00 sec)
3.3 Batched Key Access
當被join的表能夠使用索引時，就先好順序，然后再去檢索被join的表。對這些行按照索引字段進行排序，因此減少了隨機IO。如果被Join的表上沒有索引，則使用老版本的BNL策略。
參考：