深度剖析HashMap的數據存儲實現原理(看完必懂篇)
具體的原理分析可以參考以下兩篇文章,有透徹的分析!本文在此基礎上加入個人理解和完善部分紕漏!!!
參考資料:
1. https://www.jianshu.com/p/17177c12f849 [JDK8中的HashMap實現原理及源碼分析]
2. https://tech.meituan.com/java-hashmap.html [Java 8系列之重新認識HashMap]
1、關鍵字段:
/**
* The default initial capacity - MUST be a power of two.
*/
static final int DEFAULT_INITIAL_CAPACITY = 1 << 4; // 2^4
/**
* The maximum capacity, used if a higher value is implicitly specified
* by either of the constructors with arguments.
* MUST be a power of two <= 1<<30.
*/
static final int MAXIMUM_CAPACITY = 1 << 30; // 2^30
/**
* The load factor used when none specified in constructor.
*/
static final float DEFAULT_LOAD_FACTOR = 0.75f;
/**
* The bin count threshold for using a tree rather than list for a
* bin. Bins are converted to trees when adding an element to a
* bin with at least this many nodes. The value must be greater
* than 2 and should be at least 8 to mesh with assumptions in
* tree removal about conversion back to plain bins upon
* shrinkage.
*
* 一個桶的樹化閾值
* 當桶中元素個數超過這個值時,需要使用紅黑樹節點替換鏈表節點
* 這個值必須為 8,要不然頻繁轉換效率也不高
*/
static final int TREEIFY_THRESHOLD = 8;
/**
* The bin count threshold for untreeifying a (split) bin during a
* resize operation. Should be less than TREEIFY_THRESHOLD, and at
* most 6 to mesh with shrinkage detection under removal.
*
* 一個樹的鏈表還原閾值
* 當擴容時,桶中元素個數小於這個值,就會把樹形的桶元素 還原(切分)為鏈表結構
* 這個值應該比上面那個小,至少為 6,避免頻繁轉換
*/
static final int UNTREEIFY_THRESHOLD = 6;
/**
* The smallest table capacity for which bins may be treeified.
* (Otherwise the table is resized if too many nodes in a bin.)
* Should be at least 4 * TREEIFY_THRESHOLD to avoid conflicts
* between resizing and treeification thresholds.
*
* 哈希表的最小樹形化容量
* 當哈希表中的容量大於這個值時,表中的桶才能進行樹形化
* 否則桶內元素太多時會擴容,而不是樹形化
* 為了避免進行擴容、樹形化選擇的沖突,這個值不能小於 4 * TREEIFY_THRESHOLD
*/
static final int MIN_TREEIFY_CAPACITY = 64;
/* ---------------- Fields -------------- */
/**
* The table, initialized on first use, and resized as
* necessary. When allocated, length is always a power of two.
* (We also tolerate length zero in some operations to allow
* bootstrapping mechanics that are currently not needed.)
*
* 為了更好表示本文稱之為桶數組
*/
transient Node<K,V>[] table;
/**
* Holds cached entrySet(). Note that AbstractMap fields are used
* for keySet() and values().
*/
transient Set<Map.Entry<K,V>> entrySet;
/**
* The number of key-value mappings contained in this map.
*/
transient int size;
/**
* The number of times this HashMap has been structurally modified
* Structural modifications are those that change the number of mappings in
* the HashMap or otherwise modify its internal structure (e.g.,
* rehash). This field is used to make iterators on Collection-views of
* the HashMap fail-fast. (See ConcurrentModificationException).
*/
transient int modCount;
/**
* The next size value at which to resize (capacity * load factor).
*
* @serial
*/
// (The javadoc description is true upon serialization.
// Additionally, if the table array has not been allocated, this
// field holds the initial array capacity, or zero signifying
// DEFAULT_INITIAL_CAPACITY.)
int threshold;
/**
* The load factor for the hash table.
*
* @serial
*/
final float loadFactor;
/**
* Constructs an empty <tt>HashMap</tt> with the specified initial
* capacity and load factor.
*
* @param initialCapacity the initial capacity
* @param loadFactor the load factor
* @throws IllegalArgumentException if the initial capacity is negative
* or the load factor is nonpositive
*/
public HashMap(int initialCapacity, float loadFactor) {
if (initialCapacity < 0)
throw new IllegalArgumentException("Illegal initial capacity: " +
initialCapacity);
if (initialCapacity > MAXIMUM_CAPACITY)
initialCapacity = MAXIMUM_CAPACITY;
if (loadFactor <= 0 || Float.isNaN(loadFactor))
throw new IllegalArgumentException("Illegal load factor: " +
loadFactor);
this.loadFactor = loadFactor;
this.threshold = tableSizeFor(initialCapacity);
}
/**
* Constructs an empty <tt>HashMap</tt> with the specified initial
* capacity and the default load factor (0.75).
*
* @param initialCapacity the initial capacity.
* @throws IllegalArgumentException if the initial capacity is negative.
*/
public HashMap(int initialCapacity) {
this(initialCapacity, DEFAULT_LOAD_FACTOR);
}
/**
* Constructs an empty <tt>HashMap</tt> with the default initial capacity
* (16) and the default load factor (0.75).
*/
public HashMap() {
this.loadFactor = DEFAULT_LOAD_FACTOR; // all other fields defaulted
}
/**
* Constructs a new <tt>HashMap</tt> with the same mappings as the
* specified <tt>Map</tt>. The <tt>HashMap</tt> is created with
* default load factor (0.75) and an initial capacity sufficient to
* hold the mappings in the specified <tt>Map</tt>.
*
* @param m the map whose mappings are to be placed in this map
* @throws NullPointerException if the specified map is null
*/
public HashMap(Map<? extends K, ? extends V> m) {
this.loadFactor = DEFAULT_LOAD_FACTOR;
putMapEntries(m, false);
}
2、首先針對很多文章中的紕漏語句:如果一個桶中鏈表內的元素個數超過 TREEIFY_THRESHOLD(默認是8),就使用紅黑樹來替換鏈表。
// 插入圖片1張
圖片中紅色標記的地方個人理解是不夠嚴謹的!!!數據插入HashMap的時候,如果當前桶中的元素個數 > TREEIFY_THRESHOLD時,則會進行桶的樹形化處理(見代碼片段1:treeifyBin())。
注意這里只是進行桶的樹形化處理,並不是把桶(如果是鏈表結構)直接轉換為紅黑樹,這里面是有條件的!!!具體規則如下:
條件1. 如果當前桶數組為null或者桶數組的長度 < MIN_TREEIFY_CAPACITY,則進行擴容處理(見代碼片段2:resize());
條件2. 當不滿足條件1的時候則將桶中鏈表內的元素轉換成紅黑樹!!!稍后再詳細討論紅黑樹。
3、再來分析下HashMap擴容機制的實現:
概念:
1. 擴容(resize)就是重新計算容量。當向HashMap對象里不停的添加元素,而HashMap對象內部的桶數組無法裝載更多的元素時,HashMap對象就需要擴大桶數組的長度,以便能裝入更多的元素。
2. capacity 就是數組的長度/大小,loadFactor 是這個數組填滿程度的最大比比例。
3. size表示當前HashMap中已經儲存的Node<key,value>的數量,包括桶數組和鏈表 / 紅黑樹中的的Node<key,value>。
4. threshold表示擴容的臨界值,如果size大於這個值,則必需調用resize()方法進行擴容。
5. 在jdk1.7及以前,threshold = capacity * loadFactor,其中 capacity 為桶數組的長度。
這里需要說明一點,默認負載因子0.75是是對空間和時間(縱向橫向)效率的一個平衡選擇,建議大家不要修改。
jdk1.8對threshold值進行了改進,通過一系列位移操作算法最后得到一個power of two size的值,見代碼片段4。
擴容過程:
1. 使用new Hashap<>()時,新桶數組初始容量設置為默認值DEFAULT_INITIAL_CAPACITY,默認容量下的閾值為DEFAULT_LOAD_FACTOR * DEFAULT_INITIAL_CAPACITY。
2. 使用new Hashap<>(int initialCapacity)或new HashMap(int initialCapacity, float loadFactor)時,newCap, newThr均重新計算。
3. 如果使用過程中HashMap中的數據過多,原始容量不夠使用,那么需要擴容。擴容是以2^1為單位擴容的, newCap = oldCap << 1和newThr = oldThr << 1。
4. 如果原來的桶數組長度大於最大值MAXIMUM_CAPACITY時,擴容臨界值提高到正無窮(Integer.MAX_VALUE),返回原來的數組,也就是系統已經管不了了,隨便你怎么玩吧。
正常擴容之后需要將老的桶數組數據重新放到新的桶數組中,同時對每個桶上的鏈表進行了重排,再介紹重排之前先來看看代碼片段5里面的hash()計算:
首先將得到key對應的哈希值:h = key.hashCode(),然后通過hashCode()的高16位異或低16位計算得到最終的key.hash值((h = key.hashCode()) ^ (h >>> 16))。
1. 取key的hashcode值:
① Object類的hashCode
返回對象的經過處理后的內存地址,由於每個對象的內存地址都不一樣,所以哈希碼也不一樣。這個是native方法,取決於JVM的內部設計,一般是某種C地址的偏移。
② String類的hashCode
根據String類包含的字符串的內容,根據一種特殊算法返回哈希碼,只要字符串的內容相同,返回的哈希碼也相同。
③ Integer等包裝類
返回的哈希碼就是Integer對象里所包含的那個整數的數值,例如Integer i1=new Integer(100),i1.hashCode的值就是100。
由此可見,2個一樣大小的Integer對象,返回的哈希碼也一樣。
④ int,char這樣的基礎類
它們不需要hashCode,如果需要存儲時,將進行自動裝箱操作,計算方法包裝類。
2. hashCode()的高16位異或低16位
在JDK1.8的實現中,優化了高位運算的算法,通過hashCode()的高16位異或低16位實現的:key.hash = (h = k.hashCode()) ^ (h >>> 16),
主要是從速度、功效、質量來考慮的,這么做可以在數組table的length比較小的時候,也能保證考慮到高低Bit都參與到Hash的計算中,同時不會有太大的開銷。
3. key.hash & (n - 1) 取模運算
這個n我們說過是table的長度,那么n-1就是table數組元素應有的下表。這個方法非常巧妙,它通過 key.hash & (table.length - 1) 來得到該對象的保存位,
而HashMap底層數組的長度總是2的n次方,這是HashMap在速度上的優化。當length總是2的n次方時,key.hash & (table.length - 1) 運算等價於對length取模,也就是key.hash % length,但是&比%具有更高的效率。
鏈表重排:
1. 如果原桶上只有一個節點,並且該節點不是紅黑樹節點,那么直接放到新桶原索引key.hash & (table.length - 1)下;
2. 如果原桶上的節點是紅黑樹節點,那么則對該樹進行分割split();
3. 如果原桶上的節點是一個鏈表,則進行鏈表重排算法:
由於桶數組的容量是按2次冪的擴展(指容量擴為原來2倍),所以,元素的位置要么是在“原索引”,要么是在“原索引 + oldCap”的位置。
所以,只需要看看原來key.hash值新增的那個bit是1還是0就好了,是0的話索引沒變,是1的話索引變成“原索引 + oldCap”。
// 插入圖片2張
4、HashMap的數據存儲實現原理
流程:
1. 根據key計算得到key.hash = (h = k.hashCode()) ^ (h >>> 16);
2. 根據key.hash計算得到桶數組的索引index = key.hash & (table.length - 1),這樣就找到該key的存放位置了:
① 如果該位置沒有數據,用該數據新生成一個節點保存新數據,返回null;
② 如果該位置有數據是一個紅黑樹,那么執行相應的插入 / 更新操作,稍后再詳細討論紅黑樹;
③ 如果該位置有數據是一個鏈表,分兩種情況一是該鏈表沒有這個節點,另一個是該鏈表上有這個節點,注意這里判斷的依據是key.hash是否一樣:
如果該鏈表沒有這個節點,那么采用尾插法新增節點保存新數據,返回null;
如果該鏈表已經有這個節點了,那么找到該節點並更新新數據,返回老數據。
注意:
HashMap的put會返回key的上一次保存的數據,比如:
HashMap<String, String> map = new HashMap<String, String>();
System.out.println(map.put("a", "A")); // 打印null
System.out.println(map.put("a", "AA")); // 打印A
System.out.println(map.put("a", "AB")); // 打印AA
5、紅黑樹
上面的討論中對於紅黑樹並沒有深入分析,HashMap的數據存儲中主要有兩種場景用到紅黑樹的操作:
場景1. 當滿足一定條件(條件2,見上文)時,單鏈表內的數據會轉換為紅黑樹存儲(見代碼片段2:treeifyBin())。
場景2. 當HashMap桶結構由鏈表轉換為紅黑樹后,再往里put數據將變成往紅黑樹插入 / 更新數據,這和鏈表又不太一樣了。
下面進行逐一詳細分析:
場景1:
代碼片段2:treeifyBin()樹形化函數完成的第一件事是將單鏈表的Node節點轉換為紅黑樹TreeNode節點,然后將其首位相連,並將桶指向紅黑樹的頭結點。
其實此時所謂的紅黑樹只是一個雙向鏈表,並不嚴格意義上的紅黑樹結構!!! 因為此時每個節點只有prev和next有值,那么接着就需要對這個雙向鏈表樹形化將其塑造成一個標准的紅黑樹。
這時候用到了函數treeify(),轉換過程未完待續。。。。。。
場景2:
這時候用到了函數putTreeVal(),處理過程未完待續。。。。。。
源碼片段1:
/**
* Associates the specified value with the specified key in this map.
* If the map previously contained a mapping for the key, the old
* value is replaced.
*
* @param key key with which the specified value is to be associated
* @param value value to be associated with the specified key
* @return the previous value associated with <tt>key</tt>, or
* <tt>null</tt> if there was no mapping for <tt>key</tt>.
* (A <tt>null</tt> return can also indicate that the map
* previously associated <tt>null</tt> with <tt>key</tt>.)
*/
public V put(K key, V value) {
return putVal(hash(key), key, value, false, true);
}
/**
* Implements Map.put and related methods
*
* @param hash hash for key
* @param key the key
* @param value the value to put
* @param onlyIfAbsent if true, don't change existing value
* @param evict if false, the table is in creation mode.
* @return previous value, or null if none
*/
final V putVal(int hash, K key, V value, boolean onlyIfAbsent,
boolean evict) {
Node<K,V>[] tab; Node<K,V> p; int n, i;
if ((tab = table) == null || (n = tab.length) == 0)
n = (tab = resize()).length;
if ((p = tab[i = (n - 1) & hash]) == null)
tab[i] = newNode(hash, key, value, null);
else {
Node<K,V> e; K k;
if (p.hash == hash &&
((k = p.key) == key || (key != null && key.equals(k))))
e = p;
else if (p instanceof TreeNode)
e = ((TreeNode<K,V>)p).putTreeVal(this, tab, hash, key, value);
else {
for (int binCount = 0; ; ++binCount) {
if ((e = p.next) == null) {
p.next = newNode(hash, key, value, null);
if (binCount >= TREEIFY_THRESHOLD - 1) // -1 for 1st
treeifyBin(tab, hash);
break;
}
if (e.hash == hash &&
((k = e.key) == key || (key != null && key.equals(k))))
break;
p = e;
}
}
if (e != null) { // existing mapping for key
V oldValue = e.value;
if (!onlyIfAbsent || oldValue == null)
e.value = value;
afterNodeAccess(e);
return oldValue;
}
}
++modCount;
if (++size > threshold)
resize();
afterNodeInsertion(evict);
return null;
}
源碼片段2:
//將桶內所有的 鏈表節點 替換成 紅黑樹節點
final void treeifyBin(Node<K,V>[] tab, int hash) {
int n, index; Node<K,V> e;
//如果當前哈希表為空,或者哈希表中元素的個數小於進行樹形化的閾值(默認為64),就去新建/擴容
if (tab == null || (n = tab.length) < MIN_TREEIFY_CAPACITY)
resize();
else if ((e = tab[index = (n - 1) & hash]) != null) {
// 如果哈希表中的元素個數超過了 樹形化閾值,進行樹形化
// e 是哈希表中指定位置桶里的鏈表節點,從頭結點開始
TreeNode<K,V> hd = null, tl = null; // 紅黑樹的頭、尾節點
do {
// 新建一個紅黑樹節點,內容和當前鏈表節點e一致
TreeNode<K,V> p = replacementTreeNode(e, null);
if (tl == null)// 確定紅黑樹頭節點
hd = p;
else {
p.prev = tl;
tl.next = p;
}
tl = p;
} while ((e = e.next) != null);
// 讓桶的第一個元素指向新建的紅黑樹頭結點,以后這個桶里的元素就是紅黑樹而不是鏈表了
if ((tab[index] = hd) != null)
hd.treeify(tab);
}
}
源碼片段3:
/**
* Initializes or doubles table size. If null, allocates in
* accord with initial capacity target held in field threshold.
* Otherwise, because we are using power-of-two expansion, the
* elements from each bin must either stay at same index, or move
* with a power of two offset in the new table.
*
* @return the table
*/
final Node<K,V>[] resize() {
Node<K,V>[] oldTab = table;
int oldCap = (oldTab == null) ? 0 : oldTab.length;
int oldThr = threshold;
int newCap, newThr = 0;
if (oldCap > 0) {
if (oldCap >= MAXIMUM_CAPACITY) {
threshold = Integer.MAX_VALUE;
return oldTab;
}
else if ((newCap = oldCap << 1) < MAXIMUM_CAPACITY &&
oldCap >= DEFAULT_INITIAL_CAPACITY)
newThr = oldThr << 1; // double threshold
}
else if (oldThr > 0) // initial capacity was placed in threshold
newCap = oldThr;
else { // zero initial threshold signifies using defaults
newCap = DEFAULT_INITIAL_CAPACITY;
newThr = (int)(DEFAULT_LOAD_FACTOR * DEFAULT_INITIAL_CAPACITY);
}
if (newThr == 0) {
float ft = (float)newCap * loadFactor;
newThr = (newCap < MAXIMUM_CAPACITY && ft < (float)MAXIMUM_CAPACITY ?
(int)ft : Integer.MAX_VALUE);
}
threshold = newThr;
@SuppressWarnings({"rawtypes","unchecked"})
Node<K,V>[] newTab = (Node<K,V>[])new Node[newCap];
table = newTab;
if (oldTab != null) {
for (int j = 0; j < oldCap; ++j) {
Node<K,V> e;
if ((e = oldTab[j]) != null) {
oldTab[j] = null;
if (e.next == null)
newTab[e.hash & (newCap - 1)] = e;
else if (e instanceof TreeNode)
((TreeNode<K,V>)e).split(this, newTab, j, oldCap);
else { // preserve order
Node<K,V> loHead = null, loTail = null;
Node<K,V> hiHead = null, hiTail = null;
Node<K,V> next;
do {
next = e.next;
// 原索引
if ((e.hash & oldCap) == 0) {
if (loTail == null)
loHead = e;
else
loTail.next = e;
loTail = e;
}
// 原索引 + oldCap
else {
if (hiTail == null)
hiHead = e;
else
hiTail.next = e;
hiTail = e;
}
} while ((e = next) != null);
// 原索引放到桶數組里
if (loTail != null) {
loTail.next = null;
newTab[j] = loHead;
}
// 原索引 + oldCap放到桶數組里
if (hiTail != null) {
hiTail.next = null;
newTab[j + oldCap] = hiHead;
}
}
}
}
}
return newTab;
}
源碼片段4:
/**
* Returns a power of two size for the given target capacity.
*/
static final int tableSizeFor(int cap) {
int n = cap - 1;
n |= n >>> 1;
n |= n >>> 2;
n |= n >>> 4;
n |= n >>> 8;
n |= n >>> 16;
return (n < 0) ? 1 : (n >= MAXIMUM_CAPACITY) ? MAXIMUM_CAPACITY : n + 1;
}
源碼片段5:
/**
* Computes key.hashCode() and spreads (XORs) higher bits of hash
* to lower. Because the table uses power-of-two masking, sets of
* hashes that vary only in bits above the current mask will
* always collide. (Among known examples are sets of Float keys
* holding consecutive whole numbers in small tables.) So we
* apply a transform that spreads the impact of higher bits
* downward. There is a tradeoff between speed, utility, and
* quality of bit-spreading. Because many common sets of hashes
* are already reasonably distributed (so don't benefit from
* spreading), and because we use trees to handle large sets of
* collisions in bins, we just XOR some shifted bits in the
* cheapest possible way to reduce systematic lossage, as well as
* to incorporate impact of the highest bits that would otherwise
* never be used in index calculations because of table bounds.
*/
static final int hash(Object key) {
int h;
return (key == null) ? 0 : (h = key.hashCode()) ^ (h >>> 16);
}
謝謝!歡迎批評指正!!!