PG內存上下文管理（MemoryContext）——AllocSetContext：抽象類MemoryContext的實現

本文轉載自查看原文 2020-12-03 23:52 356 PostgreSQL/ PostgreSQL源碼分析

　　MemoryContext是一個抽象類，可以有對個實現，但目前只有AllocSetContext一個實現。事實上，MemoryContext並不管理實際上的內存分配，僅僅是用作對MemoryContext樹的控制。管理一個內存上下文中的內存塊是通過AllocSet結構來完成的，而MemoryContext僅作為AllocSet的頭部信息存在，AllocSet是一個指向AllocSetContext結構的類型指針。

SET結構

header就是一個MemoryContextData結構，header是進入一個內存上下文的唯一外部接口，事實上管理內存上下文的接口函數都是通過對header的管理來實現。內存塊鏈表blocks為一個指向AllocBlockData結構體的指針，AllocBlockData用於表示一個內存塊。AllocBlockData之間通過其next字段鏈接成一個單向鏈表，而AllocSet的blocks字段則指向這個鏈表的頭部。

 1 typedef struct AllocSetContext {
 2     MemoryContextData header; //對應於該內存上下文的頭部信息
 3     AllocBlock    blocks; /*head of list of blocks in this set*/ 該內存上下文中所有內存塊的鏈表
 4     AllocChunk freelist[ALLOCSET_NUM_FREELISTS]; /*free chunk lists*/ 該內存上下文中空閑內存片的數組
 5     bool isReset; /*T = no space alloced since last reset*/ 如果為真表示從上次重置以來沒有分配過內存
 6     /* Allocation parameters for this context: */
 7     Size initBlockSize;  /*initial block size*/ 初始內存塊的大小
 8     Size maxBlockSize;/*maximum block size*/ 最大內存塊大小
 9     Size nextBlockSize;/*next block size to allocate*/ 下一個要分配的內存塊的大小
10     Size allocChunkLimit;    /*effective chunk size limit*/ 分配內存片的尺寸閾值，該值在分配內存片時會用到
11     AllocBlock    keeper; /*if not NULL, keep this block over resets */ 保存在keeper中的內存塊在內存上下文重置時會被保留不做釋放
12 } AllocSetContext;

isReset：PG中提供了對內存上下文的重置操作，所謂重置就是釋放內存上下文中所有分配的內存，並將這些內存交還給操作系統。在一個內存上下文被創建時，其isReset字段置為True，表示從上一次重置到當前沒有內存被分配。只要在該內存上下文中進行了分配，則將其isReset字段置為False。這樣在進行重置時，可以檢查內存上下文的isReset字段，如果為True則表示該內存上下文中沒有進行過內存分配，所以不需要進行實際的重置工作，從而提高操作效率。

initBlockSize、maxBlockSize、nextBlockSize：initBlockSize和maxBlockSize字段在內存上下文創建時指定，且在創建時nextBlockSize會置為與initBlockSize相同的值。nextBlockSize表示下一次分配的內存塊的大小，在進行內存分配時，如果需要分配一個新的內存塊，則這個新內存塊的大小將采用nextBlockSize的值。maxBlockSize字段指定了內存塊可以到達的最大尺寸。

allocChunkLimit：內存塊內會分成多個稱為內存片的內存單元，在分配內存片時，如果一個內存片的尺寸超過了宏ALLOC_CHUNK_LIMIT時，將會為該內存片單獨分配一個獨立的內存塊，這樣做是為了避免日后進行內存回收時造成過多的碎片。由於宏ALLOC_CHUNK_LIMIT是不能在運行時更改的，因此PG提供了allocChunkLimit用於自定義一個閾值。如果定義了這個字段的值，則在進行超限檢查時會用該字段來替換宏定義進行判斷。

keeper：在內存上下文進行重置時不會對keeper中記錄的內存塊進行釋放，而是對其內容進行清空。這樣可以保證內存上下文重置結束后就已經包含一定的可用內存空間，而不需要通過malloc另行申請。另外也可以避免在某個內存上下文被反復重置時，反復進行malloc帶來的風險。

Block結構

由aset.c通過malloc申請的內存單元（內存塊），它包含了多個由palloc申請的內存片(AllocChunk)。內存片由pfree函數釋放到freelist中，由下一次palloc調用重新使用。AllocBlockData記錄在一塊內存區域的起始地址處，這塊內存區域通過標准庫函數malloc進行分配，稱為一個內存塊。在每個內存塊中進行內存分配時產生的內存片段稱之為內存片，freeptr指向該塊空閑區域的首地址，endptr指向該內存塊的末地址。

1 typedef struct AllocBlockData {
2     AllocSet    aset;     /* aset that owns this block */ 該內存塊所在AllocSet
3     AllocBlock    next; /* next block in aset's blocks list */ 指向下一個內存塊的指針
4     char       *freeptr; /* start of free space in this block */ 指向該塊空閑區域的首地址
5     char       *endptr;     /* end of space in this block */ 該內存塊的末地址
6 } AllocBlockData;
typedef AllocSetContext *AllocSet;

Chunk結構

每個內存片包括一個頭部信息和數據區域，其中頭部信息包括該內存片所屬的內存上下文以及該內存區的其他相關信息，數據區則存儲實際數據。內存片的頭部信息由數據結構AllocChunkData描述，內存片的數據區域則緊跟在其頭部信息之后分配。通過PG中定義的palloc函數和pfree函數，可以自由地在內存上下文中申請和釋放內存片，被釋放的內存片將被加入到FreeList中以被重復使用。

 1 typedef struct AllocChunkData {
 2     /* aset is the owning aset if allocated, or the freelist link if free */
 3     void       *aset;
 4     /* size is always the size of the usable space in the chunk */
 5     Size        size;
 6 #ifdef MEMORY_CONTEXT_CHECKING
 7     /* when debugging memory usage, also store actual requested size */
 8     /* this is zero in a free chunk */
 9     Size        requested_size;
10 #endif
11 } AllocChunkData;
typedef struct AllocChunkData *AllocChunk;

FreeList結構

FreeList數組用於維護在內存塊中被回收的空閑內存片，這些空閑內存片將被用於再分配。FreeList數組元素類型為AllocChunk，數組長度默認為11（由宏ALLOCSET_NUM_FREELIST定義）。FreeList數組中的每個元素指向一個由特定大小空閑內存片組成的鏈表，這個大小與該元素在數組中的順序有關。比如，FreeList數組中第K個元素所指向鏈表的每個空閑數據塊的大小為2^(k+2)字節，空閑內存片最小為8字節，最大不超過8K字節。因此，FreeList數組中實際上維護了11個空閑鏈表。通過aset字段來鏈接空閑鏈表，如果一個內存正在使用，則它的aset字段指向其所屬的AllocSet。如果內存片是空閑的，也就是說它處於某個空閑鏈表中，那么它的aset字段指向空閑鏈表中在它之后的內存片。這樣從FreeList數組元素所指向的鏈表頭部開始，順着aset字段指向的下一個內存片就可以找到該空閑鏈表中所有的空閑內存片。

當需要申請一塊內存時，我們可以迅速地定位到相應的空閑鏈表中。對於一個大小為size的內存分配請求，將會在第K個空閑鏈表中為其分配內存片，K的計算規則是：

當size<(1<<ALLOC_MINBITS)時，K為0，其中ALLOC_MINBITS系統預定為3，即表是FreeList中保存的最小字節數位2^3=8字節
當(1<<ALLOC_MINBITS)<size<ALLOC_CHUNK_LIMIT時，若2^(N-1)<size<2^N，則K=N-3，其中ALLOC_MINBITS為FreeList中所維持空閑內存片的最大值

 1 static inline int AllocSetFreeIndex(Size size) {
 2     int            idx = 0;
 3     if (size > 0) {
 4         size = (size - 1) >> ALLOC_MINBITS;
 5         while (size != 0) {
 6             idx++;
 7             size >>= 1;
 8         }
 9         Assert(idx < ALLOCSET_NUM_FREELISTS);
10     }
11     return idx;
12 }

在創建AllocSet時，可以為其指定allocChunkLimit字段的值，如果申請分配的內存大小超過這個值，那么將為這次請求分配一個獨立的內存塊，這個塊中只存放一個內存片，當該內存片釋放的時候，整個內存塊將被釋放，而不是將內存片加到FreeList中。一個內存片的大小不會超過ALLOC_CHUNK_LIMIT，否則將會被當作一個獨立的塊來看待。

API

MemoryContext API for AllocSet contexts 用於初始化AllocSetContext中MemoryContext的內存處理函數指針

 1 static MemoryContextMethods AllocSetMethods = {
 2     AllocSetAlloc,
 3     AllocSetFree,   // 釋放一個內存上下文中指定的內存片
 4     AllocSetRealloc,
 5     AllocSetInit,
 6     AllocSetReset,  // 重置內存上下文
 7     AllocSetDelete, // 釋放當前內存上下文中全部內存塊
 8     AllocSetGetChunkSpace,
 9     AllocSetIsEmpty,
10     AllocSetStats
11 #ifdef MEMORY_CONTEXT_CHECKING
12     ,AllocSetCheck
13 #endif
14 };
15 static void *AllocSetAlloc(MemoryContext context, Size size);
16 static void AllocSetFree(MemoryContext context, void *pointer);
17 static void *AllocSetRealloc(MemoryContext context, void *pointer, Size size);
18 static void AllocSetInit(MemoryContext context);
19 static void AllocSetReset(MemoryContext context);
20 static void AllocSetDelete(MemoryContext context);
21 static Size AllocSetGetChunkSpace(MemoryContext context, void *pointer);
22 static bool AllocSetIsEmpty(MemoryContext context);
23 static void AllocSetStats(MemoryContext context, int level);
24 #ifdef MEMORY_CONTEXT_CHECKING
25 static void AllocSetCheck(MemoryContext context);
26 #endif

　　AllocSetInit函數在建立generic內存上下文，將新內存上下文鏈接到上下文樹之前由MemoryContexCreate函數調用。

1 static void AllocSetInit(MemoryContext context) {
2     /* Since MemoryContextCreate already zeroed the context node, we don't have to do anything here: it's already OK. */
3 }

　　AllocSetReset函數釋放在給定的set中分配的內存，在進行重置時，內存上下文中除了在keeper字段中指定要保留的內存塊外，其他內存塊全部釋放，包括空閑鏈表中的內存。keeper中指定保留的內存塊被清空內容，它使得內存上下文重置之后就立刻有一塊內存可供使用。

 1 static void AllocSetReset(MemoryContext context) {
 2     AllocSet    set = (AllocSet) context;
 3     AllocBlock    block;
 4     AssertArg(AllocSetIsValid(set));
 5     /* Nothing to do if no pallocs since startup or last reset */
 6     if (set->isReset)
 7         return;
 8 #ifdef MEMORY_CONTEXT_CHECKING
 9     /* Check for corruption and leaks before freeing */
10     AllocSetCheck(context);
11 #endif
12     /* Clear chunk freelists */
13     MemSetAligned(set->freelist, 0, sizeof(set->freelist));
14     block = set->blocks;
15     /* New blocks list is either empty or just the keeper block */
16     set->blocks = set->keeper;
17     while (block != NULL) {
18         AllocBlock    next = block->next;
19         if (block == set->keeper) {
20             /* Reset the block, but don't return it to malloc */
21             char       *datastart = ((char *) block) + ALLOC_BLOCKHDRSZ;
22 #ifdef CLOBBER_FREED_MEMORY
23             /* Wipe freed memory for debugging purposes */
24             memset(datastart, 0x7F, block->freeptr - datastart);
25 #endif
26             block->freeptr = datastart;
27             block->next = NULL;
28         } else {
29             /* Normal case, release the block */
30 #ifdef CLOBBER_FREED_MEMORY
31             /* Wipe freed memory for debugging purposes */
32             memset(block, 0x7F, block->freeptr - ((char *) block));
33 #endif
34             free(block);
35         }
36         block = next;
37     }
38     /* Reset block size allocation sequence, too */
39     set->nextBlockSize = set->initBlockSize;
40     set->isReset = true;
41 }

　　AllocSetDelete函數釋放當前內存上下文中的全部內存塊，包括keeper指定的內存塊在內。但內存上下文節點並不釋放，因為內存上下文是在TopMemoryContext中申請的內存，將在進程運行結束時統一釋放。

 1 static void AllocSetDelete(MemoryContext context) {
 2     AllocSet    set = (AllocSet) context;
 3     AllocBlock    block = set->blocks;
 4     AssertArg(AllocSetIsValid(set));
 5 #ifdef MEMORY_CONTEXT_CHECKING
 6     /* Check for corruption and leaks before freeing */
 7     AllocSetCheck(context);
 8 #endif
 9     /* Make it look empty, just in case... */
10     MemSetAligned(set->freelist, 0, sizeof(set->freelist));
11     set->blocks = NULL;
12     set->keeper = NULL;
13     while (block != NULL) {
14         AllocBlock    next = block->next;
15 #ifdef CLOBBER_FREED_MEMORY
16         /* Wipe freed memory for debugging purposes */
17         memset(block, 0x7F, block->freeptr - ((char *) block));
18 #endif
19         free(block);
20         block = next;
21     }
22 }

　　AllocSetContextCreate創建新的AllocSet上下文，參數parent是父內存上下文節點，或者為Null為top-level內存上下文，minContextSize內存上下文最小大小，initBlockSize初始分配塊大小，maxBlockSize最大分配塊大小。內存上下文的創建由AllocSetContextCreate函數完成，主要有兩個工作：創建內存上下文節點以及分配內存塊。內存上下文節點的創建由MemoryContextCreate函數來完成，主要流程如下：從TopMemoryContext節點中分配一塊內存用於存放內存上下文節點（該內存上下文管理的內存塊不包括在內），該內存塊略大於一個AllocSet結構體的大小；初始化MemoryContext節點，設置其父節點、節點類型、節點名等相關信息，還要設置該節點的methods屬性為AllocSetMethods。

 1 MemoryContext AllocSetContextCreate(MemoryContext parent,
 2                       const char *name,
 3                       Size minContextSize,
 4                       Size initBlockSize,
 5                       Size maxBlockSize) {
 6     AllocSet    context;
 7     /* Do the type-independent part of context creation */
 8     context = (AllocSet) MemoryContextCreate(T_AllocSetContext,sizeof(AllocSetContext),&AllocSetMethods,parent,name);
 9 
10     /* Make sure alloc parameters are reasonable, and save them. We somewhat arbitrarily enforce a minimum 1K block size. */
11     initBlockSize = MAXALIGN(initBlockSize);
12     if (initBlockSize < 1024)
13         initBlockSize = 1024;
14     maxBlockSize = MAXALIGN(maxBlockSize);
15     if (maxBlockSize < initBlockSize)
16         maxBlockSize = initBlockSize;
17     context->initBlockSize = initBlockSize;
18     context->maxBlockSize = maxBlockSize;
19     context->nextBlockSize = initBlockSize;
20 
21     /* Compute the allocation chunk size limit for this context.  It can't be more than ALLOC_CHUNK_LIMIT because of the fixed number of freelists. If maxBlockSize is small then requests exceeding the maxBlockSize should be treated as large chunks, too.    We have to have allocChunkLimit a power of two, because the requested and actually-allocated sizes of any chunk must be on the same side of the limit, else we get confused about whether the chunk is "big".*/
22     context->allocChunkLimit = ALLOC_CHUNK_LIMIT;
23     while (context->allocChunkLimit >
24            (Size) (maxBlockSize - ALLOC_BLOCKHDRSZ - ALLOC_CHUNKHDRSZ))
25         context->allocChunkLimit >>= 1;
26 
27     /* Grab always-allocated space, if requested */
28     if (minContextSize > ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ){
29         Size        blksize = MAXALIGN(minContextSize);
30         AllocBlock    block;
31         block = (AllocBlock) malloc(blksize);
32         if (block == NULL){
33             MemoryContextStats(TopMemoryContext);
34             ereport(ERROR,
35                 (errcode(ERRCODE_OUT_OF_MEMORY),
36                      errmsg("out of memory"),
37                      errdetail("Failed while creating memory context \"%s\".",
38                                name)));
39         }
40         block->aset = context;
41         block->freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ;
42         block->endptr = ((char *) block) + blksize;
43         block->next = context->blocks;
44         context->blocks = block;
45         /* Mark block as not to be released at reset time */
46         context->keeper = block;
47     }
48     context->isReset = true;
49     return (MemoryContext) context;
50 }

AllocSetContextCreate函數的處理流程如下：

1）調用MemoryContextCreate創建MemoryContext結構

2）將上一步創建的MemoryContext結構強制轉換為AllocSet結構體，填充AllocSet其他結構體元素的信息，包括最小塊大小、初始化塊大小以及最大塊大小，並根據最大塊大小設置其allocChunkLimit的值

3）如果minContextSize超過一定限制（內存塊block頭部信息尺寸和內存片chunk頭部信息尺寸之和）時，調用標准庫函數，以minContextSize為大小分配一個內存塊，並初始化塊結構體，加入AllocSet的內存塊鏈表中。需要注意的是，預分配的內存塊將被記入到內存上下文的keeper字段中，作為內存上下文的保留塊，以便重置內存上下文的時候，該內存塊不會被釋放。

　　AllocSetAlloc函數負責處理具體內存分配工作，該函數的參數為一個內存上下文節點以及需要申請的內存大小。具體的內存分配流程如下所示：判斷需要申請的內存大小是否超過當前內存上下文允許分配內存片的最大值。若超過則為其分配一個新的獨立的內存塊，然后在該內存塊中分配指定大小的內存片。接下來將該內存塊加入到內存鏈表中，最后設置內存上下文的isReset字段為False並返回內存片的指針。

 1 static void * AllocSetAlloc(MemoryContext context, Size size) {
 2     AllocSet    set = (AllocSet) context;
 3     AllocBlock    block;
 4     AllocChunk chunk;
 5     int    fidx;
 6     Size    chunk_size;
 7     Size    blksize;
 8     AssertArg(AllocSetIsValid(set));
 9     /* If requested size exceeds maximum for chunks, allocate an entire block for this request. */
10     if (size > set->allocChunkLimit) {
11         chunk_size = MAXALIGN(size);
12         blksize = chunk_size + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
13         block = (AllocBlock) malloc(blksize);
14         if (block == NULL) {
15             MemoryContextStats(TopMemoryContext);
16             ereport(ERROR(errcode(ERRCODE_OUT_OF_MEMORY),
17                      errmsg("out of memory"),
18                      errdetail("Failed on request of size %lu.",
19                                (unsigned long) size)));
20         }
21         block->aset = set;
22         block->freeptr = block->endptr = ((char *) block) + blksize;
23         chunk = (AllocChunk) (((char *) block) + ALLOC_BLOCKHDRSZ);
24         chunk->aset = set;
25         chunk->size = chunk_size;
26 #ifdef MEMORY_CONTEXT_CHECKING
27         chunk->requested_size = size;
28         /* set mark to catch clobber of "unused" space */
29         if (size < chunk_size)
30             ((char *) AllocChunkGetPointer(chunk))[size] = 0x7E;
31 #endif
32 #ifdef RANDOMIZE_ALLOCATED_MEMORY
33         /* fill the allocated space with junk */
34         randomize_mem((char *) AllocChunkGetPointer(chunk), size);
35 #endif
36         /* Stick the new block underneath the active allocation block, so that we don't lose the use of the space remaining therein. */
37         if (set->blocks != NULL) { // 新申請的block進行頭插
38             block->next = set->blocks->next; 
39             set->blocks->next = block;
40         } else {
41             block->next = NULL;
42             set->blocks = block;
43         }
44         set->isReset = false;
45         AllocAllocInfo(set, chunk);
46         return AllocChunkGetPointer(chunk);
47     }

計算申請的內存大小在FreeList數組中對應的位置，如果存在合適的空閑內存片，則將空閑鏈表的指針（Freelist數組的某個元素）指向該內存片的aset字段所指向的地址（在空閑內存片中，aset字段指向它在空閑鏈表中的下一個內存片）。然后將該內存片的aset字段指向其所屬的內存上下文節點，最后返回該內存片的指針。

 1     fidx = AllocSetFreeIndex(size);
 2     chunk = set->freelist[fidx];
 3     if (chunk != NULL){
 4         Assert(chunk->size >= size);
 5         set->freelist[fidx] = (AllocChunk) chunk->aset;
 6         chunk->aset = (void *) set;
 7 #ifdef MEMORY_CONTEXT_CHECKING
 8         chunk->requested_size = size;
 9         /* set mark to catch clobber of "unused" space */
10         if (size < chunk->size)
11             ((char *) AllocChunkGetPointer(chunk))[size] = 0x7E;
12 #endif
13 #ifdef RANDOMIZE_ALLOCATED_MEMORY
14         /* fill the allocated space with junk */
15         randomize_mem((char *) AllocChunkGetPointer(chunk), size);
16 #endif
17         /* isReset must be false already */
18         Assert(!set->isReset);
19         AllocAllocInfo(set, chunk);
20         return AllocChunkGetPointer(chunk);
21     }

如果空閑鏈表中沒有滿足要求的內存片，則對內存上下文的內存塊鏈表（blocks字段）的第一個內存塊進行檢查，如果該內存塊中的未分配空間足以滿足申請的內存，則直接在該內存塊中分配內存片並返回內存片的指針。這里可以看到，在內存上下文中進行內存分配時，總是在內存塊鏈表中的第一個內存塊中進行，當該內存塊中空間用完之后會分配新的內存塊並作為新的內存塊鏈表首部，因此內存塊鏈表中的第一塊也稱作活動內存塊。

 1     chunk_size = (1 << ALLOC_MINBITS) << fidx;
 2     Assert(chunk_size >= size);
 3     /* If there is enough room in the active allocation block, we will put the chunk into that block.  Else must start a new one. */
 4     if ((block = set->blocks) != NULL) {
 5         Size    availspace = block->endptr - block->freeptr;
 6         if (availspace < (chunk_size + ALLOC_CHUNKHDRSZ)){
 7             while (availspace >= ((1 << ALLOC_MINBITS) + ALLOC_CHUNKHDRSZ)) {
 8                 Size    availchunk = availspace - ALLOC_CHUNKHDRSZ;
 9                 int    a_fidx = AllocSetFreeIndex(availchunk);
10                 if (availchunk != (1 << (a_fidx + ALLOC_MINBITS))){
11                     a_fidx--;
12                     Assert(a_fidx >= 0);
13                     availchunk = (1 << (a_fidx + ALLOC_MINBITS));
14                 }
15                 chunk = (AllocChunk) (block->freeptr);
16                 block->freeptr += (availchunk + ALLOC_CHUNKHDRSZ);
17                 availspace -= (availchunk + ALLOC_CHUNKHDRSZ);
18                 chunk->size = availchunk;
19 #ifdef MEMORY_CONTEXT_CHECKING
20                 chunk->requested_size = 0;        /* mark it free */
21 #endif
22                 chunk->aset = (void *) set->freelist[a_fidx];
23                 set->freelist[a_fidx] = chunk;
24             }
25             /* Mark that we need to create a new block */
26             block = NULL;  // 現有的內存塊都不能滿足這一次內存分配的要求
27         }
28     }

while (availspace >= ((1 << ALLOC_MINBITS) + ALLOC_CHUNKHDRSZ))的邏輯是：當前active(top)塊沒有足夠的內存用於分配，但是其中可能仍然有足夠的空間。一旦我們將它放入block list中，就不會嘗試從其中分配空間。按照空閑內存片鏈表所需的大小規則將空閑空間分片，並將它們放入freelist。

如果內存塊鏈表中的第一個內存塊沒有足夠的未分配空間，由於現有的內存塊都不能滿足這一次內存分配的要求，因此需要申請新的內存塊，但是當前的活動內存塊中還有未分配空間，如果申請新的內存塊並將之作為新的活動內存塊，則當前活動內存塊中的未分配空間就會被浪費。為了避免浪費，這里會先將當前活動內存塊中的未分配空間分解成個數盡可能少的內存片（即每個內存片盡可能大），並將它們加入到FreeList數組中，然后創建一個新的內存塊（其大小為前一次分配的內存塊的兩倍，但不超過maxBlockSize）並將之作為新的活動內存塊（即加入到內存塊鏈表首部）。最后再活動內存塊中分配一個滿足申請內存大小的內存片，並返回其指針。

 1     if (block == NULL) {
 2         Size    required_size;
 3         /* The first such block has size initBlockSize, and we double the space in each succeeding block, but not more than maxBlockSize. */
 4         blksize = set->nextBlockSize;
 5         set->nextBlockSize <<= 1;
 6         if (set->nextBlockSize > set->maxBlockSize)
 7             set->nextBlockSize = set->maxBlockSize;
 8 
 9         /* If initBlockSize is less than ALLOC_CHUNK_LIMIT, we could need more space... but try to keep it a power of 2. */
10         required_size = chunk_size + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
11         while (blksize < required_size)
12             blksize <<= 1;
13 
14         /* Try to allocate it */
15         block = (AllocBlock) malloc(blksize);
16 
17         /*
18          * We could be asking for pretty big blocks here, so cope if malloc
19          * fails.  But give up if there's less than a meg or so available...
20          */
21         while (block == NULL && blksize > 1024 * 1024) {
22             blksize >>= 1;
23             if (blksize < required_size)
24                 break;
25             block = (AllocBlock) malloc(blksize);
26         }
27         if (block == NULL){
28             MemoryContextStats(TopMemoryContext);
29             ereport(ERROR,
30                     (errcode(ERRCODE_OUT_OF_MEMORY),
31                      errmsg("out of memory"),
32                      errdetail("Failed on request of size %lu.",
33                                (unsigned long) size)));
34         }
35 
36         block->aset = set;
37         block->freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ;
38         block->endptr = ((char *) block) + blksize;
39 
40         /*
41          * If this is the first block of the set, make it the "keeper" block.
42          * Formerly, a keeper block could only be created during context
43          * creation, but allowing it to happen here lets us have fast reset
44          * cycling even for contexts created with minContextSize = 0; that way
45          * we don't have to force space to be allocated in contexts that might
46          * never need any space.  Don't mark an oversize block as a keeper,
47          * however.
48          */
49         if (set->keeper == NULL && blksize == set->initBlockSize)
50             set->keeper = block;
51 
52         block->next = set->blocks;
53         set->blocks = block;
54     }
55 
56     /*
57      * OK, do the allocation
58      */
59     chunk = (AllocChunk) (block->freeptr);
60 
61     block->freeptr += (chunk_size + ALLOC_CHUNKHDRSZ);
62     Assert(block->freeptr <= block->endptr);
63 
64     chunk->aset = (void *) set;
65     chunk->size = chunk_size;
66 #ifdef MEMORY_CONTEXT_CHECKING
67     chunk->requested_size = size;
68     /* set mark to catch clobber of "unused" space */
69     if (size < chunk->size)
70         ((char *) AllocChunkGetPointer(chunk))[size] = 0x7E;
71 #endif
72 #ifdef RANDOMIZE_ALLOCATED_MEMORY
73     /* fill the allocated space with junk */
74     randomize_mem((char *) AllocChunkGetPointer(chunk), size);
75 #endif
76     set->isReset = false;
77     AllocAllocInfo(set, chunk);
78     return AllocChunkGetPointer(chunk);
79 }

AllocSetAlloc為內存片分配空間時並不是嚴格按照申請的大小來分配的，而是將申請的大小向上對齊為2的冪，然后按照對齊后的大小來分配空間。例如，我們要申請一塊大小為30字節的空間，則AllocSetAlloc實際會為我們分配一塊大小為2^5=32字節的內存片，該內存片對應的AllocChunkData中的size字段設置為32，而requested_size設置為30.

　　AllocSetFree釋放一個內存上下文中指定的內存片，如果指定要釋放的內存片是內存塊中唯一的一個內存片，則將該內存塊直接釋放。否則，將指定的內存片加入到Freelist鏈表中以便下次分配。

 1 static void AllocSetFree(MemoryContext context, void *pointer) {
 2     AllocSet    set = (AllocSet) context;
 3     AllocChunk    chunk = AllocPointerGetChunk(pointer);
 4     AllocFreeInfo(set, chunk);
 5 #ifdef MEMORY_CONTEXT_CHECKING
 6     /* Test for someone scribbling on unused space in chunk */
 7     if (chunk->requested_size < chunk->size)
 8         if (((char *) pointer)[chunk->requested_size] != 0x7E)
 9             elog(WARNING, "detected write past chunk end in %s %p",
10                  set->header.name, chunk);
11 #endif
12     if (chunk->size > set->allocChunkLimit) {
13         /* Big chunks are certain to have been allocated as single-chunk blocks.    Find the containing block and return it to malloc(). */
14         AllocBlock    block = set->blocks;
15         AllocBlock    prevblock = NULL;
16         while (block != NULL){
17             if (chunk == (AllocChunk) (((char *) block) + ALLOC_BLOCKHDRSZ))
18                 break;
19             prevblock = block;
20             block = block->next;
21         }
22         if (block == NULL)
23             elog(ERROR, "could not find block containing chunk %p", chunk);
24         /* let's just make sure chunk is the only one in the block */
25         Assert(block->freeptr == ((char *) block) +
26                (chunk->size + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ));
27 
28         /* OK, remove block from aset's list and free it */
29         if (prevblock == NULL)
30             set->blocks = block->next;
31         else
32             prevblock->next = block->next;
33 #ifdef CLOBBER_FREED_MEMORY
34         /* Wipe freed memory for debugging purposes */
35         memset(block, 0x7F, block->freeptr - ((char *) block));
36 #endif
37         free(block);
38     } else {
39         /* Normal case, put the chunk into appropriate freelist */
40         int            fidx = AllocSetFreeIndex(chunk->size);
41         chunk->aset = (void *) set->freelist[fidx];
42 #ifdef CLOBBER_FREED_MEMORY
43         /* Wipe freed memory for debugging purposes */
44         memset(pointer, 0x7F, chunk->size);
45 #endif
46 
47 #ifdef MEMORY_CONTEXT_CHECKING
48         /* Reset requested_size to 0 in chunks that are on freelist */
49         chunk->requested_size = 0;
50 #endif
51         set->freelist[fidx] = chunk;
52     }
53 }

　　AllocSetRealloc內存重分配函數將在指定的內存上下文中對參數pointer指向的內存空間進行重新分配，新分配的內存大小由參數size指定。pointer所指向的內存中的內容將被復制到新的內存中，並釋放pointer指向的內存空間。AllocSetRealloc函數返回值就是指向新內存空間的指針。整個重分配內存流程如下：

1）由於內存片在分配之初就被對齊為2的冪，因此有可能參數pointer指向的舊的內存空間的大小本來就大於參數size指定的新的大小。如果是這樣情況，則修改pointer所指向的AllocChunkData的requested_size為新的內存大小並返回pointer

2）若pointer所指向的內存片占據一個內存塊時，則找到這個內存塊並增大這個內存塊的空間，即將該內存塊的freeptr和endptr指針都向后移動到size所指定的位置。如果pointer指向的內存片不是獨占一個內存塊則轉而指向下一步

3）調用AllocSetAlloc分配一個新內存片，並將pointer所指向內存片的數據復制到其中。然后調用AllocSetFree函數釋放舊的內存片，如果是占有一個內存塊的內存片則直接釋放（直接釋放不會造成過多內存碎片，因為該內存片占有空間較大）；否則將其加入到FreeList中以便下次分配。

  1 static void * AllocSetRealloc(MemoryContext context, void *pointer, Size size) {
  2     AllocSet    set = (AllocSet) context;
  3     AllocChunk    chunk = AllocPointerGetChunk(pointer);
  4     Size        oldsize = chunk->size;
  5 #ifdef MEMORY_CONTEXT_CHECKING
  6     /* Test for someone scribbling on unused space in chunk */
  7     if (chunk->requested_size < oldsize)
  8         if (((char *) pointer)[chunk->requested_size] != 0x7E)
  9             elog(WARNING, "detected write past chunk end in %s %p",
 10                  set->header.name, chunk);
 11 #endif
 12     /* isReset must be false already */
 13     Assert(!set->isReset);
 14     /* Chunk sizes are aligned to power of 2 in AllocSetAlloc(). Maybe the allocated area already is >= the new size.  (In particuar, we always fall out here if the requested size is a decrease.) */
 15     if (oldsize >= size) {
 16 #ifdef MEMORY_CONTEXT_CHECKING
 17 #ifdef RANDOMIZE_ALLOCATED_MEMORY
 18         /* We can only fill the extra space if we know the prior request */
 19         if (size > chunk->requested_size)
 20             randomize_mem((char *) AllocChunkGetPointer(chunk) + chunk->requested_size,
 21                           size - chunk->requested_size);
 22 #endif
 23         chunk->requested_size = size;
 24         /* set mark to catch clobber of "unused" space */
 25         if (size < oldsize)
 26             ((char *) pointer)[size] = 0x7E;
 27 #endif
 28         return pointer;
 29     }
 30 
 31     if (oldsize > set->allocChunkLimit) { 
 32         /* The chunk must have been allocated as a single-chunk block.    Find the containing block and use realloc() to make it bigger with minimum space wastage. */
 33         AllocBlock    block = set->blocks;
 34         AllocBlock    prevblock = NULL;
 35         Size        chksize;
 36         Size        blksize;
 37         while (block != NULL) { // 找到該內存片所處的內存塊
 38             if (chunk == (AllocChunk) (((char *) block) + ALLOC_BLOCKHDRSZ))
 39                 break;
 40             prevblock = block;
 41             block = block->next;
 42         }
 43         if (block == NULL)
 44             elog(ERROR, "could not find block containing chunk %p", chunk);
 45         /* let's just make sure chunk is the only one in the block */
 46         Assert(block->freeptr == ((char *) block) +
 47                (chunk->size + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ));
 48         /* Do the realloc */
 49         chksize = MAXALIGN(size);
 50         blksize = chksize + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
 51         block = (AllocBlock) realloc(block, blksize);
 52         if (block == NULL){
 53             MemoryContextStats(TopMemoryContext);
 54             ereport(ERROR,
 55                     (errcode(ERRCODE_OUT_OF_MEMORY),
 56                      errmsg("out of memory"),
 57                      errdetail("Failed on request of size %lu.",
 58                                (unsigned long) size)));
 59         }
 60         block->freeptr = block->endptr = ((char *) block) + blksize;
 61         /* Update pointers since block has likely been moved */
 62         chunk = (AllocChunk) (((char *) block) + ALLOC_BLOCKHDRSZ);
 63         if (prevblock == NULL)
 64             set->blocks = block;
 65         else
 66             prevblock->next = block;
 67         chunk->size = chksize;
 68 #ifdef MEMORY_CONTEXT_CHECKING
 69 #ifdef RANDOMIZE_ALLOCATED_MEMORY
 70         /* We can only fill the extra space if we know the prior request */
 71         randomize_mem((char *) AllocChunkGetPointer(chunk) + chunk->requested_size,
 72                       size - chunk->requested_size);
 73 #endif
 74         chunk->requested_size = size;
 75         /* set mark to catch clobber of "unused" space */
 76         if (size < chunk->size)
 77             ((char *) AllocChunkGetPointer(chunk))[size] = 0x7E;
 78 #endif
 79         return AllocChunkGetPointer(chunk);
 80     }else{
 81         /*
 82          * Small-chunk case.  We just do this by brute force, ie, allocate a
 83          * new chunk and copy the data.  Since we know the existing data isn't
 84          * huge, this won't involve any great memcpy expense, so it's not
 85          * worth being smarter.  (At one time we tried to avoid memcpy when it
 86          * was possible to enlarge the chunk in-place, but that turns out to
 87          * misbehave unpleasantly for repeated cycles of
 88          * palloc/repalloc/pfree: the eventually freed chunks go into the
 89          * wrong freelist for the next initial palloc request, and so we leak
 90          * memory indefinitely.  See pgsql-hackers archives for 2007-08-11.)
 91          */
 92         AllocPointer newPointer;
 93         /* allocate new chunk */
 94         newPointer = AllocSetAlloc((MemoryContext) set, size);
 95         /* transfer existing data (certain to fit) */
 96         memcpy(newPointer, pointer, oldsize);
 97         /* free old chunk */
 98         AllocSetFree((MemoryContext) set, pointer);
 99         return newPointer;
100     }
101 }

在PG中，內存的分配、重分配和釋放都是在內存上下文中進行，因此不能使用C語言的標准庫函數malloc、realloc和free來操作，PG實現了palloc、repalloc和pfree來分別實現內存上下文中對於內存的分配、重分配和釋放。

　　palloc是一個宏定義，它會被轉換為在當前內存上下文中對MemoryContextAlloc函數的調用，而MemoryContextAlloc函數實際上是調用了當前內存上下文的methods字段中所指定的AllocSetAlloc函數。對於目前PG實現來說，調用palloc實際就是調用了alloc指向的AllocSetAlloc函數。使用palloc分配的內存空間中的內存是隨機的，與之相對應的還定義了一個紅palloc0，后者會將分配的內存中的內容全部置0。

　　realloc是一個函數，其參數是一個內存片的指針和新的內存片大小。realloc將調用內存片所屬的內存上下文的realloc函數指針，把該內存片調整為新的大小，並返回新內存片的指針。目前，realloc函數指針對應於AllocSetRealloc函數。

　　pfree是一個函數，其參數是一個內存片指針，pfree將調用內存片所屬的內存上下文的methods字段中的free_p函數指針來釋放內存片的空間。目前，PG中free_p指針實際指向AllocSetFree函數

#define palloc(sz)    MemoryContextAlloc(CurrentMemoryContext, (sz))
#define palloc0(sz) MemoryContextAllocZero(CurrentMemoryContext, (sz))

void * repalloc(void *pointer, Size size) {
    StandardChunkHeader *header;
    /* Try to detect bogus pointers handed to us, poorly though we can. Presumably, a pointer that isn't MAXALIGNED isn't pointing at an allocated chunk. */
    Assert(pointer != NULL);
    Assert(pointer == (void *) MAXALIGN(pointer));
    /* OK, it's probably safe to look at the chunk header. */
    header = (StandardChunkHeader *)
        ((char *) pointer - STANDARDCHUNKHEADERSIZE);
    AssertArg(MemoryContextIsValid(header->context));
    if (!AllocSizeIsValid(size))
        elog(ERROR, "invalid memory alloc request size %lu",
             (unsigned long) size);
    return (*header->context->methods->realloc) (header->context,    pointer, size);
}

void pfree(void *pointer) {
    StandardChunkHeader *header;
    /* Try to detect bogus pointers handed to us, poorly though we can. Presumably, a pointer that isn't MAXALIGNED isn't pointing at an allocated chunk. */
    Assert(pointer != NULL);
    Assert(pointer == (void *) MAXALIGN(pointer));
    /* OK, it's probably safe to look at the chunk header. */
    header = (StandardChunkHeader *)
        ((char *) pointer - STANDARDCHUNKHEADERSIZE);
    AssertArg(MemoryContextIsValid(header->context));
    (*header->context->methods->free_p) (header->context, pointer);
}

免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。

猜您在找 PG內存上下文管理（MemoryContext）——內存上下文樹抽象類實現接口抽象類實現接口抽象類實現多態用抽象類來實現接口的意義實現多態的方式——抽象類 Csharp多態的實現(抽象類) 抽象類、方法的定義及實現 JavaScript實現抽象類與虛方法（六）什么是抽象類，為什么要寫抽象類