PG内存上下文管理（MemoryContext）——AllocSetContext：抽象类MemoryContext的实现

本文转载自查看原文 2020-12-03 23:52 356 PostgreSQL/ PostgreSQL源码分析

　　MemoryContext是一个抽象类，可以有对个实现，但目前只有AllocSetContext一个实现。事实上，MemoryContext并不管理实际上的内存分配，仅仅是用作对MemoryContext树的控制。管理一个内存上下文中的内存块是通过AllocSet结构来完成的，而MemoryContext仅作为AllocSet的头部信息存在，AllocSet是一个指向AllocSetContext结构的类型指针。

SET结构

header就是一个MemoryContextData结构，header是进入一个内存上下文的唯一外部接口，事实上管理内存上下文的接口函数都是通过对header的管理来实现。内存块链表blocks为一个指向AllocBlockData结构体的指针，AllocBlockData用于表示一个内存块。AllocBlockData之间通过其next字段链接成一个单向链表，而AllocSet的blocks字段则指向这个链表的头部。

 1 typedef struct AllocSetContext {
 2     MemoryContextData header; //对应于该内存上下文的头部信息
 3     AllocBlock    blocks; /*head of list of blocks in this set*/ 该内存上下文中所有内存块的链表
 4     AllocChunk freelist[ALLOCSET_NUM_FREELISTS]; /*free chunk lists*/ 该内存上下文中空闲内存片的数组
 5     bool isReset; /*T = no space alloced since last reset*/ 如果为真表示从上次重置以来没有分配过内存
 6     /* Allocation parameters for this context: */
 7     Size initBlockSize;  /*initial block size*/ 初始内存块的大小
 8     Size maxBlockSize;/*maximum block size*/ 最大内存块大小
 9     Size nextBlockSize;/*next block size to allocate*/ 下一个要分配的内存块的大小
10     Size allocChunkLimit;    /*effective chunk size limit*/ 分配内存片的尺寸阈值，该值在分配内存片时会用到
11     AllocBlock    keeper; /*if not NULL, keep this block over resets */ 保存在keeper中的内存块在内存上下文重置时会被保留不做释放
12 } AllocSetContext;

isReset：PG中提供了对内存上下文的重置操作，所谓重置就是释放内存上下文中所有分配的内存，并将这些内存交还给操作系统。在一个内存上下文被创建时，其isReset字段置为True，表示从上一次重置到当前没有内存被分配。只要在该内存上下文中进行了分配，则将其isReset字段置为False。这样在进行重置时，可以检查内存上下文的isReset字段，如果为True则表示该内存上下文中没有进行过内存分配，所以不需要进行实际的重置工作，从而提高操作效率。

initBlockSize、maxBlockSize、nextBlockSize：initBlockSize和maxBlockSize字段在内存上下文创建时指定，且在创建时nextBlockSize会置为与initBlockSize相同的值。nextBlockSize表示下一次分配的内存块的大小，在进行内存分配时，如果需要分配一个新的内存块，则这个新内存块的大小将采用nextBlockSize的值。maxBlockSize字段指定了内存块可以到达的最大尺寸。

allocChunkLimit：内存块内会分成多个称为内存片的内存单元，在分配内存片时，如果一个内存片的尺寸超过了宏ALLOC_CHUNK_LIMIT时，将会为该内存片单独分配一个独立的内存块，这样做是为了避免日后进行内存回收时造成过多的碎片。由于宏ALLOC_CHUNK_LIMIT是不能在运行时更改的，因此PG提供了allocChunkLimit用于自定义一个阈值。如果定义了这个字段的值，则在进行超限检查时会用该字段来替换宏定义进行判断。

keeper：在内存上下文进行重置时不会对keeper中记录的内存块进行释放，而是对其内容进行清空。这样可以保证内存上下文重置结束后就已经包含一定的可用内存空间，而不需要通过malloc另行申请。另外也可以避免在某个内存上下文被反复重置时，反复进行malloc带来的风险。

Block结构

由aset.c通过malloc申请的内存单元（内存块），它包含了多个由palloc申请的内存片(AllocChunk)。内存片由pfree函数释放到freelist中，由下一次palloc调用重新使用。AllocBlockData记录在一块内存区域的起始地址处，这块内存区域通过标准库函数malloc进行分配，称为一个内存块。在每个内存块中进行内存分配时产生的内存片段称之为内存片，freeptr指向该块空闲区域的首地址，endptr指向该内存块的末地址。

1 typedef struct AllocBlockData {
2     AllocSet    aset;     /* aset that owns this block */ 该内存块所在AllocSet
3     AllocBlock    next; /* next block in aset's blocks list */ 指向下一个内存块的指针
4     char       *freeptr; /* start of free space in this block */ 指向该块空闲区域的首地址
5     char       *endptr;     /* end of space in this block */ 该内存块的末地址
6 } AllocBlockData;
typedef AllocSetContext *AllocSet;

Chunk结构

每个内存片包括一个头部信息和数据区域，其中头部信息包括该内存片所属的内存上下文以及该内存区的其他相关信息，数据区则存储实际数据。内存片的头部信息由数据结构AllocChunkData描述，内存片的数据区域则紧跟在其头部信息之后分配。通过PG中定义的palloc函数和pfree函数，可以自由地在内存上下文中申请和释放内存片，被释放的内存片将被加入到FreeList中以被重复使用。

 1 typedef struct AllocChunkData {
 2     /* aset is the owning aset if allocated, or the freelist link if free */
 3     void       *aset;
 4     /* size is always the size of the usable space in the chunk */
 5     Size        size;
 6 #ifdef MEMORY_CONTEXT_CHECKING
 7     /* when debugging memory usage, also store actual requested size */
 8     /* this is zero in a free chunk */
 9     Size        requested_size;
10 #endif
11 } AllocChunkData;
typedef struct AllocChunkData *AllocChunk;

FreeList结构

FreeList数组用于维护在内存块中被回收的空闲内存片，这些空闲内存片将被用于再分配。FreeList数组元素类型为AllocChunk，数组长度默认为11（由宏ALLOCSET_NUM_FREELIST定义）。FreeList数组中的每个元素指向一个由特定大小空闲内存片组成的链表，这个大小与该元素在数组中的顺序有关。比如，FreeList数组中第K个元素所指向链表的每个空闲数据块的大小为2^(k+2)字节，空闲内存片最小为8字节，最大不超过8K字节。因此，FreeList数组中实际上维护了11个空闲链表。通过aset字段来链接空闲链表，如果一个内存正在使用，则它的aset字段指向其所属的AllocSet。如果内存片是空闲的，也就是说它处于某个空闲链表中，那么它的aset字段指向空闲链表中在它之后的内存片。这样从FreeList数组元素所指向的链表头部开始，顺着aset字段指向的下一个内存片就可以找到该空闲链表中所有的空闲内存片。

当需要申请一块内存时，我们可以迅速地定位到相应的空闲链表中。对于一个大小为size的内存分配请求，将会在第K个空闲链表中为其分配内存片，K的计算规则是：

当size<(1<<ALLOC_MINBITS)时，K为0，其中ALLOC_MINBITS系统预定为3，即表是FreeList中保存的最小字节数位2^3=8字节
当(1<<ALLOC_MINBITS)<size<ALLOC_CHUNK_LIMIT时，若2^(N-1)<size<2^N，则K=N-3，其中ALLOC_MINBITS为FreeList中所维持空闲内存片的最大值

 1 static inline int AllocSetFreeIndex(Size size) {
 2     int            idx = 0;
 3     if (size > 0) {
 4         size = (size - 1) >> ALLOC_MINBITS;
 5         while (size != 0) {
 6             idx++;
 7             size >>= 1;
 8         }
 9         Assert(idx < ALLOCSET_NUM_FREELISTS);
10     }
11     return idx;
12 }

在创建AllocSet时，可以为其指定allocChunkLimit字段的值，如果申请分配的内存大小超过这个值，那么将为这次请求分配一个独立的内存块，这个块中只存放一个内存片，当该内存片释放的时候，整个内存块将被释放，而不是将内存片加到FreeList中。一个内存片的大小不会超过ALLOC_CHUNK_LIMIT，否则将会被当作一个独立的块来看待。

API

MemoryContext API for AllocSet contexts 用于初始化AllocSetContext中MemoryContext的内存处理函数指针

 1 static MemoryContextMethods AllocSetMethods = {
 2     AllocSetAlloc,
 3     AllocSetFree,   // 释放一个内存上下文中指定的内存片
 4     AllocSetRealloc,
 5     AllocSetInit,
 6     AllocSetReset,  // 重置内存上下文
 7     AllocSetDelete, // 释放当前内存上下文中全部内存块
 8     AllocSetGetChunkSpace,
 9     AllocSetIsEmpty,
10     AllocSetStats
11 #ifdef MEMORY_CONTEXT_CHECKING
12     ,AllocSetCheck
13 #endif
14 };
15 static void *AllocSetAlloc(MemoryContext context, Size size);
16 static void AllocSetFree(MemoryContext context, void *pointer);
17 static void *AllocSetRealloc(MemoryContext context, void *pointer, Size size);
18 static void AllocSetInit(MemoryContext context);
19 static void AllocSetReset(MemoryContext context);
20 static void AllocSetDelete(MemoryContext context);
21 static Size AllocSetGetChunkSpace(MemoryContext context, void *pointer);
22 static bool AllocSetIsEmpty(MemoryContext context);
23 static void AllocSetStats(MemoryContext context, int level);
24 #ifdef MEMORY_CONTEXT_CHECKING
25 static void AllocSetCheck(MemoryContext context);
26 #endif

　　AllocSetInit函数在建立generic内存上下文，将新内存上下文链接到上下文树之前由MemoryContexCreate函数调用。

1 static void AllocSetInit(MemoryContext context) {
2     /* Since MemoryContextCreate already zeroed the context node, we don't have to do anything here: it's already OK. */
3 }

　　AllocSetReset函数释放在给定的set中分配的内存，在进行重置时，内存上下文中除了在keeper字段中指定要保留的内存块外，其他内存块全部释放，包括空闲链表中的内存。keeper中指定保留的内存块被清空内容，它使得内存上下文重置之后就立刻有一块内存可供使用。

 1 static void AllocSetReset(MemoryContext context) {
 2     AllocSet    set = (AllocSet) context;
 3     AllocBlock    block;
 4     AssertArg(AllocSetIsValid(set));
 5     /* Nothing to do if no pallocs since startup or last reset */
 6     if (set->isReset)
 7         return;
 8 #ifdef MEMORY_CONTEXT_CHECKING
 9     /* Check for corruption and leaks before freeing */
10     AllocSetCheck(context);
11 #endif
12     /* Clear chunk freelists */
13     MemSetAligned(set->freelist, 0, sizeof(set->freelist));
14     block = set->blocks;
15     /* New blocks list is either empty or just the keeper block */
16     set->blocks = set->keeper;
17     while (block != NULL) {
18         AllocBlock    next = block->next;
19         if (block == set->keeper) {
20             /* Reset the block, but don't return it to malloc */
21             char       *datastart = ((char *) block) + ALLOC_BLOCKHDRSZ;
22 #ifdef CLOBBER_FREED_MEMORY
23             /* Wipe freed memory for debugging purposes */
24             memset(datastart, 0x7F, block->freeptr - datastart);
25 #endif
26             block->freeptr = datastart;
27             block->next = NULL;
28         } else {
29             /* Normal case, release the block */
30 #ifdef CLOBBER_FREED_MEMORY
31             /* Wipe freed memory for debugging purposes */
32             memset(block, 0x7F, block->freeptr - ((char *) block));
33 #endif
34             free(block);
35         }
36         block = next;
37     }
38     /* Reset block size allocation sequence, too */
39     set->nextBlockSize = set->initBlockSize;
40     set->isReset = true;
41 }

　　AllocSetDelete函数释放当前内存上下文中的全部内存块，包括keeper指定的内存块在内。但内存上下文节点并不释放，因为内存上下文是在TopMemoryContext中申请的内存，将在进程运行结束时统一释放。

 1 static void AllocSetDelete(MemoryContext context) {
 2     AllocSet    set = (AllocSet) context;
 3     AllocBlock    block = set->blocks;
 4     AssertArg(AllocSetIsValid(set));
 5 #ifdef MEMORY_CONTEXT_CHECKING
 6     /* Check for corruption and leaks before freeing */
 7     AllocSetCheck(context);
 8 #endif
 9     /* Make it look empty, just in case... */
10     MemSetAligned(set->freelist, 0, sizeof(set->freelist));
11     set->blocks = NULL;
12     set->keeper = NULL;
13     while (block != NULL) {
14         AllocBlock    next = block->next;
15 #ifdef CLOBBER_FREED_MEMORY
16         /* Wipe freed memory for debugging purposes */
17         memset(block, 0x7F, block->freeptr - ((char *) block));
18 #endif
19         free(block);
20         block = next;
21     }
22 }

　　AllocSetContextCreate创建新的AllocSet上下文，参数parent是父内存上下文节点，或者为Null为top-level内存上下文，minContextSize内存上下文最小大小，initBlockSize初始分配块大小，maxBlockSize最大分配块大小。内存上下文的创建由AllocSetContextCreate函数完成，主要有两个工作：创建内存上下文节点以及分配内存块。内存上下文节点的创建由MemoryContextCreate函数来完成，主要流程如下：从TopMemoryContext节点中分配一块内存用于存放内存上下文节点（该内存上下文管理的内存块不包括在内），该内存块略大于一个AllocSet结构体的大小；初始化MemoryContext节点，设置其父节点、节点类型、节点名等相关信息，还要设置该节点的methods属性为AllocSetMethods。

 1 MemoryContext AllocSetContextCreate(MemoryContext parent,
 2                       const char *name,
 3                       Size minContextSize,
 4                       Size initBlockSize,
 5                       Size maxBlockSize) {
 6     AllocSet    context;
 7     /* Do the type-independent part of context creation */
 8     context = (AllocSet) MemoryContextCreate(T_AllocSetContext,sizeof(AllocSetContext),&AllocSetMethods,parent,name);
 9 
10     /* Make sure alloc parameters are reasonable, and save them. We somewhat arbitrarily enforce a minimum 1K block size. */
11     initBlockSize = MAXALIGN(initBlockSize);
12     if (initBlockSize < 1024)
13         initBlockSize = 1024;
14     maxBlockSize = MAXALIGN(maxBlockSize);
15     if (maxBlockSize < initBlockSize)
16         maxBlockSize = initBlockSize;
17     context->initBlockSize = initBlockSize;
18     context->maxBlockSize = maxBlockSize;
19     context->nextBlockSize = initBlockSize;
20 
21     /* Compute the allocation chunk size limit for this context.  It can't be more than ALLOC_CHUNK_LIMIT because of the fixed number of freelists. If maxBlockSize is small then requests exceeding the maxBlockSize should be treated as large chunks, too.    We have to have allocChunkLimit a power of two, because the requested and actually-allocated sizes of any chunk must be on the same side of the limit, else we get confused about whether the chunk is "big".*/
22     context->allocChunkLimit = ALLOC_CHUNK_LIMIT;
23     while (context->allocChunkLimit >
24            (Size) (maxBlockSize - ALLOC_BLOCKHDRSZ - ALLOC_CHUNKHDRSZ))
25         context->allocChunkLimit >>= 1;
26 
27     /* Grab always-allocated space, if requested */
28     if (minContextSize > ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ){
29         Size        blksize = MAXALIGN(minContextSize);
30         AllocBlock    block;
31         block = (AllocBlock) malloc(blksize);
32         if (block == NULL){
33             MemoryContextStats(TopMemoryContext);
34             ereport(ERROR,
35                 (errcode(ERRCODE_OUT_OF_MEMORY),
36                      errmsg("out of memory"),
37                      errdetail("Failed while creating memory context \"%s\".",
38                                name)));
39         }
40         block->aset = context;
41         block->freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ;
42         block->endptr = ((char *) block) + blksize;
43         block->next = context->blocks;
44         context->blocks = block;
45         /* Mark block as not to be released at reset time */
46         context->keeper = block;
47     }
48     context->isReset = true;
49     return (MemoryContext) context;
50 }

AllocSetContextCreate函数的处理流程如下：

1）调用MemoryContextCreate创建MemoryContext结构

2）将上一步创建的MemoryContext结构强制转换为AllocSet结构体，填充AllocSet其他结构体元素的信息，包括最小块大小、初始化块大小以及最大块大小，并根据最大块大小设置其allocChunkLimit的值

3）如果minContextSize超过一定限制（内存块block头部信息尺寸和内存片chunk头部信息尺寸之和）时，调用标准库函数，以minContextSize为大小分配一个内存块，并初始化块结构体，加入AllocSet的内存块链表中。需要注意的是，预分配的内存块将被记入到内存上下文的keeper字段中，作为内存上下文的保留块，以便重置内存上下文的时候，该内存块不会被释放。

　　AllocSetAlloc函数负责处理具体内存分配工作，该函数的参数为一个内存上下文节点以及需要申请的内存大小。具体的内存分配流程如下所示：判断需要申请的内存大小是否超过当前内存上下文允许分配内存片的最大值。若超过则为其分配一个新的独立的内存块，然后在该内存块中分配指定大小的内存片。接下来将该内存块加入到内存链表中，最后设置内存上下文的isReset字段为False并返回内存片的指针。

 1 static void * AllocSetAlloc(MemoryContext context, Size size) {
 2     AllocSet    set = (AllocSet) context;
 3     AllocBlock    block;
 4     AllocChunk chunk;
 5     int    fidx;
 6     Size    chunk_size;
 7     Size    blksize;
 8     AssertArg(AllocSetIsValid(set));
 9     /* If requested size exceeds maximum for chunks, allocate an entire block for this request. */
10     if (size > set->allocChunkLimit) {
11         chunk_size = MAXALIGN(size);
12         blksize = chunk_size + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
13         block = (AllocBlock) malloc(blksize);
14         if (block == NULL) {
15             MemoryContextStats(TopMemoryContext);
16             ereport(ERROR(errcode(ERRCODE_OUT_OF_MEMORY),
17                      errmsg("out of memory"),
18                      errdetail("Failed on request of size %lu.",
19                                (unsigned long) size)));
20         }
21         block->aset = set;
22         block->freeptr = block->endptr = ((char *) block) + blksize;
23         chunk = (AllocChunk) (((char *) block) + ALLOC_BLOCKHDRSZ);
24         chunk->aset = set;
25         chunk->size = chunk_size;
26 #ifdef MEMORY_CONTEXT_CHECKING
27         chunk->requested_size = size;
28         /* set mark to catch clobber of "unused" space */
29         if (size < chunk_size)
30             ((char *) AllocChunkGetPointer(chunk))[size] = 0x7E;
31 #endif
32 #ifdef RANDOMIZE_ALLOCATED_MEMORY
33         /* fill the allocated space with junk */
34         randomize_mem((char *) AllocChunkGetPointer(chunk), size);
35 #endif
36         /* Stick the new block underneath the active allocation block, so that we don't lose the use of the space remaining therein. */
37         if (set->blocks != NULL) { // 新申请的block进行头插
38             block->next = set->blocks->next; 
39             set->blocks->next = block;
40         } else {
41             block->next = NULL;
42             set->blocks = block;
43         }
44         set->isReset = false;
45         AllocAllocInfo(set, chunk);
46         return AllocChunkGetPointer(chunk);
47     }

计算申请的内存大小在FreeList数组中对应的位置，如果存在合适的空闲内存片，则将空闲链表的指针（Freelist数组的某个元素）指向该内存片的aset字段所指向的地址（在空闲内存片中，aset字段指向它在空闲链表中的下一个内存片）。然后将该内存片的aset字段指向其所属的内存上下文节点，最后返回该内存片的指针。

 1     fidx = AllocSetFreeIndex(size);
 2     chunk = set->freelist[fidx];
 3     if (chunk != NULL){
 4         Assert(chunk->size >= size);
 5         set->freelist[fidx] = (AllocChunk) chunk->aset;
 6         chunk->aset = (void *) set;
 7 #ifdef MEMORY_CONTEXT_CHECKING
 8         chunk->requested_size = size;
 9         /* set mark to catch clobber of "unused" space */
10         if (size < chunk->size)
11             ((char *) AllocChunkGetPointer(chunk))[size] = 0x7E;
12 #endif
13 #ifdef RANDOMIZE_ALLOCATED_MEMORY
14         /* fill the allocated space with junk */
15         randomize_mem((char *) AllocChunkGetPointer(chunk), size);
16 #endif
17         /* isReset must be false already */
18         Assert(!set->isReset);
19         AllocAllocInfo(set, chunk);
20         return AllocChunkGetPointer(chunk);
21     }

如果空闲链表中没有满足要求的内存片，则对内存上下文的内存块链表（blocks字段）的第一个内存块进行检查，如果该内存块中的未分配空间足以满足申请的内存，则直接在该内存块中分配内存片并返回内存片的指针。这里可以看到，在内存上下文中进行内存分配时，总是在内存块链表中的第一个内存块中进行，当该内存块中空间用完之后会分配新的内存块并作为新的内存块链表首部，因此内存块链表中的第一块也称作活动内存块。

 1     chunk_size = (1 << ALLOC_MINBITS) << fidx;
 2     Assert(chunk_size >= size);
 3     /* If there is enough room in the active allocation block, we will put the chunk into that block.  Else must start a new one. */
 4     if ((block = set->blocks) != NULL) {
 5         Size    availspace = block->endptr - block->freeptr;
 6         if (availspace < (chunk_size + ALLOC_CHUNKHDRSZ)){
 7             while (availspace >= ((1 << ALLOC_MINBITS) + ALLOC_CHUNKHDRSZ)) {
 8                 Size    availchunk = availspace - ALLOC_CHUNKHDRSZ;
 9                 int    a_fidx = AllocSetFreeIndex(availchunk);
10                 if (availchunk != (1 << (a_fidx + ALLOC_MINBITS))){
11                     a_fidx--;
12                     Assert(a_fidx >= 0);
13                     availchunk = (1 << (a_fidx + ALLOC_MINBITS));
14                 }
15                 chunk = (AllocChunk) (block->freeptr);
16                 block->freeptr += (availchunk + ALLOC_CHUNKHDRSZ);
17                 availspace -= (availchunk + ALLOC_CHUNKHDRSZ);
18                 chunk->size = availchunk;
19 #ifdef MEMORY_CONTEXT_CHECKING
20                 chunk->requested_size = 0;        /* mark it free */
21 #endif
22                 chunk->aset = (void *) set->freelist[a_fidx];
23                 set->freelist[a_fidx] = chunk;
24             }
25             /* Mark that we need to create a new block */
26             block = NULL;  // 现有的内存块都不能满足这一次内存分配的要求
27         }
28     }

while (availspace >= ((1 << ALLOC_MINBITS) + ALLOC_CHUNKHDRSZ))的逻辑是：当前active(top)块没有足够的内存用于分配，但是其中可能仍然有足够的空间。一旦我们将它放入block list中，就不会尝试从其中分配空间。按照空闲内存片链表所需的大小规则将空闲空间分片，并将它们放入freelist。

如果内存块链表中的第一个内存块没有足够的未分配空间，由于现有的内存块都不能满足这一次内存分配的要求，因此需要申请新的内存块，但是当前的活动内存块中还有未分配空间，如果申请新的内存块并将之作为新的活动内存块，则当前活动内存块中的未分配空间就会被浪费。为了避免浪费，这里会先将当前活动内存块中的未分配空间分解成个数尽可能少的内存片（即每个内存片尽可能大），并将它们加入到FreeList数组中，然后创建一个新的内存块（其大小为前一次分配的内存块的两倍，但不超过maxBlockSize）并将之作为新的活动内存块（即加入到内存块链表首部）。最后再活动内存块中分配一个满足申请内存大小的内存片，并返回其指针。

 1     if (block == NULL) {
 2         Size    required_size;
 3         /* The first such block has size initBlockSize, and we double the space in each succeeding block, but not more than maxBlockSize. */
 4         blksize = set->nextBlockSize;
 5         set->nextBlockSize <<= 1;
 6         if (set->nextBlockSize > set->maxBlockSize)
 7             set->nextBlockSize = set->maxBlockSize;
 8 
 9         /* If initBlockSize is less than ALLOC_CHUNK_LIMIT, we could need more space... but try to keep it a power of 2. */
10         required_size = chunk_size + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
11         while (blksize < required_size)
12             blksize <<= 1;
13 
14         /* Try to allocate it */
15         block = (AllocBlock) malloc(blksize);
16 
17         /*
18          * We could be asking for pretty big blocks here, so cope if malloc
19          * fails.  But give up if there's less than a meg or so available...
20          */
21         while (block == NULL && blksize > 1024 * 1024) {
22             blksize >>= 1;
23             if (blksize < required_size)
24                 break;
25             block = (AllocBlock) malloc(blksize);
26         }
27         if (block == NULL){
28             MemoryContextStats(TopMemoryContext);
29             ereport(ERROR,
30                     (errcode(ERRCODE_OUT_OF_MEMORY),
31                      errmsg("out of memory"),
32                      errdetail("Failed on request of size %lu.",
33                                (unsigned long) size)));
34         }
35 
36         block->aset = set;
37         block->freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ;
38         block->endptr = ((char *) block) + blksize;
39 
40         /*
41          * If this is the first block of the set, make it the "keeper" block.
42          * Formerly, a keeper block could only be created during context
43          * creation, but allowing it to happen here lets us have fast reset
44          * cycling even for contexts created with minContextSize = 0; that way
45          * we don't have to force space to be allocated in contexts that might
46          * never need any space.  Don't mark an oversize block as a keeper,
47          * however.
48          */
49         if (set->keeper == NULL && blksize == set->initBlockSize)
50             set->keeper = block;
51 
52         block->next = set->blocks;
53         set->blocks = block;
54     }
55 
56     /*
57      * OK, do the allocation
58      */
59     chunk = (AllocChunk) (block->freeptr);
60 
61     block->freeptr += (chunk_size + ALLOC_CHUNKHDRSZ);
62     Assert(block->freeptr <= block->endptr);
63 
64     chunk->aset = (void *) set;
65     chunk->size = chunk_size;
66 #ifdef MEMORY_CONTEXT_CHECKING
67     chunk->requested_size = size;
68     /* set mark to catch clobber of "unused" space */
69     if (size < chunk->size)
70         ((char *) AllocChunkGetPointer(chunk))[size] = 0x7E;
71 #endif
72 #ifdef RANDOMIZE_ALLOCATED_MEMORY
73     /* fill the allocated space with junk */
74     randomize_mem((char *) AllocChunkGetPointer(chunk), size);
75 #endif
76     set->isReset = false;
77     AllocAllocInfo(set, chunk);
78     return AllocChunkGetPointer(chunk);
79 }

AllocSetAlloc为内存片分配空间时并不是严格按照申请的大小来分配的，而是将申请的大小向上对齐为2的幂，然后按照对齐后的大小来分配空间。例如，我们要申请一块大小为30字节的空间，则AllocSetAlloc实际会为我们分配一块大小为2^5=32字节的内存片，该内存片对应的AllocChunkData中的size字段设置为32，而requested_size设置为30.

　　AllocSetFree释放一个内存上下文中指定的内存片，如果指定要释放的内存片是内存块中唯一的一个内存片，则将该内存块直接释放。否则，将指定的内存片加入到Freelist链表中以便下次分配。

 1 static void AllocSetFree(MemoryContext context, void *pointer) {
 2     AllocSet    set = (AllocSet) context;
 3     AllocChunk    chunk = AllocPointerGetChunk(pointer);
 4     AllocFreeInfo(set, chunk);
 5 #ifdef MEMORY_CONTEXT_CHECKING
 6     /* Test for someone scribbling on unused space in chunk */
 7     if (chunk->requested_size < chunk->size)
 8         if (((char *) pointer)[chunk->requested_size] != 0x7E)
 9             elog(WARNING, "detected write past chunk end in %s %p",
10                  set->header.name, chunk);
11 #endif
12     if (chunk->size > set->allocChunkLimit) {
13         /* Big chunks are certain to have been allocated as single-chunk blocks.    Find the containing block and return it to malloc(). */
14         AllocBlock    block = set->blocks;
15         AllocBlock    prevblock = NULL;
16         while (block != NULL){
17             if (chunk == (AllocChunk) (((char *) block) + ALLOC_BLOCKHDRSZ))
18                 break;
19             prevblock = block;
20             block = block->next;
21         }
22         if (block == NULL)
23             elog(ERROR, "could not find block containing chunk %p", chunk);
24         /* let's just make sure chunk is the only one in the block */
25         Assert(block->freeptr == ((char *) block) +
26                (chunk->size + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ));
27 
28         /* OK, remove block from aset's list and free it */
29         if (prevblock == NULL)
30             set->blocks = block->next;
31         else
32             prevblock->next = block->next;
33 #ifdef CLOBBER_FREED_MEMORY
34         /* Wipe freed memory for debugging purposes */
35         memset(block, 0x7F, block->freeptr - ((char *) block));
36 #endif
37         free(block);
38     } else {
39         /* Normal case, put the chunk into appropriate freelist */
40         int            fidx = AllocSetFreeIndex(chunk->size);
41         chunk->aset = (void *) set->freelist[fidx];
42 #ifdef CLOBBER_FREED_MEMORY
43         /* Wipe freed memory for debugging purposes */
44         memset(pointer, 0x7F, chunk->size);
45 #endif
46 
47 #ifdef MEMORY_CONTEXT_CHECKING
48         /* Reset requested_size to 0 in chunks that are on freelist */
49         chunk->requested_size = 0;
50 #endif
51         set->freelist[fidx] = chunk;
52     }
53 }

　　AllocSetRealloc内存重分配函数将在指定的内存上下文中对参数pointer指向的内存空间进行重新分配，新分配的内存大小由参数size指定。pointer所指向的内存中的内容将被复制到新的内存中，并释放pointer指向的内存空间。AllocSetRealloc函数返回值就是指向新内存空间的指针。整个重分配内存流程如下：

1）由于内存片在分配之初就被对齐为2的幂，因此有可能参数pointer指向的旧的内存空间的大小本来就大于参数size指定的新的大小。如果是这样情况，则修改pointer所指向的AllocChunkData的requested_size为新的内存大小并返回pointer

2）若pointer所指向的内存片占据一个内存块时，则找到这个内存块并增大这个内存块的空间，即将该内存块的freeptr和endptr指针都向后移动到size所指定的位置。如果pointer指向的内存片不是独占一个内存块则转而指向下一步

3）调用AllocSetAlloc分配一个新内存片，并将pointer所指向内存片的数据复制到其中。然后调用AllocSetFree函数释放旧的内存片，如果是占有一个内存块的内存片则直接释放（直接释放不会造成过多内存碎片，因为该内存片占有空间较大）；否则将其加入到FreeList中以便下次分配。

  1 static void * AllocSetRealloc(MemoryContext context, void *pointer, Size size) {
  2     AllocSet    set = (AllocSet) context;
  3     AllocChunk    chunk = AllocPointerGetChunk(pointer);
  4     Size        oldsize = chunk->size;
  5 #ifdef MEMORY_CONTEXT_CHECKING
  6     /* Test for someone scribbling on unused space in chunk */
  7     if (chunk->requested_size < oldsize)
  8         if (((char *) pointer)[chunk->requested_size] != 0x7E)
  9             elog(WARNING, "detected write past chunk end in %s %p",
 10                  set->header.name, chunk);
 11 #endif
 12     /* isReset must be false already */
 13     Assert(!set->isReset);
 14     /* Chunk sizes are aligned to power of 2 in AllocSetAlloc(). Maybe the allocated area already is >= the new size.  (In particuar, we always fall out here if the requested size is a decrease.) */
 15     if (oldsize >= size) {
 16 #ifdef MEMORY_CONTEXT_CHECKING
 17 #ifdef RANDOMIZE_ALLOCATED_MEMORY
 18         /* We can only fill the extra space if we know the prior request */
 19         if (size > chunk->requested_size)
 20             randomize_mem((char *) AllocChunkGetPointer(chunk) + chunk->requested_size,
 21                           size - chunk->requested_size);
 22 #endif
 23         chunk->requested_size = size;
 24         /* set mark to catch clobber of "unused" space */
 25         if (size < oldsize)
 26             ((char *) pointer)[size] = 0x7E;
 27 #endif
 28         return pointer;
 29     }
 30 
 31     if (oldsize > set->allocChunkLimit) { 
 32         /* The chunk must have been allocated as a single-chunk block.    Find the containing block and use realloc() to make it bigger with minimum space wastage. */
 33         AllocBlock    block = set->blocks;
 34         AllocBlock    prevblock = NULL;
 35         Size        chksize;
 36         Size        blksize;
 37         while (block != NULL) { // 找到该内存片所处的内存块
 38             if (chunk == (AllocChunk) (((char *) block) + ALLOC_BLOCKHDRSZ))
 39                 break;
 40             prevblock = block;
 41             block = block->next;
 42         }
 43         if (block == NULL)
 44             elog(ERROR, "could not find block containing chunk %p", chunk);
 45         /* let's just make sure chunk is the only one in the block */
 46         Assert(block->freeptr == ((char *) block) +
 47                (chunk->size + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ));
 48         /* Do the realloc */
 49         chksize = MAXALIGN(size);
 50         blksize = chksize + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
 51         block = (AllocBlock) realloc(block, blksize);
 52         if (block == NULL){
 53             MemoryContextStats(TopMemoryContext);
 54             ereport(ERROR,
 55                     (errcode(ERRCODE_OUT_OF_MEMORY),
 56                      errmsg("out of memory"),
 57                      errdetail("Failed on request of size %lu.",
 58                                (unsigned long) size)));
 59         }
 60         block->freeptr = block->endptr = ((char *) block) + blksize;
 61         /* Update pointers since block has likely been moved */
 62         chunk = (AllocChunk) (((char *) block) + ALLOC_BLOCKHDRSZ);
 63         if (prevblock == NULL)
 64             set->blocks = block;
 65         else
 66             prevblock->next = block;
 67         chunk->size = chksize;
 68 #ifdef MEMORY_CONTEXT_CHECKING
 69 #ifdef RANDOMIZE_ALLOCATED_MEMORY
 70         /* We can only fill the extra space if we know the prior request */
 71         randomize_mem((char *) AllocChunkGetPointer(chunk) + chunk->requested_size,
 72                       size - chunk->requested_size);
 73 #endif
 74         chunk->requested_size = size;
 75         /* set mark to catch clobber of "unused" space */
 76         if (size < chunk->size)
 77             ((char *) AllocChunkGetPointer(chunk))[size] = 0x7E;
 78 #endif
 79         return AllocChunkGetPointer(chunk);
 80     }else{
 81         /*
 82          * Small-chunk case.  We just do this by brute force, ie, allocate a
 83          * new chunk and copy the data.  Since we know the existing data isn't
 84          * huge, this won't involve any great memcpy expense, so it's not
 85          * worth being smarter.  (At one time we tried to avoid memcpy when it
 86          * was possible to enlarge the chunk in-place, but that turns out to
 87          * misbehave unpleasantly for repeated cycles of
 88          * palloc/repalloc/pfree: the eventually freed chunks go into the
 89          * wrong freelist for the next initial palloc request, and so we leak
 90          * memory indefinitely.  See pgsql-hackers archives for 2007-08-11.)
 91          */
 92         AllocPointer newPointer;
 93         /* allocate new chunk */
 94         newPointer = AllocSetAlloc((MemoryContext) set, size);
 95         /* transfer existing data (certain to fit) */
 96         memcpy(newPointer, pointer, oldsize);
 97         /* free old chunk */
 98         AllocSetFree((MemoryContext) set, pointer);
 99         return newPointer;
100     }
101 }

在PG中，内存的分配、重分配和释放都是在内存上下文中进行，因此不能使用C语言的标准库函数malloc、realloc和free来操作，PG实现了palloc、repalloc和pfree来分别实现内存上下文中对于内存的分配、重分配和释放。

　　palloc是一个宏定义，它会被转换为在当前内存上下文中对MemoryContextAlloc函数的调用，而MemoryContextAlloc函数实际上是调用了当前内存上下文的methods字段中所指定的AllocSetAlloc函数。对于目前PG实现来说，调用palloc实际就是调用了alloc指向的AllocSetAlloc函数。使用palloc分配的内存空间中的内存是随机的，与之相对应的还定义了一个红palloc0，后者会将分配的内存中的内容全部置0。

　　realloc是一个函数，其参数是一个内存片的指针和新的内存片大小。realloc将调用内存片所属的内存上下文的realloc函数指针，把该内存片调整为新的大小，并返回新内存片的指针。目前，realloc函数指针对应于AllocSetRealloc函数。

　　pfree是一个函数，其参数是一个内存片指针，pfree将调用内存片所属的内存上下文的methods字段中的free_p函数指针来释放内存片的空间。目前，PG中free_p指针实际指向AllocSetFree函数

#define palloc(sz)    MemoryContextAlloc(CurrentMemoryContext, (sz))
#define palloc0(sz) MemoryContextAllocZero(CurrentMemoryContext, (sz))

void * repalloc(void *pointer, Size size) {
    StandardChunkHeader *header;
    /* Try to detect bogus pointers handed to us, poorly though we can. Presumably, a pointer that isn't MAXALIGNED isn't pointing at an allocated chunk. */
    Assert(pointer != NULL);
    Assert(pointer == (void *) MAXALIGN(pointer));
    /* OK, it's probably safe to look at the chunk header. */
    header = (StandardChunkHeader *)
        ((char *) pointer - STANDARDCHUNKHEADERSIZE);
    AssertArg(MemoryContextIsValid(header->context));
    if (!AllocSizeIsValid(size))
        elog(ERROR, "invalid memory alloc request size %lu",
             (unsigned long) size);
    return (*header->context->methods->realloc) (header->context,    pointer, size);
}

void pfree(void *pointer) {
    StandardChunkHeader *header;
    /* Try to detect bogus pointers handed to us, poorly though we can. Presumably, a pointer that isn't MAXALIGNED isn't pointing at an allocated chunk. */
    Assert(pointer != NULL);
    Assert(pointer == (void *) MAXALIGN(pointer));
    /* OK, it's probably safe to look at the chunk header. */
    header = (StandardChunkHeader *)
        ((char *) pointer - STANDARDCHUNKHEADERSIZE);
    AssertArg(MemoryContextIsValid(header->context));
    (*header->context->methods->free_p) (header->context, pointer);
}

免责声明！

本站转载的文章为个人学习借鉴使用，本站对版权不负任何法律责任。如果侵犯了您的隐私权益，请联系本站邮箱yoyou2525@163.com删除。

猜您在找 PG内存上下文管理（MemoryContext）——内存上下文树抽象类实现接口抽象类实现接口抽象类实现多态用抽象类来实现接口的意义实现多态的方式——抽象类 Csharp多态的实现(抽象类) 抽象类、方法的定义及实现 JavaScript实现抽象类与虚方法（六）什么是抽象类，为什么要写抽象类