擴容Linux文件系統
騰訊雲 雲硬盤擴容
https://cloud.tencent.com/product/cbs
https://cloud.tencent.com/document/product/362/6738
普通雲硬盤（HDD Cloud Storage） 容量最大為16TB
高性能雲硬盤（Premium Cloud Storage） 容量最大為4TB
SSD雲硬盤（SSD Cloud Storage） 容量最大為4TB
單台虛擬機最多可掛載 10 塊雲盤，容量達 40TB。您可以輕松搭建大容量的文件系統，用於大數據、數據倉庫、日志處理等業務。
請注意，由於MBR的限制，選擇任何一種方式時，請保持任意分區的大小不超過2TB（若您擴容后的空間已經大於2TB則不可選擇第二種方式。
一般系統分區方案
/boot 500M
/ 30G
SWAP 8G
/data 剩下空間
擴容分三步
1) 擴容實體雲硬盤大小
2) 擴容分區
確定文件系統分區表形式
擴容分區
3) 擴容文件系統
GPT分區雲硬盤擴容
umount 掛載點
parted [磁盤路徑]
e2fsck -f /dev/vdb1
resize2fs /dev/vdb1
mount 分區路徑 掛載點
# 具體命令 umount -lf /data parted /dev/vdb e2fsck -f /dev/vdb1 resize2fs /dev/vdb1 mount /dev/vdb1 /data 或 echo "/dev/vdb /data ext4 defaults,noatime,nodiratime 0 0" >>/etc/fstab mount -a
新空間增加到已有分區中(GPT分區格式)
查看數據盤信息
執行命令
parted 磁盤路徑 print
確認雲硬盤的容量變化。如在過程中收到如下提示,請輸入Fix：
這里擴容后的雲硬盤大小為107GB，已有分區的大小為10.7GB。
卸載已掛載數據盤
執行以下命令確認該雲硬盤是否還有分區已掛載：
mount | grep '磁盤路徑'
這里雲硬盤上有一個分區(vdb1)掛載在/data上，需要將其解掛。
使用以下命令解掛：
umount 掛載點
本例中即執行umount /data進行卸載。
注：要將雲硬盤上所有分區的文件系統都解掛，如vdb1、vdb2......
再次使用mount | grep '/dev/vdb'命令來確認此硬盤上所有分區的文件系統都已解掛。
數據盤分區
確認雲硬盤所有分區均已卸載后，執行以下命令，將原分區刪除並以同樣的起始偏移新建一個分區：
parted [磁盤路徑]
接下來輸入unit s，將顯示和操縱單位變成sector（默認為GB），輸入print來查看分區信息，記住已有分區的Start值。
刪除分區並新建后，Start值必須與這個相同，否則數據將會丟失。
執行以下命令刪除原有分區：
rm [分區Number]
由上圖可知雲硬盤上有一個分區，Number號為“1”，執行rm 1，結果如下圖：
輸入mkpart primary [原分區起始扇區] 100%
新建一個主分區。
本例中使用mkpart primary 2048s 100%，此主分區從第2048個扇區開始（必須與刪除之前的分區一致），100%表示此分區到磁盤的最末尾。
如果出現如圖狀態請輸入Ignore：
再次輸入print可發現新分區已經新建成功，輸入quit，即可退出parted工具：
檢查擴容后分區的文件系統
使用以下命令檢查擴容后的分區：
e2fsck -f 分區路徑
前述步驟中本例已新建了分區1，使用e2fsck -f /dev/vdb1進行操作。結果如下：
擴容文件系統
執行以下命令進行分區上文件系統的擴容操作：
resize2fs 分區路徑
掛載新分區
執行以下命令掛載分區：
mount 分區路徑 掛載點
這里通過mount /dev/vdb1 /data
手動掛載新分區，並使用df -h命令查看，出現以下信息說明掛載成功，即可以查看到數據盤了。
MBR分區雲硬盤擴容
下面兩種情況都可以選擇使用自動擴容工具（devresize.py）進行擴容
1、原有的硬盤（數據盤）只有一個MBR主分區並制作了文件系統
2、原有的硬盤（數據盤）沒有分區，直接在此硬盤上制作了文件系統
自動擴容工具適用於Linux操作系統，用於將擴容時新擴的雲硬盤存儲空間添加到已存在的文件系統中，擴容能夠成功必須滿足下面3個條件：
1、文件系統是ext2/ext3/ext4,並且只有一個主分區沒有其他主分區和擴展分區
2、當前文件系統不能有錯誤
3、擴容后的磁盤大小不超過2TB
umount 掛載點
wget -O /tmp/devresize.py http://mirrors.tencentyun.com/install/virts/devresize.py 下載一鍵擴容工具
python /tmp/devresize.py 硬盤路徑 雲硬盤，而不是分區名
mount 分區路徑 掛載點
# 具體命令 umount -lf /data wget -O /tmp/devresize.py http://mirrors.tencentyun.com/install/virts/devresize.py python /tmp/devresize.py /dev/vdb mount /dev/vdb1 /data 或 echo "/dev/vdb /data ext4 defaults,noatime,nodiratime 0 0" >>/etc/fstab mount -a
新空間增加到已有分區中(MBR分區格式)
卸載正在使用的硬盤分區
執行以下命令卸載分區：
umount 掛載點
下載一鍵擴容工具
執行以下命令下載工具：
wget -O /tmp/devresize.py http://mirrors.tencentyun.com/install/virts/devresize.py
執行擴容工具
執行以下命令進行擴容：
python /tmp/devresize.py 硬盤路徑
請注意，這里硬盤路徑是需要擴容的雲硬盤，而不是分區名。若您的文件系統在vdb1上，則應執行
python /tmp/devresize.py /dev/vdb
若輸出“The filesystem on /dev/vdb1 is now XXXXX blocks long.“則表示擴容成功。
重新掛載擴容后的分區
執行以下命令掛載擴容后的分區：
mount 分區路徑 掛載點
並通過以下命令查看擴容后的分區容量：
df -h
這里通過mount /dev/vdb1 /data命令手動掛載擴容后的分區(如果原先是沒有分區的，執行mount /dev/vdb /data)，
用df -h命令查看，出現以下信息說明掛載成功，即可以查看到數據盤了:
再執行ll /data命令，可以查看到，擴容后原分區的數據沒有丟失，新增加的存儲空間已經擴容到文件系統中。
devresize.py
#!/usr/bin/env python # coding: utf-8 # FileName: devresize.py """ It only handle the following two situations: 1. There is only one primary partiion in the disk with a format of ext2/3/4; 2. The disk is raw with a file system whose format is ext2/3/4. """ import struct import array import fcntl import time import sys import os import glob import logging BLKSSZGET = 0x1268 BLKGETSIZE = 0x1260 BLKRRPART = 0x125f BLKGETSIZE64 = 0x80041272 logger = None def read_ub(data): return struct.unpack('B', data[0])[0] def read_us(data): return struct.unpack('<H', data[0:2])[0] def read_ui(data): return struct.unpack('<I', data[0:4])[0] def read_ul(data): return struct.unpack('<Q', data[0:8])[0] def init_log(): global logger log_file = 'devresize.log' fmt_file = '%(asctime)s - [%(levelname)-5.5s]- %(filename)s:%(lineno)s - %(message)s' fmt_stream = '[%(levelname)s] - %(message)s' logger = logging.getLogger('devresize') logger.setLevel(logging.DEBUG) file_handler = logging.FileHandler(log_file) file_handler.setLevel(logging.DEBUG) file_handler.setFormatter(logging.Formatter(fmt_file)) logger.addHandler(file_handler) stream_handler = logging.StreamHandler() stream_handler.setLevel(logging.INFO) stream_handler.setFormatter(logging.Formatter(fmt_stream)) logger.addHandler(stream_handler) class PartitionEntry(object): PartitionTypes = { 0x05: "Microsoft Extended", 0x83: "Linux", 0x85: "Linux Extended" } def __init__(self, data): self.data = data self.boot_sig = data[0] self.start_head, self.start_sector, self.start_cylinder = ( PartitionEntry.get_hsc(data[1:1 + 3])) self.partition_type = read_ub(data[4]) self.end_head, self.end_sector, self.end_cylinder = ( PartitionEntry.get_hsc(data[5:5 + 3])) self.start_lba = read_ui(data[8:8 + 4]) self.sector_num = read_ui(data[12:12 + 4]) self.partition_type_name = PartitionEntry.PartitionTypes.get(self.partition_type, "other") @staticmethod def get_hsc(data): h, s, c = struct.unpack('BBB', data[0:3]) c = (c | ((s & 0xC0) << 2)) s = (s & 0x3F) return h, s, c @staticmethod def cal_hsc(sector, hh, ss): s = sector % ss + 1 sector /= ss h = sector % hh sector /= hh c = sector & 0xFF s |= (sector >> 2) & 0xC0 return h, s, c def vaild_type(self): return self.partition_type in self.PartitionTypes def isprimary(self): return self.partition_type == 0x83 def __str__(self): if not self.vaild_type(): print "%x" % self.partition_type return "This isn't a Linux Partition!" return """ Start h,s,c: %u %u %u End h,s,c: %u %u %u Partition Type Name:%s Start LBA: %u Sector Number: %u """ % (self.start_head, self.start_sector, self.start_cylinder, self.end_head, self.end_sector, self.end_cylinder, self.partition_type_name, self.start_lba, self.sector_num) class MBR(object): def __init__(self, data): self.data = data self.boot_code = data[:446] self.mbr_sig = data[510:512] if self.check_mbr_sig(): self.partitions = ([PartitionEntry(data[446 + 16 * i:446 + 16 * (i + 1)]) for i in range(0, 4)]) else: self.partitions = None if self.partitions is not None: self.vaild_part_num = len(filter(lambda x: x.vaild_type(), self.partitions)) else: self.vaild_part_num = 0 self.device_heads = 0 self.device_sectors = 0 self.cal_device_hs() def cal_device_hs(self): if self.partitions is not None and self.vaild_part_num == 1: self.device_heads = self.partitions[0].end_head + 1 self.device_sectors = self.partitions[0].end_sector & 0x3F def check_mbr_sig(self): mbr_sig = read_us(self.mbr_sig) if mbr_sig == 0xAA55: return True else: return False def get_device_size(fd): buf = array.array('c', [chr(0)] * 8) fcntl.ioctl(fd, BLKSSZGET, buf, True) logical_sector_size = read_ul(buf) buf = array.array('c', [chr(0)] * 8) try: fcntl.ioctl(fd, BLKGETSIZE, buf, True) device_size = read_ul(buf) * 512 except IOError: fcntl.ioctl(fd, BLKGETSIZE64, buf, True) device_size = read_ul(buf) device_sector_number = device_size / logical_sector_size logger.debug( '''device_size:%d device_sector_number:%d logical_sector_size:%d''' % (device_size, device_sector_number, logical_sector_size)) return device_size, device_sector_number, logical_sector_size def backup_mbr(data): bak_name = '/tmp/MBR_%s_bak' % time.strftime("%Y-%m-%d_%X", time.localtime()) bak_file = open(bak_name, 'w') bak_file.write(data) bak_file.close() logger.info("Backup MBR to %s" % bak_name) return bak_name def is_first_start(): bak_file_list = glob.glob('/tmp/MBR_%s*bak' % time.strftime("%Y-%m-%d", time.localtime())) return len(bak_file_list) == 0 def cal_new_part(part_data, mbr, start_lab, new_end): device_heads, device_sectors = mbr.device_heads, mbr.device_sectors new_partition_sector_num = new_end - start_lab + 1 begin_h, begin_s, begin_c = PartitionEntry.cal_hsc(start_lab, device_heads, device_sectors) end_h, end_s, end_c = PartitionEntry.cal_hsc(new_end, device_heads, device_sectors) new_part_data = list(part_data[:]) new_part_data[1:1 + 3] = list(struct.pack('BBB', begin_h, begin_s, begin_c)) new_part_data[5:5 + 3] = list(struct.pack('BBB', end_h, end_s, end_c)) new_part_data[0xc:] = list( struct.pack('BBBB', (new_partition_sector_num & 0xff), ((new_partition_sector_num >> 8) & 0xff), ((new_partition_sector_num >> 16) & 0xff), ((new_partition_sector_num >> 24) & 0xff))) logger.debug(""" Start h,s,c: %u %u %u End h,s,c: %u %u %u Partition Type Name:%s Start LBA: %u Sector Number: %u """ % (begin_h, begin_s, begin_c, end_h, end_s, end_c, mbr.partitions[0].partition_type_name, mbr.partitions[0].start_lba, new_partition_sector_num)) return new_part_data # filesystem type must be ext2/3/4 def check_format(dev): blkid_ret = os.popen('blkid %s' % dev) s = blkid_ret.read() if blkid_ret.close() is not None: return False return True in [i in s for i in ['ext2', 'ext3', 'ext4']] def part_probe(fd): logger.debug('part_probe') fd.flush() fcntl.ioctl(fd, BLKRRPART) def resize2fs(dev): ret = os.system('e2fsck -f %s' % dev) logger.debug('e2fsck ret is %d' % ret) if ret not in (0, 1): raise RuntimeError('e2fsck failed!!') if ret == 1: logger.info('File system errors corrected') ret = os.system('resize2fs %s' % dev) logger.debug('resize2fs ret is %d' % ret) if ret != 0: raise RuntimeError('resize2fs failed!!') def check_mount(target_dev): # target_dev is mounted! return os.system('mount | grep "%s " > /dev/null' % target_dev) == 0 def write_mbr(fd, mbr_data): fd.seek(0) fd.write(mbr_data) part_probe(fd) fd.close() time.sleep(1) def check_arg(device): return not device[-1].isdigit() def get_disk_path(partation_name): for i, ch in enumerate(os.path.basename(partation_name)[::-1]): if not ch.isdigit(): return partation_name[::-1][i::][::-1] logger.error("invalid para %s" % partation_name) raise Exception("invalid para %s" % partation_name) def main(): if len(sys.argv) < 2: print "Usage: %s block_device" % sys.argv[0] sys.exit(1) init_log() logger.debug("user input:%s" % ' '.join(sys.argv)) device = sys.argv[1] if not check_arg(device): logger.error("The argument should be a whole disk not a partation!like %s" % get_disk_path(device)) sys.exit(1) if not os.access(device, os.W_OK): logger.error("Permission denied") sys.exit(1) fd = open(device, 'r+') data = fd.read(512) mbr = MBR(data) device_size, device_sector_number, logical_sector_size = get_device_size(fd) if mbr.vaild_part_num > 1: logger.error("Disk %s have multi partition." % device) sys.exit(1) target_partition = '' resize_part_flag = True if mbr.vaild_part_num == 1: # only one partition, which is the primary partition if not mbr.partitions[0].isprimary(): # and the filesystem type is ext2/3/4. logger.error("Must be primary partition.") resize_part_flag = True if device[-1].isdigit(): target_partition = device + 'p1' # ex: /dev/nbd0 -> /dev/nbd0p1 else: target_partition = device + '1' # ex: /dev/vdb -> /dev/vdb1 logger.debug(mbr.partitions[0]) if mbr.vaild_part_num == 0: # no partition but whole disk is ext2/3/4 resize_part_flag = False target_partition = device logger.debug('target_partition:%s' % target_partition) if not check_format(target_partition): logger.error("Only can process ext2/3/4.") sys.exit(1) if check_mount(target_partition): logger.error("Target device %s must be unmounted." % target_partition) sys.exit(1) logger.info("It will resize (%s).\n" "This operation may take from several minutes to several hours, continue? [Y/n]" % target_partition) user_input = raw_input() if user_input.lower() != 'y' and user_input != '': logger.warn("User input neither 'y' nor '[Enter]',exit.") sys.exit(1) if not is_first_start(): logger.warn('We find some MBR backup file in /tmp,maybe the MBR is already changed,' 'do you want to just resize the filesystem? [Y/n]') user_input = raw_input() if user_input.lower() == 'y' or user_input == '': resize_part_flag = False if resize_part_flag: logger.debug("Begin to change the partation") if (mbr.partitions[0].start_lba + mbr.partitions[0].sector_num) == device_sector_number: logger.error("No free sectors available.") sys.exit(1) if mbr.partitions[0].sector_num > 0xFFFFFFFF * 512 / logical_sector_size: logger.error("Can't process the partition which have exceeded 2TB.") sys.exit(1) new_start_sector = mbr.partitions[0].start_lba new_end_sector = device_sector_number - 1 if (new_end_sector - new_start_sector + 1) * logical_sector_size > 0xFFFFFFFF * 512: user_input = raw_input("The size of this disk is %.2fTB (%d bytes).\n" "But DOS partition table format can not be used on drives for volumes larger than 2TB (2199023255040 bytes).\n" "Do you want to resize (%s) to 2TB?[Y/n]" % (round(device_size / 1024.0 / 1024 / 1024 / 1024, 2), device_size, target_partition)) if user_input.lower() != 'y' and user_input != '': logger.warn("User input neither 'y' nor '[Enter]',exit.") sys.exit(1) new_end_sector = 0xFFFFFFFF * 512 / logical_sector_size + new_start_sector - 1 new_mbr_data = list(data)[:] new_mbr_data[446:446 + 16] = cal_new_part(data[446:446 + 16], mbr, new_start_sector, new_end_sector) backup_mbr(data) try: if resize_part_flag: write_mbr(fd, ''.join(new_mbr_data)) resize2fs(target_partition) except Exception, e: logger.error(e) logger.error('Some error occurred!!Maybe you should call the customer service staff.') if __name__ == '__main__': main()
騰訊雲主機做raid
https://cloud.tencent.com/document/product/362/2932
都是系統層面的raid
windows主機
windows2012R2：使用系統自帶磁盤管理器里的 ，新建鏡像卷功能，把兩個彈性雲硬盤連接到雲主機，然后新建鏡像卷做raid1
Linux主機
centos6：Linux內核提供了md模塊在底層管理RAID設備，我們可以使用mdadm工具來調用md模塊。安裝mdadm（以CentOS為例）
f