1、需求分析
1、比對分析
比對的時候以那個數據源為主？
old [1,2,3 ] db數據庫
new [2,3,4 ] 客戶端匯報過來的
當然以客戶端匯報過來的數據為主
2、更新分析
不同的表到底拿那個字段當做唯一值？
old nic {
n1 10.0.0.3 --> 192.168.3.22 #源數據里要拿那個字段去找
n2 172.168.10.2
n3 192.165.52.33
}
- sn：網卡沒有sn ip一個客戶端三個ip 都不靠譜，只有mac是唯一的地址，
- 拿新ip對應的mac地址去到網卡表里搜。拿出來之后，對比ip地址、name
- 如果改變了就更新，不同的表到底拿誰當做唯一值，所以要聲明
3、偽代碼：
def compile_component(table_name， compare_fields=[ip，mac，netmask] unique=mac， client_date=nic_data) for i in compare_fields: db_field = getattr(table_name,i) if db_field==nic_data[i]: else: table_name.db_field = nic_data[i]
1、很多字段，你咋知道那個變了？
- 任何一個字段變了都要記錄下來，改完了之后還要記錄下來
2、有什么好的辦法沒？
- 沒有什么好辦法，你就必須每一個方法都一一對比，你可有N個字段，並且分布在不同的表
- 比對的時候以那個數據源為主
2、update_asset函數解析
def update_asset(self):
func = getattr(self, '_update_%s' % self.clean_data['asset_type'])
create_obj = func()
#每一個表里面的數據都不一樣,服務器端的字段是自己創建的，客戶端沒有，所以只能用客戶端的比
3、_update_server函數解析
def _update_server(self):
nic = self.__update_asset_component(data_source=self.clean_data['nic'],
fk='nic_set',
'''
fk是什么？
網卡關聯asset 反向查找如何查找 a.nic_set;nic_set.select_related()網卡就出來了看到幾個網卡，三個網卡
此處有截圖
為什么要把'nic_set'傳進去？
我通過asset反查我的網卡，內存，name就等於告訴去找那些表，
'''
update_fields=['name', 'sn', 'model', 'macaddress', 'ipaddress', 'netmask',
'bonding'], #要查找的字段
identify_field='macaddress' #唯一值
)
disk = self.__update_asset_component(data_source=self.clean_data['physical_disk_driver'],
fk='disk_set',
update_fields=['slot', 'sn', 'model', 'manufactory', 'capacity',
'iface_type'],
identify_field='slot'
)
ram = self.__update_asset_component(data_source=self.clean_data['ram'],
fk='ram_set',
update_fields=['slot', 'sn', 'model', 'capacity'],
identify_field='slot'
)
cpu = self.__update_cpu_component()
manufactory = self.__update_manufactory_component()
server = self.__update_server_component()
fk是什么？
- 網卡關聯asset 反向查找如何查找 a.nic_set;nic_set.select_related()網卡就出來了看到幾個網卡，三個網卡
- 為什么要把'nic_set'傳進去？
- 我通過asset反查我的網卡，內存，name就等於告訴去找那些表
4、__update_asset_component函數解析
def __update_asset_component(self, data_source, fk, update_fields, identify_field=None):
'''
data_source: the data source of this component from reporting data
fk: which key to use to find the connection between main Asset obj and each asset component
update_fields: what fields in DB will be compared and updated
identify_field: use this field to identify each component of an Asset , if set to None,means only use asset id to identify
'''
print(data_source, update_fields, identify_field)
try:
component_obj = getattr(self.asset_obj, fk)
#傳進來的是字符串，要通過字符串反射成對象
if hasattr(component_obj, 'select_related'): # this component is reverse m2m relation with Asset model
objects_from_db = component_obj.select_related() #把數據取出來，循環數據庫里面的數據
for obj in objects_from_db:
#n1 10.0.0.3 --> 192.168.3.22 源數據里要拿那個字段去找
#b.macaddress；getttr(b.'macaddress')； 8C-EC-4B-16-F9-9B
key_field_data = getattr(obj, identify_field)
# use this key_field_data to find the relative data source from reporting data
if type(data_source) is list:
for source_data_item in data_source:
#到客戶端的數據 源里去找到跟服務器數據庫中key_field_data對應的條目
key_field_data_from_source_data = source_data_item.get(identify_field)
if key_field_data_from_source_data:
if key_field_data == key_field_data_from_source_data: # find the matched source data for this component,then should compare each field in this component to see if there's any changes since last update
self.__compare_componet(model_obj=obj, fields_from_db=update_fields,
data_source=source_data_item)
'''
拿着8C-EC-4B-16-F9-9B去客戶端里找，如果找到就是一樣的，循環是的是三個，
拿着db的網卡，去客戶端里循環，如果找不到要循環三次，
第一沒獲取，不是這塊網卡，
循環第二次找到了，進行對比，就是這塊網卡
如果第二次循環找到，就不要往下摘了，如果我循環了三次還找不到就沒,刪除了，
'''
break #已經根據identify_field找到客戶端中對應的數據條目，且對比完了，后面的loop沒必要繼續了
else: # key field data from source data cannot be none
self.response_msg('warning', 'AssetUpdateWarning',
"Asset component [%s]'s key field [%s] is not provided in reporting data " % (
fk, identify_field))
else: # couldn't find any matches, the asset component must be broken or changed manually
print(
'\033[33;1mError:cannot find any matches in source data by using key field val [%s],component data is missing in reporting data!\033[0m' % (
key_field_data))
self.response_msg("error", "AssetUpdateWarning",
"Cannot find any matches in source data by using key field val [%s],component data is missing in reporting data!" % (
key_field_data))
5、傳進來的是字符串，要通過字符串反射成對象
6、刪除一張網卡驗證一下上面的函數
后台頁面
報錯代碼如下比對一下
self.response_msg("error", "AssetUpdateWarning",
"Cannot find any matches in source data by using key field val [%s],component data is missing in reporting data!" % (
key_field_data))
服務器端控制台（整個比對過程很清晰了）
- 拿着db的網卡，去客戶端里循環，如果找不到要循環三次，
- 第一沒獲取，不是這塊網卡，
- 循環第二次找到了，進行對比，就是這塊網卡
- 如果第二次循環找到，就不要往下摘了，如果我循環了三次還找不到就沒,刪除了，
7、__compare_component函數刨析
源代碼
def __compare_component(self, model_obj, fields_from_db, data_source):
print('---going to compare:[%s]' % model_obj, fields_from_db)
print('---source data:', data_source)
for field in fields_from_db:
val_from_db = getattr(model_obj, field) #獲取到客戶端的數據
val_from_data_source = data_source.get(field) #獲取到數據庫里的數據
if val_from_data_source:
# if type(val_from_db) is unicode:val_from_data_source = unicode(val_from_data_source)#no unicode in py3
# if type(val_from_db) in (int,long):val_from_data_source = int(val_from_data_source) #no long in py3
if type(val_from_db) in (int,):
val_from_data_source = int(val_from_data_source)
#如果數據庫里的數據類型是int，就會把客戶端的數據轉換過成int
elif type(val_from_db) is float:
val_from_data_source = float(val_from_data_source)
elif type(val_from_db) is str:
val_from_data_source = str(val_from_data_source).strip()
if val_from_db == val_from_data_source: # this field haven't changed since last update
pass
# print '\033[32;1m val_from_db[%s] == val_from_data_source[%s]\033[0m' %(val_from_db,val_from_data_source)
else:
print('\033[34;1m val_from_db[%s] != val_from_data_source[%s]\033[0m' % (
val_from_db, val_from_data_source), type(val_from_db), type(val_from_data_source), field)
db_field = model_obj._meta.get_field(field)
db_field.save_form_data(model_obj, val_from_data_source)
model_obj.update_date = timezone.now()
model_obj.save()
log_msg = "Asset[%s] --> component[%s] --> field[%s] has changed from [%s] to [%s]" % (
self.asset_obj, model_obj, field, val_from_db, val_from_data_source)#記錄日志
self.response_msg('info', 'FieldChanged', log_msg) #返回給客戶端的日志
log_handler(self.asset_obj, 'FieldChanged', self.request.user, log_msg, model_obj) #真正要記錄的日志
else:
self.response_msg('warning', 'AssetUpdateWarning',
"Asset component [%s]'s field [%s] is not provided in reporting data " % (
model_obj, field))
model_obj.save()
cpu數據類型設計
雖然是int，但是是字符串形式的，我為了確保不出問題，數據格式按照數據庫里的字段來
獲取到數據庫里的數據 就會把客戶端的數據轉換成int
if val_from_data_source:
# if type(val_from_db) is unicode:val_from_data_source = unicode(val_from_data_source)#no unicode in py3
# if type(val_from_db) in (int,long):val_from_data_source = int(val_from_data_source) #no long in py3
if type(val_from_db) in (int,):
val_from_data_source = int(val_from_data_source)
#如果數據庫里的數據類型是int，就會把客戶端的數據轉換過成int
之前的寫法：
if type(val_from_db) is unicode:val_from_data_source = unicode(val_from_data_source)#no unicode in py3 if type(val_from_db) in (int,long):val_from_data_source = int(val_from_data_source) #no long in py3
記錄日志
日志文件里的日志
8、如何賦一個新值
admin.py后台配置代碼
admin.site.register(models.Asset,AssetAdmin) admin.site.register(models.Server) admin.site.register(models.NetworkDevice) admin.site.register(models.IDC) admin.site.register(models.BusinessUnit) admin.site.register(models.Contract) admin.site.register(models.CPU) admin.site.register(models.Disk) admin.site.register(models.NIC,NicAdmin) admin.site.register(models.RAM) admin.site.register(models.Manufactory) admin.site.register(models.Tag) admin.site.register(models.Software) admin.site.register(models.EventLog,EventLogAdmin) admin.site.register(models.NewAssetApprovalZone,NewAssetApprovalZoneAdmin)
最簡單的方法（同學們的智慧）
alex的方法
代碼
db_field = model_obj._meta.get_field(field)
db_field.save_form_data(model_obj, val_from_data_source)
客戶端截圖和后台截圖
9、__filter_add_or_deleted_components函數刨析
源代碼
def __filter_add_or_deleted_components(self, model_obj_name, data_from_db, data_source, identify_field):
'''This function is filter out all component data in db but missing in reporting data, and all the data in reporting data but not in DB'''
print(data_from_db, data_source, identify_field)
data_source_key_list = [] # save all the identified keys from client data,e.g: [macaddress1,macaddress2]
if type(data_source) is list:
for data in data_source:
data_source_key_list.append(data.get(identify_field))
elif type(data_source) is dict: #dprecated
for key, data in data_source.items():
if data.get(identify_field):
data_source_key_list.append(data.get(identify_field))
else: # workround for some component uses key as identified field e.g: ram
data_source_key_list.append(key)
print('-->identify field [%s] from db :', data_source_key_list) #數據庫里的數據
print('-->identify[%s] from data source:', [getattr(obj, identify_field) for obj in data_from_db])
data_source_key_list = set(data_source_key_list) #列表生成式
data_identify_val_from_db = set([getattr(obj, identify_field) for obj in data_from_db])
data_only_in_db = data_identify_val_from_db - data_source_key_list # delete all this from db 只在數據庫存在的數據
data_only_in_data_source = data_source_key_list - data_identify_val_from_db # add into db #只在客戶端里的數據
print('\033[31;1mdata_only_in_db:\033[0m', data_only_in_db)
print('\033[31;1mdata_only_in_data source:\033[0m', data_only_in_data_source)
self.__delete_components(all_components=data_from_db, delete_list=data_only_in_db, #服務器端刪除
identify_field=identify_field) #以什么作為唯一值
if data_only_in_data_source:
self.__add_components(model_obj_name=model_obj_name, all_components=data_source,#服務器添加
add_list=data_only_in_data_source, identify_field=identify_field)
10、服務器端后台改IP地址 無論怎么改，它永遠和客戶端一直
第一步：后台更改兩次皮地址如下圖
第一更改：
第二次更改
11、__delete_components函數刨析
源代碼：
def __delete_components(self, all_components, delete_list, identify_field):
'''All the objects in delete list will be deleted from DB'''
deleting_obj_list = []
print('--deleting components', delete_list, identify_field)
for obj in all_components:
val = getattr(obj, identify_field)
if val in delete_list:
deleting_obj_list.append(obj)
for i in deleting_obj_list:
log_msg = "Asset[%s] --> component[%s] --> is lacking from reporting source data, assume it has been removed or replaced,will also delete it from DB" % (
self.asset_obj, i)
self.response_msg('info', 'HardwareChanges', log_msg)
log_handler(self.asset_obj, 'HardwareChanges', self.request.user, log_msg, i)
i.delete()
12、__add_components函數刨析
源代碼
def __add_components(self, model_obj_name, all_components, add_list, identify_field):
model_class = getattr(models, model_obj_name)
will_be_creating_list = []
print('--add component list:', add_list)
if type(all_components) is list:
for data in all_components: #客戶端的源數據
if data[identify_field] in add_list:
# print data
will_be_creating_list.append(data)
elif type(all_components) is dict: #deprecated
for k, data in all_components.items():
# workround for some components uses key as identified field ,e.g ram
if data.get(identify_field):
if data[identify_field] in add_list:
# print k,data
will_be_creating_list.append(data)
else: # if the identified field cannot be found from data set,then try to compare the dict key
if k in add_list:
data[
identify_field] = k # add this key into dict , because this dict will be used to create new component item in DB
will_be_creating_list.append(data)
# creating components
try:
for component in will_be_creating_list:
data_set = {}
for field in model_class.auto_create_fields:
data_set[field] = component.get(field)
data_set['asset_id'] = self.asset_obj.id
obj = model_class(**data_set)
obj.save()
print('\033[32;1mCreated component with data:\033[0m', data_set)
log_msg = "Asset[%s] --> component[%s] has justed added a new item [%s]" % (
self.asset_obj, model_obj_name, data_set)
self.response_msg('info', 'NewComponentAdded', log_msg)
log_handler(self.asset_obj, 'NewComponentAdded', self.request.user, log_msg, model_obj_name)
except Exception as e:
print("\033[31;1m %s \033[0m" % e)
log_msg = "Asset[%s] --> component[%s] has error: %s" % (self.asset_obj, model_obj_name, str(e))
self.response_msg('error', "AddingComponentException", log_msg)
添加一張網卡
客戶端截圖
13、__filter_add_or_deleted_components函數刨析
源代碼：
def __filter_add_or_deleted_components(self, model_obj_name, data_from_db, data_source, identify_field):
'''This function is filter out all component data in db but missing in reporting data, and all the data in reporting data but not in DB'''
print(data_from_db, data_source, identify_field)
data_source_key_list = [] # save all the identified keys from client data,e.g: [macaddress1,macaddress2]
if type(data_source) is list:
for data in data_source:
data_source_key_list.append(data.get(identify_field))
elif type(data_source) is dict: #dprecated
for key, data in data_source.items():
if data.get(identify_field):
data_source_key_list.append(data.get(identify_field))
else: # workround for some component uses key as identified field e.g: ram
data_source_key_list.append(key)
print('-->identify field [%s] from db :', data_source_key_list) #數據庫里的數據
print('-->identify[%s] from data source:', [getattr(obj, identify_field) for obj in data_from_db])
data_source_key_list = set(data_source_key_list) #列表生成式
data_identify_val_from_db = set([getattr(obj, identify_field) for obj in data_from_db])
data_only_in_db = data_identify_val_from_db - data_source_key_list # delete all this from db 只在數據庫存在的數據
data_only_in_data_source = data_source_key_list - data_identify_val_from_db # add into db #只在客戶端里的數據
print('\033[31;1mdata_only_in_db:\033[0m', data_only_in_db)
print('\033[31;1mdata_only_in_data source:\033[0m', data_only_in_data_source)
self.__delete_components(all_components=data_from_db, delete_list=data_only_in_db, #服務器端刪除
identify_field=identify_field) #以什么作為唯一值
if data_only_in_data_source:
self.__add_components(model_obj_name=model_obj_name, all_components=data_source,#服務器添加
add_list=data_only_in_data_source, identify_field=identify_field)
以數據庫為主去找的思路，但是客戶端加一塊，服務器端就不知道
是因為你那着數據庫里的兩個網卡，去客戶端里去匹配
總結：上面的方法肯定解決不了問題，就要想別的方法處理
數據庫和客戶端交集並集比對
創建資產組件以什么字段為准
1、不能以客戶端為主，因為客戶端，要是被篡改了，傳過來500個字段
2、自動創建字段要提前定義好，代碼看表結構
硬盤：auto_create_fields = ['sn', 'slot', 'model', 'capacity'] 網卡：auto_create_fields = ['name', 'sn', 'model', 'macaddress', 'ipaddress', 'netmask', 'bonding'] 內存：auto_create_fields = ['sn', 'slot', 'manufactory', 'model', 'capacity', 'iface_type']
14、任意改內存 網卡硬盤數據（對自動更新的整體測試）
服務端給客戶端返回資產id，但是客戶端解析不出來，所以一直往待審批里面匯報
成功客戶端截圖如下
修改成功后台截圖
