文章推薦:(提前了解背景)
Open vSwitch源碼閱讀筆記
SDNLAB技術分享(六):Open vSwitch匹配處理流程和拓展性
ovs流表
OpenvSwitch 流表轉換
上面文章不需要全部看懂,了解即可!!
一:功能目的和擴展字段含義
(一)功能目的:實現openflow字段的延遲更新
參考hard_timeout字段:是為了使得openflow流表項在工作一段時間(即hard_timeout)后,自動失效!!!
而我所要實現的功能就是設置一個字段,可以實現微秒級別的,用來控制一條流表項的生效時間!!!
(二)擴展字段:effect_sec和effect_usec含義
這兩個字段來源於一個時間戳結構體:struct timeval; 該結構體可以精確時間到微秒級別
struct timeval { time_t tv_sec; suseconds_t tv_usec; };

#include <stdio.h> #include <sys/time.h> int main() { struct timeval tv; gettimeofday(&tv, NULL ); printf("---%d---%d-\n",tv.tv_sec,tv.tv_usec); return 0; }
(三)功能演示
1.啟動mininet環境
sudo mn --topo=single,3 --switch=ovsk --controller=none
2.下發普通流表項
sh ovs-ofctl add-flow s1 in_port=1,actions=output:3,2 sh ovs-ofctl add-flow s1 in_port=2,actions=output:1
3.下發自定義字段的流表項
sh ovs-ofctl add-flow s1 in_port=3,effect_sec=1620909655,effect_usec=1811,actions=output:1
下發包含自定義基本字段的流表項之后,使用dpctl dump-flows查看流表項目:
發現我們下發的流表並沒有被添加上去(因為還沒有到生效時間),所以沒有被添加。
此時使用ping命令,發現在沒有達到生效時間時,無法ping通,當達到生效時間后,可以ping通!!!
並且查看流表項:發現我們之前添加的流表項也生效了
二:數據結構了解以及擴展數據
(一)ofputil_flow_mod結構體:整合所有版本的openflow消息結構體(不依賴某個版本)--- 解析下發的流表項字符串到ofputil_flow_mod結構體中
在文件openvswitch-2.11.4/include/openvswitch/ofp-flow.h中為ofputil_flow_mod結構體添加自定義字段
struct ofputil_flow_mod { struct ovs_list list_node; /* For queuing flow_mods. */ struct minimatch match; int priority; ovs_be64 cookie; /* Cookie bits to match. */ ovs_be64 cookie_mask; /* 1-bit in each 'cookie' bit to match. */ ovs_be64 new_cookie; /* New cookie to install or UINT64_MAX. */ bool modify_cookie; /* Set cookie of existing flow to 'new_cookie'? */ uint8_t table_id; uint16_t command; uint16_t idle_timeout; uint16_t hard_timeout; uint32_t buffer_id; ofp_port_t out_port; uint32_t out_group; enum ofputil_flow_mod_flags flags; uint16_t importance; /* Eviction precedence. */ struct ofpact *ofpacts; /* Series of "struct ofpact"s. */ size_t ofpacts_len; /* Length of ofpacts, in bytes. */ uint64_t ofpacts_tlv_bitmap; /* 1-bit for each present TLV in 'ofpacts'. */ uint64_t effect_sec; //----------------------------------修改------------- uint64_t effect_usec; //----------------------------------修改------------- };
(二)為所有需要的openflow協議體添加自定義基本字段---用於解析ofputil_flow_mod到對應的版本下的openflow協議
1.在文件openvswitch-2.11.4/include/openflow/openflow-1.1.h中為ofp11_flow_mod結構體添加自定義字段
/* Flow setup and teardown (controller -> datapath). */ struct ofp11_flow_mod { ovs_be64 cookie; /* Opaque controller-issued identifier. */ ovs_be64 cookie_mask; /* Flow actions. */ uint8_t table_id; /* ID of the table to put the flow in */ uint8_t command; /* One of OFPFC_*. */ ovs_be16 idle_timeout; /* Idle time before discarding (seconds). */ ovs_be16 hard_timeout; /* Max time before discarding (seconds). */ ovs_be16 priority; /* Priority level of flow entry. */ ovs_be32 buffer_id; ovs_be32 out_port; ovs_be32 out_group; ovs_be16 flags; /* One of OFPFF_*. */ ovs_be16 importance; ovs_be64 effect_sec; /*---------------------------------修改-------------*/ ovs_be64 effect_usec; /*----------------------------------修改-------------*/ }; OFP_ASSERT(sizeof(struct ofp11_flow_mod) == 56); /*-------必須是8的倍數-------*/
2.在文件openvswitch-2.11.4/include/openflow/openflow-1.0.h中為ofp10_flow_mod結構體添加自定義字段
/* Flow setup and teardown (controller -> datapath). */ struct ofp10_flow_mod { struct ofp10_match match; /* Fields to match */ ovs_be64 cookie; /* Opaque controller-issued identifier. */ ovs_be16 command; /* One of OFPFC_*. */ ovs_be16 idle_timeout; /* Idle time before discarding (seconds). */ ovs_be16 hard_timeout; /* Max time before discarding (seconds). */ ovs_be16 priority; /* Priority level of flow entry. */ ovs_be32 buffer_id; ovs_be16 out_port; ovs_be16 flags; /* One of OFPFF_*. */ ovs_be64 effect_sec; /*---------------------------------修改-------------*/ ovs_be64 effect_usec; /*----------------------------------修改-------------*/ }; OFP_ASSERT(sizeof(struct ofp10_flow_mod) == 80);
(三)為所有需要的openflow stats協議體添加自定義基本字段,響應OFPST_FLOW請求---用於使用dump-flows命令后顯示流表項信息
1.在文件openvswitch-2.11.4/include/openflow/nicira-ext.h中為nx_flow_stats結構體添加自定義字段
struct nx_flow_stats { ovs_be16 length; /* Length of this entry. */ uint8_t table_id; /* ID of table flow came from. */ uint8_t pad; ovs_be32 duration_sec; /* Time flow has been alive in seconds. */ ovs_be32 duration_nsec; ovs_be16 priority; /* Priority of the entry. */ ovs_be16 idle_timeout; /* Number of seconds idle before expiration. */ ovs_be16 hard_timeout; /* Number of seconds before expiration. */ ovs_be16 match_len; /* Length of nx_match. */ ovs_be16 idle_age; /* Seconds since last packet, plus one. */ ovs_be16 hard_age; /* Seconds since last modification, plus one. */ ovs_be64 cookie; /* Opaque controller-issued identifier. */ ovs_be64 packet_count; /* Number of packets, UINT64_MAX if unknown. */ ovs_be64 byte_count; /* Number of bytes, UINT64_MAX if unknown. */
ovs_be64 effect_sec; /*---------------------------------修改-------------*/ ovs_be64 effect_usec; /*----------------------------------修改-------------*/ }; OFP_ASSERT(sizeof(struct nx_flow_stats) == 64);
2.在文件openvswitch-2.11.4/include/openflow/openflow-1.0.h中為ofp10_flow_stats結構體添加自定義字段
/* Body of reply to OFPST_FLOW request. */ struct ofp10_flow_stats { ovs_be16 length; /* Length of this entry. */ uint8_t table_id; /* ID of table flow came from. */ uint8_t pad; struct ofp10_match match; /* Description of fields. */ ovs_be32 duration_sec; /* Time flow has been alive in seconds. */ ovs_be32 duration_nsec; ovs_be16 priority; ovs_be16 idle_timeout; /* Number of seconds idle before expiration. */ ovs_be16 hard_timeout; /* Number of seconds before expiration. */ uint8_t pad2[6]; /* Align to 64 bits. */ ovs_32aligned_be64 cookie; /* Opaque controller-issued identifier. */ ovs_32aligned_be64 packet_count; /* Number of packets in flow. */ ovs_32aligned_be64 byte_count; /* Number of bytes in flow. */ ovs_be64 effect_sec; /*---------------------------------修改-------------*/ ovs_be64 effect_usec; /*----------------------------------修改-------------*/ }; OFP_ASSERT(sizeof(struct ofp10_flow_stats) == 104);
3.在文件openvswitch-2.11.4/include/openflow/openflow-1.1.h中為ofp11_flow_stats結構體添加自定義字段
/* Body of reply to OFPST_FLOW request. */ struct ofp11_flow_stats { ovs_be16 length; /* Length of this entry. */ uint8_t table_id; /* ID of table flow came from. */ uint8_t pad; ovs_be32 duration_sec; /* Time flow has been alive in seconds. */ ovs_be32 duration_nsec; ovs_be16 priority; ovs_be16 idle_timeout; /* Number of seconds idle before expiration. */ ovs_be16 hard_timeout; /* Number of seconds before expiration. */ ovs_be16 flags; /* OF 1.3: Set of OFPFF*. */ ovs_be16 importance; /* Eviction precedence (OF1.4+). */ uint8_t pad2[2]; /* Align to 64-bits. */ ovs_be64 cookie; /* Opaque controller-issued identifier. */ ovs_be64 packet_count; /* Number of packets in flow. */ ovs_be64 byte_count; /* Number of bytes in flow. */ ovs_be64 effect_sec; /*---------------------------------修改-------------*/ ovs_be64 effect_usec; /*----------------------------------修改-------------*/ }; OFP_ASSERT(sizeof(struct ofp11_flow_stats) == 64);
4.在文件openvswitch-2.11.4/include/openflow/ofp-flow.h中為ofputil_flow_stats結構體添加自定義字段---重點:這個結構體和前面1、2、3結構體的關系同(一)與(二)的關系
(1)通過ofctl_dump_flows--回調-->vconn_dump_flows(內部傳參修改struct ofputil_flow_stats **fsesp)--調用--> ofputil_encode_flow_stats_request去解析各個版本下的openflow協議狀態;最后將所有要顯示的數據放入ofputil_flow_stats中
(2)再通過ofctl_dump_flows--調用-->ofputil_flow_stats_format(傳參struct ofputil_flow_stats )--調用-->ds_put_format格式化要顯示的流表項字符串信息
/* Flow stats reply, independent of protocol. */ struct ofputil_flow_stats { struct match match; ovs_be64 cookie; uint8_t table_id; uint16_t priority; uint16_t idle_timeout; uint16_t hard_timeout; uint32_t duration_sec; uint32_t duration_nsec; int idle_age; /* Seconds since last packet, -1 if unknown. */ int hard_age; /* Seconds since last change, -1 if unknown. */ uint64_t packet_count; /* Packet count, UINT64_MAX if unknown. */ uint64_t byte_count; /* Byte count, UINT64_MAX if unknown. */ const struct ofpact *ofpacts; size_t ofpacts_len; enum ofputil_flow_mod_flags flags; uint16_t importance; /* Eviction precedence. */ uint64_t effect_sec; //----------------------------------修改------------- uint64_t effect_usec; //----------------------------------修改------------- };
(四)添加規則狀態(未生效狀態)和修改規則結構體---最后會將上面的openflow流表轉換為規則插入datapath中進行緩存
在openvswitch/openvswitch-2.11.4/ofproto/ofproto-provider.h文件中
1.添加規則狀態
enum OVS_PACKED_ENUM rule_state { RULE_INITIALIZED, RULE_INSERTED, RULE_REMOVED, RULE_EFFECTED, /* ----------rule effect time------------*/ };
2.修改規則結構體
struct rule { struct ofproto *const ofproto; /* The ofproto that contains this rule. */ const struct cls_rule cr; /* In owning ofproto's classifier. */ const uint8_t table_id; /* Index in ofproto's 'tables' array. */ enum rule_state state; ...... /* Timeouts. */ uint16_t hard_timeout OVS_GUARDED; /* In seconds from ->modified. */ uint16_t idle_timeout OVS_GUARDED; /* In seconds from ->used. */ /*------------------- effect time -------------------*/ uint64_t effect_sec OVS_GUARDED; uint64_t effect_usec OVS_GUARDED; const struct ofproto_flow_mod* ofm; /*----------存儲部分classifier_insert需要用到的參數---------*/ ...... const struct rule_actions * const actions; ...... struct ovs_list expirable OVS_GUARDED_BY(ofproto_mutex); struct ovs_list effectable OVS_GUARDED_BY(ofproto_mutex); /*-----effect time --------*/ ...... };
3.修改struct ofproto_flow_mod結構體,因為需要保留請求中的數據
/* flow_mod with execution context. */ struct ofproto_flow_mod { /* Allocated by 'init' phase, may be freed after 'start' phase, as these * are not needed for 'revert' nor 'finish'. * * This structure owns a reference to 'temp_rule' (if it is nonnull) that * must be eventually be released with ofproto_rule_unref(). */ struct rule *temp_rule; struct rule_criteria criteria; struct cls_conjunction *conjs; size_t n_conjs; /* Replicate needed fields from ofputil_flow_mod to not need it after the * flow has been created. */ uint16_t command; bool modify_cookie; /* Fields derived from ofputil_flow_mod. */ bool modify_may_add_flow; bool modify_keep_counts; enum nx_flow_update_event event; /* These are only used during commit execution. * ofproto_flow_mod_uninit() does NOT clean these up. */ ovs_version_t version; /* Version in which changes take * effect. */ bool learn_adds_rule; /* Learn execution adds a rule. */ struct rule_collection old_rules; /* Affected rules. */ struct rule_collection new_rules; /* Replacement rules. */ /*--------------存儲請求--------------------*/ struct openflow_mod_requester * omr; };
三:控制面實現,解析下發的流表項字符串到ofputil_flow_mod結構體中
流表下發一般是通過以下兩種方式:
1.controller通過openflow協議下發FLOW_MOD命令給ovs的Userspace流表。
2.ovs-ofctl通過openflow協議下發FLOW_MOD給ovs的Userspace流表。
ovs-ofctl add-flow最終調用 ofctl_flow_mod(ctx->argc, ctx->argv, OFPFC_ADD);
(一)調用流程
ofctl_flow_mod--調用-->parse_ofp_flow_mod_str--調用-->parse_ofp_str--調用-->parse_ofp_str__實現解析字符串到ofputil_flow_mod結構體中
(二)修改parse_ofp_str__函數,使得解析字符串中的effect_sec和effect_usec字段

static char * OVS_WARN_UNUSED_RESULT parse_ofp_str__(struct ofputil_flow_mod *fm, int command, char *string, //----------------重點:1.字符串到ofputil_flow_mod const struct ofputil_port_map *port_map, const struct ofputil_table_map *table_map, enum ofputil_protocol *usable_protocols) { VLOG_INFO("---------------parse_ofp_str__------------start--------\n"); enum { F_OUT_PORT = 1 << 0, F_ACTIONS = 1 << 1, F_IMPORTANCE = 1 << 2, F_TIMEOUT = 1 << 3, F_PRIORITY = 1 << 4, F_FLAGS = 1 << 5, } fields; char *act_str = NULL; char *name, *value; *usable_protocols = OFPUTIL_P_ANY; if (command == -2) { size_t len; string += strspn(string, " \t\r\n"); /* Skip white space. */ len = strcspn(string, ", \t\r\n"); /* Get length of the first token. */ if (!strncmp(string, "add", len)) { command = OFPFC_ADD; } else if (!strncmp(string, "delete", len)) { command = OFPFC_DELETE; } else if (!strncmp(string, "delete_strict", len)) { command = OFPFC_DELETE_STRICT; } else if (!strncmp(string, "modify", len)) { command = OFPFC_MODIFY; } else if (!strncmp(string, "modify_strict", len)) { command = OFPFC_MODIFY_STRICT; } else { len = 0; command = OFPFC_ADD; } string += len; } switch (command) { case -1: fields = F_OUT_PORT; break; case OFPFC_ADD: fields = F_ACTIONS | F_TIMEOUT | F_PRIORITY | F_FLAGS | F_IMPORTANCE; break; case OFPFC_DELETE: fields = F_OUT_PORT; break; case OFPFC_DELETE_STRICT: fields = F_OUT_PORT | F_PRIORITY; break; case OFPFC_MODIFY: fields = F_ACTIONS | F_TIMEOUT | F_PRIORITY | F_FLAGS; break; case OFPFC_MODIFY_STRICT: fields = F_ACTIONS | F_TIMEOUT | F_PRIORITY | F_FLAGS; break; default: OVS_NOT_REACHED(); } *fm = (struct ofputil_flow_mod) { .priority = OFP_DEFAULT_PRIORITY, .table_id = 0xff, .command = command, .buffer_id = UINT32_MAX, .out_port = OFPP_ANY, .out_group = OFPG_ANY, .effect_sec = 0, .effect_usec = 0, }; VLOG_INFO("---------------parse_ofp_str__----flow_mod idle_timeout:%d--------start--------\n",fm->idle_timeout); VLOG_INFO("---------------parse_ofp_str__----flow_mod effect_time:%d %d--------start--------\n",fm->effect_sec,fm->effect_usec); /* For modify, by default, don't update the cookie. */ if (command == OFPFC_MODIFY || command == OFPFC_MODIFY_STRICT) { fm->new_cookie = OVS_BE64_MAX; } if (fields & F_ACTIONS) { act_str = ofp_extract_actions(string); if (!act_str) { return xstrdup("must specify an action"); } } struct match match = MATCH_CATCHALL_INITIALIZER; while (ofputil_parse_key_value(&string, &name, &value)) { const struct ofp_protocol *p; const struct mf_field *mf; char *error = NULL; if (ofp_parse_protocol(name, &p)) { match_set_dl_type(&match, htons(p->dl_type)); if (p->nw_proto) { match_set_nw_proto(&match, p->nw_proto); } match_set_default_packet_type(&match); } else if (!strcmp(name, "eth")) { match_set_packet_type(&match, htonl(PT_ETH)); } else if (fields & F_FLAGS && !strcmp(name, "send_flow_rem")) { fm->flags |= OFPUTIL_FF_SEND_FLOW_REM; } else if (fields & F_FLAGS && !strcmp(name, "check_overlap")) { fm->flags |= OFPUTIL_FF_CHECK_OVERLAP; } else if (fields & F_FLAGS && !strcmp(name, "reset_counts")) { fm->flags |= OFPUTIL_FF_RESET_COUNTS; *usable_protocols &= OFPUTIL_P_OF12_UP; } else if (fields & F_FLAGS && !strcmp(name, "no_packet_counts")) { fm->flags |= OFPUTIL_FF_NO_PKT_COUNTS; *usable_protocols &= OFPUTIL_P_OF13_UP; } else if (fields & F_FLAGS && !strcmp(name, "no_byte_counts")) { fm->flags |= OFPUTIL_FF_NO_BYT_COUNTS; *usable_protocols &= OFPUTIL_P_OF13_UP; } else if (!strcmp(name, "no_readonly_table") || !strcmp(name, "allow_hidden_fields")) { /* ignore these fields. */ } else if ((mf = mf_from_name(name)) != NULL) { error = ofp_parse_field(mf, value, port_map, &match, usable_protocols); } else if (strchr(name, '[')) { error = parse_subfield(name, value, &match, usable_protocols); } else { if (!*value) { return xasprintf("field %s missing value", name); } if (!strcmp(name, "table")) { if (!ofputil_table_from_string(value, table_map, &fm->table_id)) { return xasprintf("unknown table \"%s\"", value); } if (fm->table_id != 0xff) { *usable_protocols &= OFPUTIL_P_TID; } } else if (fields & F_OUT_PORT && !strcmp(name, "out_port")) { if (!ofputil_port_from_string(value, port_map, &fm->out_port)) { error = xasprintf("%s is not a valid OpenFlow port", value); } } else if (fields & F_OUT_PORT && !strcmp(name, "out_group")) { *usable_protocols &= OFPUTIL_P_OF11_UP; if (!ofputil_group_from_string(value, &fm->out_group)) { error = xasprintf("%s is not a valid OpenFlow group", value); } } else if (fields & F_PRIORITY && !strcmp(name, "priority")) { uint16_t priority = 0; error = str_to_u16(value, name, &priority); fm->priority = priority; } else if (fields & F_TIMEOUT && !strcmp(name, "idle_timeout")) { error = str_to_u16(value, name, &fm->idle_timeout); } else if (fields & F_TIMEOUT && !strcmp(name, "hard_timeout")) { error = str_to_u16(value, name, &fm->hard_timeout); } else if (fields & F_IMPORTANCE && !strcmp(name, "importance")) { error = str_to_u16(value, name, &fm->importance); } else if(!strcmp(name, "effect_sec")){ //-------------------effect_sec--------------- error = str_to_u64(value, &fm->effect_sec); } else if(!strcmp(name, "effect_usec")){ //-----------------effect_usec----------------- error = str_to_u64(value, &fm->effect_usec); } else if (!strcmp(name, "cookie")) { char *mask = strchr(value, '/'); if (mask) { /* A mask means we're searching for a cookie. */ if (command == OFPFC_ADD) { return xstrdup("flow additions cannot use " "a cookie mask"); } *mask = '\0'; error = str_to_be64(value, &fm->cookie); if (error) { return error; } error = str_to_be64(mask + 1, &fm->cookie_mask); /* Matching of the cookie is only supported through NXM or * OF1.1+. */ if (fm->cookie_mask != htonll(0)) { *usable_protocols &= OFPUTIL_P_NXM_OF11_UP; } } else { /* No mask means that the cookie is being set. */ if (command != OFPFC_ADD && command != OFPFC_MODIFY && command != OFPFC_MODIFY_STRICT) { return xstrdup("cannot set cookie"); } error = str_to_be64(value, &fm->new_cookie); fm->modify_cookie = true; } } else if (!strcmp(name, "duration") || !strcmp(name, "n_packets") || !strcmp(name, "n_bytes") || !strcmp(name, "idle_age") || !strcmp(name, "hard_age")) { /* Ignore these, so that users can feed the output of * "ovs-ofctl dump-flows" back into commands that parse * flows. */ } else { error = xasprintf("unknown keyword %s", name); } } if (error) { return error; } } VLOG_INFO("---------------parse_ofp_str__----flow_mod idle_timeout:%d--------start--------\n",fm->idle_timeout); VLOG_INFO("---------------parse_ofp_str__----flow_mod effect_sec:%d--effect_usec:%d------start--------\n",fm->effect_sec,fm->effect_usec); /* Copy ethertype to flow->dl_type for matches on packet_type * (OFPHTN_ETHERTYPE, ethertype). */ if (match.wc.masks.packet_type == OVS_BE32_MAX && pt_ns(match.flow.packet_type) == OFPHTN_ETHERTYPE) { match.flow.dl_type = pt_ns_type_be(match.flow.packet_type); } /* Check for usable protocol interdependencies between match fields. */ if (match.flow.dl_type == htons(ETH_TYPE_IPV6)) { const struct flow_wildcards *wc = &match.wc; /* Only NXM and OXM support matching L3 and L4 fields within IPv6. * * (IPv6 specific fields as well as arp_sha, arp_tha, nw_frag, and * nw_ttl are covered elsewhere so they don't need to be included in * this test too.) */ if (wc->masks.nw_proto || wc->masks.nw_tos || wc->masks.tp_src || wc->masks.tp_dst) { *usable_protocols &= OFPUTIL_P_NXM_OXM_ANY; } } if (!fm->cookie_mask && fm->new_cookie == OVS_BE64_MAX && (command == OFPFC_MODIFY || command == OFPFC_MODIFY_STRICT)) { /* On modifies without a mask, we are supposed to add a flow if * one does not exist. If a cookie wasn't been specified, use a * default of zero. */ fm->new_cookie = htonll(0); } if (fields & F_ACTIONS) { enum ofputil_protocol action_usable_protocols; struct ofpbuf ofpacts; char *error; ofpbuf_init(&ofpacts, 32); struct ofpact_parse_params pp = { .port_map = port_map, .table_map = table_map, .ofpacts = &ofpacts, .usable_protocols = &action_usable_protocols }; error = ofpacts_parse_instructions(act_str, &pp); *usable_protocols &= action_usable_protocols; if (!error) { enum ofperr err; struct ofpact_check_params cp = { .match = &match, .max_ports = OFPP_MAX, .table_id = fm->table_id, .n_tables = 255, }; err = ofpacts_check(ofpacts.data, ofpacts.size, &cp); *usable_protocols &= cp.usable_protocols; if (!err && !*usable_protocols) { err = OFPERR_OFPBAC_MATCH_INCONSISTENT; } if (err) { error = xasprintf("actions are invalid with specified match " "(%s)", ofperr_to_string(err)); } } if (error) { ofpbuf_uninit(&ofpacts); return error; } fm->ofpacts_len = ofpacts.size; fm->ofpacts = ofpbuf_steal_data(&ofpacts); } else { fm->ofpacts_len = 0; fm->ofpacts = NULL; } minimatch_init(&fm->match, &match); return NULL; }
static char * OVS_WARN_UNUSED_RESULT parse_ofp_str__(struct ofputil_flow_mod *fm, int command, char *string, //----------------重點:1.字符串到ofputil_flow_mod const struct ofputil_port_map *port_map, const struct ofputil_table_map *table_map, enum ofputil_protocol *usable_protocols) {
......*fm = (struct ofputil_flow_mod) { .priority = OFP_DEFAULT_PRIORITY, .table_id = 0xff, .command = command, .buffer_id = UINT32_MAX, .out_port = OFPP_ANY, .out_group = OFPG_ANY, .effect_sec = 0, //設置默認值 .effect_usec = 0, };
struct match match = MATCH_CATCHALL_INITIALIZER; while (ofputil_parse_key_value(&string, &name, &value)) { const struct ofp_protocol *p; const struct mf_field *mf; char *error = NULL; if (ofp_parse_protocol(name, &p)) { match_set_dl_type(&match, htons(p->dl_type)); if (p->nw_proto) { match_set_nw_proto(&match, p->nw_proto); } match_set_default_packet_type(&match); } else if (!strcmp(name, "eth")) { match_set_packet_type(&match, htonl(PT_ETH)); } else if (fields & F_FLAGS && !strcmp(name, "send_flow_rem")) { fm->flags |= OFPUTIL_FF_SEND_FLOW_REM; } else if (fields & F_FLAGS && !strcmp(name, "check_overlap")) { fm->flags |= OFPUTIL_FF_CHECK_OVERLAP; } else if (fields & F_FLAGS && !strcmp(name, "reset_counts")) { fm->flags |= OFPUTIL_FF_RESET_COUNTS; *usable_protocols &= OFPUTIL_P_OF12_UP; } else if (fields & F_FLAGS && !strcmp(name, "no_packet_counts")) { fm->flags |= OFPUTIL_FF_NO_PKT_COUNTS; *usable_protocols &= OFPUTIL_P_OF13_UP; } else if (fields & F_FLAGS && !strcmp(name, "no_byte_counts")) { fm->flags |= OFPUTIL_FF_NO_BYT_COUNTS; *usable_protocols &= OFPUTIL_P_OF13_UP; } else if (!strcmp(name, "no_readonly_table") || !strcmp(name, "allow_hidden_fields")) { /* ignore these fields. */ } else if ((mf = mf_from_name(name)) != NULL) { error = ofp_parse_field(mf, value, port_map, &match, usable_protocols); } else if (strchr(name, '[')) { error = parse_subfield(name, value, &match, usable_protocols); } else { if (!*value) { return xasprintf("field %s missing value", name); } if (!strcmp(name, "table")) { if (!ofputil_table_from_string(value, table_map, &fm->table_id)) { return xasprintf("unknown table \"%s\"", value); } if (fm->table_id != 0xff) { *usable_protocols &= OFPUTIL_P_TID; } } else if (fields & F_OUT_PORT && !strcmp(name, "out_port")) { if (!ofputil_port_from_string(value, port_map, &fm->out_port)) { error = xasprintf("%s is not a valid OpenFlow port", value); } } else if (fields & F_OUT_PORT && !strcmp(name, "out_group")) { *usable_protocols &= OFPUTIL_P_OF11_UP; if (!ofputil_group_from_string(value, &fm->out_group)) { error = xasprintf("%s is not a valid OpenFlow group", value); } } else if (fields & F_PRIORITY && !strcmp(name, "priority")) { uint16_t priority = 0; error = str_to_u16(value, name, &priority); fm->priority = priority; } else if (fields & F_TIMEOUT && !strcmp(name, "idle_timeout")) { error = str_to_u16(value, name, &fm->idle_timeout); } else if (fields & F_TIMEOUT && !strcmp(name, "hard_timeout")) { error = str_to_u16(value, name, &fm->hard_timeout); } else if (fields & F_IMPORTANCE && !strcmp(name, "importance")) { error = str_to_u16(value, name, &fm->importance); } else if(!strcmp(name, "effect_sec")){ //-------------------effect_sec--------------- error = str_to_u64(value, &fm->effect_sec); } else if(!strcmp(name, "effect_usec")){ //-----------------effect_usec----------------- error = str_to_u64(value, &fm->effect_usec); } else if (!strcmp(name, "cookie")) { ...... } else if (!strcmp(name, "duration") || !strcmp(name, "n_packets") || !strcmp(name, "n_bytes") || !strcmp(name, "idle_age") || !strcmp(name, "hard_age")) { } else { error = xasprintf("unknown keyword %s", name); } } if (error) { return error; } }
......
處理match和action
...... }
四:控制面實現,轉換由三獲取ofputil_flow_mod結構體數據到對應的openflow版本中去
(一)調用流程
ofctl_flow_mod--調用-->ofctl_flow_mod__--調用-->ofputil_encode_flow_mod--返回-->struct ofpbuf *msg,其中msg中封裝了各個版本的openflow協議信息
msg = ofpraw_alloc(OFPRAW_OFPT11_FLOW_MOD, version, tailroom);
最后在ofctl_flow_mod__中通過下面函數轉發到對應的網橋中去(控制面和數據面交互)!!!https://www.cnblogs.com/liuhongru/p/11399046.html
transact_noreply(vconn, ofputil_encode_flow_mod(fm, protocol));
ofctl_flow_mod__會打開一個指向ovs-vswitchd的socket,將ofputil_flow_mod變成openflow的協議,發出去transact_noreply
補充:vconn與OVS網橋的連接
connmgr即connect manager連接管理器,主要完成OVS網橋的連接管理。每一個網橋ofproto都有一個connmgr實體來管理連接。
(二)修改ofputil_encode_flow_mod函數,ofputil_flow_mod結構體數據到對應的openflow版本中去

struct ofpbuf * ofputil_encode_flow_mod(const struct ofputil_flow_mod *fm, //---------------重點:從ofputil_flow_mod中解析ofp11_flow_mod enum ofputil_protocol protocol) { enum ofp_version version = ofputil_protocol_to_ofp_version(protocol); ovs_be16 raw_flags = ofputil_encode_flow_mod_flags(fm->flags, version); struct ofpbuf *msg; struct match match; minimatch_expand(&fm->match, &match); VLOG_INFO("--------------ofputil_encode_flow_mod-------------start--------------"); switch (protocol) { case OFPUTIL_P_OF11_STD: case OFPUTIL_P_OF12_OXM: case OFPUTIL_P_OF13_OXM: case OFPUTIL_P_OF14_OXM: case OFPUTIL_P_OF15_OXM: case OFPUTIL_P_OF16_OXM: { struct ofp11_flow_mod *ofm; //---------------重點:從ofputil_flow_mod中解析ofp11_flow_mod int tailroom; VLOG_INFO("--------------ofputil_encode_flow_mod-------ofp11_flow_mod------start--------------"); tailroom = ofputil_match_typical_len(protocol) + fm->ofpacts_len; msg = ofpraw_alloc(OFPRAW_OFPT11_FLOW_MOD, version, tailroom); ofm = ofpbuf_put_zeros(msg, sizeof *ofm); if ((protocol == OFPUTIL_P_OF11_STD && (fm->command == OFPFC_MODIFY || fm->command == OFPFC_MODIFY_STRICT) && fm->cookie_mask == htonll(0)) || fm->command == OFPFC_ADD) { ofm->cookie = fm->new_cookie; } else { ofm->cookie = fm->cookie & fm->cookie_mask; } ofm->cookie_mask = fm->cookie_mask; if (fm->table_id != OFPTT_ALL || (protocol != OFPUTIL_P_OF11_STD && (fm->command == OFPFC_DELETE || fm->command == OFPFC_DELETE_STRICT))) { ofm->table_id = fm->table_id; } else { ofm->table_id = 0; } ofm->command = fm->command; ofm->idle_timeout = htons(fm->idle_timeout); ofm->hard_timeout = htons(fm->hard_timeout); ofm->effect_sec = htonl(fm->effect_sec); ofm->effect_usec = htonl(fm->effect_usec); ofm->priority = htons(fm->priority); ofm->buffer_id = htonl(fm->buffer_id); ofm->out_port = ofputil_port_to_ofp11(fm->out_port); ofm->out_group = htonl(fm->out_group); ofm->flags = raw_flags; if (version >= OFP14_VERSION && fm->command == OFPFC_ADD) { ofm->importance = htons(fm->importance); } else { ofm->importance = 0; } ofputil_put_ofp11_match(msg, &match, protocol); ofpacts_put_openflow_instructions(fm->ofpacts, fm->ofpacts_len, msg, version); break; } case OFPUTIL_P_OF10_STD: case OFPUTIL_P_OF10_STD_TID: { struct ofp10_flow_mod *ofm; VLOG_INFO("--------------ofputil_encode_flow_mod-------ofp10_flow_mod------start--------------"); msg = ofpraw_alloc(OFPRAW_OFPT10_FLOW_MOD, OFP10_VERSION, fm->ofpacts_len); ofm = ofpbuf_put_zeros(msg, sizeof *ofm); ofputil_match_to_ofp10_match(&match, &ofm->match); ofm->cookie = fm->new_cookie; ofm->command = ofputil_tid_command(fm, protocol); ofm->idle_timeout = htons(fm->idle_timeout); ofm->hard_timeout = htons(fm->hard_timeout); ofm->effect_sec = htonl(fm->effect_sec); ofm->effect_usec = htonl(fm->effect_usec); ofm->priority = htons(fm->priority); ofm->buffer_id = htonl(fm->buffer_id); ofm->out_port = htons(ofp_to_u16(fm->out_port)); ofm->flags = raw_flags; ofpacts_put_openflow_actions(fm->ofpacts, fm->ofpacts_len, msg, version); break; } case OFPUTIL_P_OF10_NXM: case OFPUTIL_P_OF10_NXM_TID: { struct nx_flow_mod *nfm; int match_len; VLOG_INFO("--------------ofputil_encode_flow_mod-------nx_flow_mod------start--------------"); msg = ofpraw_alloc(OFPRAW_NXT_FLOW_MOD, OFP10_VERSION, NXM_TYPICAL_LEN + fm->ofpacts_len); nfm = ofpbuf_put_zeros(msg, sizeof *nfm); nfm->command = ofputil_tid_command(fm, protocol); nfm->cookie = fm->new_cookie; match_len = nx_put_match(msg, &match, fm->cookie, fm->cookie_mask); nfm = msg->msg; nfm->idle_timeout = htons(fm->idle_timeout); nfm->hard_timeout = htons(fm->hard_timeout); nfm->priority = htons(fm->priority); nfm->buffer_id = htonl(fm->buffer_id); nfm->out_port = htons(ofp_to_u16(fm->out_port)); nfm->flags = raw_flags; nfm->match_len = htons(match_len); ofpacts_put_openflow_actions(fm->ofpacts, fm->ofpacts_len, msg, version); break; } default: OVS_NOT_REACHED(); } ofpmsg_update_length(msg); return msg; }
struct ofpbuf * ofputil_encode_flow_mod(const struct ofputil_flow_mod *fm, //---------------重點:從ofputil_flow_mod中解析ofp11_flow_mod enum ofputil_protocol protocol) { enum ofp_version version = ofputil_protocol_to_ofp_version(protocol); ovs_be16 raw_flags = ofputil_encode_flow_mod_flags(fm->flags, version); struct ofpbuf *msg; struct match match; minimatch_expand(&fm->match, &match);switch (protocol) { case OFPUTIL_P_OF11_STD: case OFPUTIL_P_OF12_OXM: case OFPUTIL_P_OF13_OXM: case OFPUTIL_P_OF14_OXM: case OFPUTIL_P_OF15_OXM: case OFPUTIL_P_OF16_OXM: { struct ofp11_flow_mod *ofm; //---------------重點:從ofputil_flow_mod中解析ofp11_flow_mod int tailroom; tailroom = ofputil_match_typical_len(protocol) + fm->ofpacts_len; msg = ofpraw_alloc(OFPRAW_OFPT11_FLOW_MOD, version, tailroom); ofm = ofpbuf_put_zeros(msg, sizeof *ofm); if ((protocol == OFPUTIL_P_OF11_STD && (fm->command == OFPFC_MODIFY || fm->command == OFPFC_MODIFY_STRICT) && fm->cookie_mask == htonll(0)) || fm->command == OFPFC_ADD) { ofm->cookie = fm->new_cookie; } else { ofm->cookie = fm->cookie & fm->cookie_mask; } ofm->cookie_mask = fm->cookie_mask; if (fm->table_id != OFPTT_ALL || (protocol != OFPUTIL_P_OF11_STD && (fm->command == OFPFC_DELETE || fm->command == OFPFC_DELETE_STRICT))) { ofm->table_id = fm->table_id; } else { ofm->table_id = 0; } ofm->command = fm->command; ofm->idle_timeout = htons(fm->idle_timeout); ofm->hard_timeout = htons(fm->hard_timeout); ofm->effect_sec = htonll(fm->effect_sec); ofm->effect_usec = htonll(fm->effect_usec); ofm->priority = htons(fm->priority); ofm->buffer_id = htonl(fm->buffer_id); ofm->out_port = ofputil_port_to_ofp11(fm->out_port); ofm->out_group = htonl(fm->out_group); ofm->flags = raw_flags; if (version >= OFP14_VERSION && fm->command == OFPFC_ADD) { ofm->importance = htons(fm->importance); } else { ofm->importance = 0; } ofputil_put_ofp11_match(msg, &match, protocol); ofpacts_put_openflow_instructions(fm->ofpacts, fm->ofpacts_len, msg, version); break; } case OFPUTIL_P_OF10_STD: case OFPUTIL_P_OF10_STD_TID: { struct ofp10_flow_mod *ofm; VLOG_INFO("--------------ofputil_encode_flow_mod-------ofp10_flow_mod------start--------------"); msg = ofpraw_alloc(OFPRAW_OFPT10_FLOW_MOD, OFP10_VERSION, fm->ofpacts_len); ofm = ofpbuf_put_zeros(msg, sizeof *ofm); ofputil_match_to_ofp10_match(&match, &ofm->match); ofm->cookie = fm->new_cookie; ofm->command = ofputil_tid_command(fm, protocol); ofm->idle_timeout = htons(fm->idle_timeout); ofm->hard_timeout = htons(fm->hard_timeout); ofm->effect_sec = htonll(fm->effect_sec); ofm->effect_usec = htonll(fm->effect_usec); ofm->priority = htons(fm->priority); ofm->buffer_id = htonl(fm->buffer_id); ofm->out_port = htons(ofp_to_u16(fm->out_port)); ofm->flags = raw_flags; ofpacts_put_openflow_actions(fm->ofpacts, fm->ofpacts_len, msg, version); break; } case OFPUTIL_P_OF10_NXM: case OFPUTIL_P_OF10_NXM_TID: { struct nx_flow_mod *nfm; int match_len; VLOG_INFO("--------------ofputil_encode_flow_mod-------nx_flow_mod------start--------------"); msg = ofpraw_alloc(OFPRAW_NXT_FLOW_MOD, OFP10_VERSION, NXM_TYPICAL_LEN + fm->ofpacts_len); nfm = ofpbuf_put_zeros(msg, sizeof *nfm); nfm->command = ofputil_tid_command(fm, protocol); nfm->cookie = fm->new_cookie; match_len = nx_put_match(msg, &match, fm->cookie, fm->cookie_mask); nfm = msg->msg; nfm->idle_timeout = htons(fm->idle_timeout); nfm->hard_timeout = htons(fm->hard_timeout); nfm->priority = htons(fm->priority); nfm->buffer_id = htonl(fm->buffer_id); nfm->out_port = htons(ofp_to_u16(fm->out_port)); nfm->flags = raw_flags; nfm->match_len = htons(match_len); ofpacts_put_openflow_actions(fm->ofpacts, fm->ofpacts_len, msg, version); break; } default: OVS_NOT_REACHED(); } ofpmsg_update_length(msg); return msg; }
五:控制面實現,dump-flows顯示流表項
(一)ofctl調用流程
ofctl_dump_flows --調用-->
static void ofctl_dump_flows(struct ovs_cmdl_context *ctx) { ....struct ofputil_flow_stats_request fsr; enum ofputil_protocol protocol; struct vconn *vconn; vconn = prepare_dump_flows(ctx->argc, ctx->argv, false, &fsr, &protocol); struct ofputil_flow_stats *fses; size_t n_fses; run(vconn_dump_flows(vconn, &fsr, protocol, &fses, &n_fses), "dump flows"); if (n_criteria) { qsort(fses, n_fses, sizeof *fses, compare_flows); } struct ds s = DS_EMPTY_INITIALIZER; for (size_t i = 0; i < n_fses; i++) { ds_clear(&s); ofputil_flow_stats_format(&s, &fses[i], ports_to_show(ctx->argv[1]), tables_to_show(ctx->argv[1]), show_stats); printf(" %s\n", ds_cstr(&s)); //打印輸出結果字符串 } ds_destroy(&s); for (size_t i = 0; i < n_fses; i++) { free(CONST_CAST(struct ofpact *, fses[i].ofpacts)); } free(fses); vconn_close(vconn); }
分支一:
--調用--> vconn_dump_flows --調用--> ofputil_encode_flow_stats_request獲取協議版本對應的請求,然后根據請求去調用recv_flow_stats_reply --調用--> ofputil_decode_flow_stats_reply 獲取統一的ofputil_flow_stats結構體,而不是某一個版本的結構體,詳細參考二(三)中!!!
分支二:獲取分支一之后的狀態結構體之后,開始處理轉換為字符串!!!
--調用--> ofputil_flow_stats_format轉換ofputil_flow_stats結構體數據為字符串
(二)調用流程(二)handle_openflow,處理方法如(一)類似,甚至相同!!!
handle_openflow --調用--> handle_single_part_openflow --調用--> handle_flow_stats_request --調用-->
(1) ofputil_decode_flow_stats_request 獲取協議版本對應的請求,然后根據請求去調用recv_flow_stats_reply
(2) ofputil_append_flow_stats_reply 響應請求 獲取統一的ofputil_flow_stats結構體
(3) ofconn_send_replies ----> ofconn_send_reply ----> ofconn_send ----> rconn_send ----> rconn_send__ ----> copy_to_monitor ----> vconn_send ----> do_send ----> ofp_to_string會打印出所要顯示的字符串 ----> ofp_to_string__
----> ofp_print_flow_stats_reply ----> ofputil_decode_flow_stats_reply
(三)修改ofputil_flow_stats_format獲取要顯示的字符串
void
ofputil_flow_stats_format(struct ds *string,
const struct ofputil_flow_stats *fs,
const struct ofputil_port_map *port_map,
const struct ofputil_table_map *table_map,
bool show_stats)
{
if (show_stats || fs->cookie) {
ds_put_format(string, "%scookie=%s0x%"PRIx64", ",
colors.param, colors.end, ntohll(fs->cookie));
}
if (show_stats) {
ds_put_format(string, "%sduration=%s", colors.param, colors.end);
ofp_print_duration(string, fs->duration_sec, fs->duration_nsec);
ds_put_cstr(string, ", ");
}
if (show_stats || fs->table_id
|| ofputil_table_map_get_name(table_map, fs->table_id) != NULL) {
ds_put_format(string, "%stable=%s", colors.special, colors.end);
ofputil_format_table(fs->table_id, table_map, string);
ds_put_cstr(string, ", ");
}
if (show_stats) {
print_flow_stat(string, "n_packets", fs->packet_count);
print_flow_stat(string, "n_bytes", fs->byte_count);
}
if (fs->idle_timeout != OFP_FLOW_PERMANENT) {
ds_put_format(string, "%sidle_timeout=%s%"PRIu16", ",
colors.param, colors.end, fs->idle_timeout);
}
if (fs->hard_timeout != OFP_FLOW_PERMANENT) {
ds_put_format(string, "%shard_timeout=%s%"PRIu16", ",
colors.param, colors.end, fs->hard_timeout);
}
if (fs->effect_sec != 0) { ds_put_format(string, "%seffect_sec=%s%"PRIu64", ", colors.param, colors.end, fs->effect_sec); } if (fs->effect_usec != 0) { ds_put_format(string, "%seffect_usec=%s%"PRIu64", ", colors.param, colors.end, fs->effect_usec); }
if (fs->flags) {
ofputil_flow_mod_flags_format(string, fs->flags);
}
if (fs->importance != 0) {
ds_put_format(string, "%simportance=%s%"PRIu16", ",
colors.param, colors.end, fs->importance);
}
if (show_stats && fs->idle_age >= 0) {
ds_put_format(string, "%sidle_age=%s%d, ",
colors.param, colors.end, fs->idle_age);
}
if (show_stats && fs->hard_age >= 0 && fs->hard_age != fs->duration_sec) {
ds_put_format(string, "%shard_age=%s%d, ",
colors.param, colors.end, fs->hard_age);
}
/* Print the match, followed by a space (but omit the space if the match
* was an empty string). */
size_t length = string->length;
match_format(&fs->match, port_map, string, fs->priority);
if (string->length != length) {
ds_put_char(string, ' ');
}
ds_put_format(string, "%sactions=%s", colors.actions, colors.end);
struct ofpact_format_params fp = {
.port_map = port_map,
.table_map = table_map,
.s = string,
};
ofpacts_format(fs->ofpacts, fs->ofpacts_len, &fp);
}
(四)修改ofputil_decode_flow_stats_reply獲取字符串
int ofputil_decode_flow_stats_reply(struct ofputil_flow_stats *fs, //對比ofputil_decode_flow_mod函數,是將ofputil_flow_mod轉openflow struct ofpbuf *msg, bool flow_age_extension, struct ofpbuf *ofpacts) { const struct ofp_header *oh; size_t instructions_len; enum ofperr error; enum ofpraw raw; error = (msg->header ? ofpraw_decode(&raw, msg->header) : ofpraw_pull(&raw, msg)); if (error) { return error; } oh = msg->header; if (!msg->size) { return EOF; } else if (raw == OFPRAW_OFPST15_FLOW_REPLY) { VLOG_INFO("------------ofputil_decode_flow_stats_reply------OFPRAW_OFPST15_FLOW_REPLY------"); const struct ofp15_flow_desc *ofd; size_t length; uint16_t padded_match_len; uint16_t stat_len; uint8_t oxs_field_set; ofd = ofpbuf_try_pull(msg, sizeof *ofd); if (!ofd) { VLOG_WARN_RL(&rl, "OFPST_FLOW reply has %" PRIu32 " leftover " "bytes at end", msg->size); return EINVAL; } length = ntohs(ofd->length); if (length < sizeof *ofd) { VLOG_WARN_RL(&rl, "OFPST_FLOW reply claims invalid " "length %" PRIuSIZE, length); return EINVAL; } if (ofputil_pull_ofp11_match(msg, NULL, NULL, &fs->match, &padded_match_len)) { VLOG_WARN_RL(&rl, "OFPST_FLOW reply bad match"); return EINVAL; } fs->priority = ntohs(ofd->priority); fs->table_id = ofd->table_id; fs->cookie = ofd->cookie; fs->idle_timeout = ntohs(ofd->idle_timeout); fs->hard_timeout = ntohs(ofd->hard_timeout); fs->importance = ntohs(ofd->importance); error = ofputil_decode_flow_mod_flags(ofd->flags, -1, oh->version, &fs->flags); if (error) { return error; } struct oxs_stats oxs; if (oxs_pull_stat(msg, &oxs, &stat_len, &oxs_field_set)) { VLOG_WARN_RL(&rl, "OXS OFPST_FLOW reply bad stats"); return EINVAL; } fs->duration_sec = oxs.duration_sec; fs->duration_nsec = oxs.duration_nsec; fs->packet_count = oxs.packet_count; fs->byte_count = oxs.byte_count; fs->idle_age = oxs.idle_age == UINT32_MAX ? -1 : oxs.idle_age; fs->hard_age = -1; instructions_len = length - sizeof *ofd - padded_match_len - stat_len; } else if (raw == OFPRAW_OFPST11_FLOW_REPLY || raw == OFPRAW_OFPST13_FLOW_REPLY) { VLOG_INFO("------------ofputil_decode_flow_stats_reply------ofp11_flow_stats------"); const struct ofp11_flow_stats *ofs; size_t length; uint16_t padded_match_len; ofs = ofpbuf_try_pull(msg, sizeof *ofs); if (!ofs) { VLOG_WARN_RL(&rl, "OFPST_FLOW reply has %"PRIu32" leftover " "bytes at end", msg->size); return OFPERR_OFPBRC_BAD_LEN; } length = ntohs(ofs->length); if (length < sizeof *ofs) { VLOG_WARN_RL(&rl, "OFPST_FLOW reply claims invalid " "length %"PRIuSIZE, length); return OFPERR_OFPBRC_BAD_LEN; } error = ofputil_pull_ofp11_match(msg, NULL, NULL, &fs->match, &padded_match_len); if (error) { VLOG_WARN_RL(&rl, "OFPST_FLOW reply bad match"); return error; } instructions_len = length - sizeof *ofs - padded_match_len; fs->priority = ntohs(ofs->priority); fs->table_id = ofs->table_id; fs->duration_sec = ntohl(ofs->duration_sec); fs->duration_nsec = ntohl(ofs->duration_nsec); fs->idle_timeout = ntohs(ofs->idle_timeout); fs->hard_timeout = ntohs(ofs->hard_timeout); fs->effect_sec = ntohll(ofs->effect_sec); //處理------effect fs->effect_usec = ntohll(ofs->effect_usec); if (oh->version >= OFP14_VERSION) { fs->importance = ntohs(ofs->importance); } else { fs->importance = 0; } if (raw == OFPRAW_OFPST13_FLOW_REPLY) { error = ofputil_decode_flow_mod_flags(ofs->flags, -1, oh->version, &fs->flags); if (error) { return error; } } else { fs->flags = 0; } fs->idle_age = -1; fs->hard_age = -1; fs->cookie = ofs->cookie; fs->packet_count = ntohll(ofs->packet_count); fs->byte_count = ntohll(ofs->byte_count); } else if (raw == OFPRAW_OFPST10_FLOW_REPLY) { const struct ofp10_flow_stats *ofs; size_t length; VLOG_INFO("------------ofputil_decode_flow_stats_reply------ofp10_flow_stats------"); ofs = ofpbuf_try_pull(msg, sizeof *ofs); if (!ofs) { VLOG_WARN_RL(&rl, "OFPST_FLOW reply has %"PRIu32" leftover " "bytes at end", msg->size); return OFPERR_OFPBRC_BAD_LEN; } length = ntohs(ofs->length); if (length < sizeof *ofs) { VLOG_WARN_RL(&rl, "OFPST_FLOW reply claims invalid " "length %"PRIuSIZE, length); return OFPERR_OFPBRC_BAD_LEN; } instructions_len = length - sizeof *ofs; fs->cookie = get_32aligned_be64(&ofs->cookie); ofputil_match_from_ofp10_match(&ofs->match, &fs->match); fs->priority = ntohs(ofs->priority); fs->table_id = ofs->table_id; fs->duration_sec = ntohl(ofs->duration_sec); fs->duration_nsec = ntohl(ofs->duration_nsec); fs->idle_timeout = ntohs(ofs->idle_timeout); fs->hard_timeout = ntohs(ofs->hard_timeout); fs->effect_sec = ntohll(ofs->effect_sec); //處理------effect fs->effect_usec = ntohll(ofs->effect_usec); fs->importance = 0; fs->idle_age = -1; fs->hard_age = -1; fs->packet_count = ntohll(get_32aligned_be64(&ofs->packet_count)); fs->byte_count = ntohll(get_32aligned_be64(&ofs->byte_count)); fs->flags = 0; } else if (raw == OFPRAW_NXST_FLOW_REPLY) { const struct nx_flow_stats *nfs; size_t match_len, length; VLOG_INFO("------------ofputil_decode_flow_stats_reply------nx_flow_stats------"); nfs = ofpbuf_try_pull(msg, sizeof *nfs); if (!nfs) { VLOG_WARN_RL(&rl, "NXST_FLOW reply has %"PRIu32" leftover " "bytes at end", msg->size); return OFPERR_OFPBRC_BAD_LEN; } length = ntohs(nfs->length); match_len = ntohs(nfs->match_len); if (length < sizeof *nfs + ROUND_UP(match_len, 8)) { VLOG_WARN_RL(&rl, "NXST_FLOW reply with match_len=%"PRIuSIZE" " "claims invalid length %"PRIuSIZE, match_len, length); return OFPERR_OFPBRC_BAD_LEN; } error = nx_pull_match(msg, match_len, &fs->match, NULL, NULL, false, NULL, NULL); if (error) { return error; } instructions_len = length - sizeof *nfs - ROUND_UP(match_len, 8); fs->cookie = nfs->cookie; fs->table_id = nfs->table_id; fs->duration_sec = ntohl(nfs->duration_sec); fs->duration_nsec = ntohl(nfs->duration_nsec); fs->priority = ntohs(nfs->priority); fs->idle_timeout = ntohs(nfs->idle_timeout); fs->hard_timeout = ntohs(nfs->hard_timeout); fs->effect_sec = ntohl(nfs->effect_sec); //處理------effect fs->effect_usec = ntohl(nfs->effect_usec); VLOG_INFO("-----tt----ofputil_decode_flow_stats_reply----%d %d------------",fs->effect_sec,fs->effect_usec); fs->importance = 0; fs->idle_age = -1; fs->hard_age = -1; if (flow_age_extension) { if (nfs->idle_age) { fs->idle_age = ntohs(nfs->idle_age) - 1; } if (nfs->hard_age) { fs->hard_age = ntohs(nfs->hard_age) - 1; } } fs->packet_count = ntohll(nfs->packet_count); fs->byte_count = ntohll(nfs->byte_count); fs->flags = 0; } else { OVS_NOT_REACHED(); } error = ofpacts_pull_openflow_instructions(msg, instructions_len, oh->version, NULL, NULL, ofpacts); if (error) { VLOG_WARN_RL(&rl, "OFPST_FLOW reply bad instructions"); return error; } fs->ofpacts = ofpacts->data; fs->ofpacts_len = ofpacts->size; return 0; }
(五)修改ofputil_append_flow_stats_reply獲取字符串
void ofputil_append_flow_stats_reply(const struct ofputil_flow_stats *fs, //----------對應ofputil_decode_flow_stats_reply函數,相反賦值 struct ovs_list *replies, const struct tun_table *tun_table) { struct ofputil_flow_stats *fs_ = CONST_CAST(struct ofputil_flow_stats *, fs); const struct tun_table *orig_tun_table; struct ofpbuf *reply = ofpbuf_from_list(ovs_list_back(replies)); size_t start_ofs = reply->size; enum ofp_version version = ofpmp_version(replies); enum ofpraw raw = ofpmp_decode_raw(replies); orig_tun_table = fs->match.flow.tunnel.metadata.tab; fs_->match.flow.tunnel.metadata.tab = tun_table; VLOG_INFO("------------ofputil_append_flow_stats_reply-----start-----%d-----%d----",fs->effect_sec,fs->effect_usec); if (raw == OFPRAW_OFPST15_FLOW_REPLY) { VLOG_INFO("------------ofputil_append_flow_stats_reply-----OFPRAW_OFPST15_FLOW_REPLY----"); struct ofp15_flow_desc *ofd; ofpbuf_put_uninit(reply, sizeof *ofd); oxm_put_match(reply, &fs->match, version); struct oxs_stats oxs = { .duration_sec = fs->duration_sec, .duration_nsec = fs->duration_nsec, .idle_age = fs->idle_age >= 0 ? fs->idle_age : UINT32_MAX, .packet_count = fs->packet_count, .byte_count = fs->byte_count, .flow_count = UINT32_MAX, }; oxs_put_stats(reply, &oxs); ofpacts_put_openflow_instructions(fs->ofpacts, fs->ofpacts_len, reply, version); ofd = ofpbuf_at_assert(reply, start_ofs, sizeof *ofd); ofd->length = htons(reply->size - start_ofs); ofd->table_id = fs->table_id; ofd->priority = htons(fs->priority); ofd->idle_timeout = htons(fs->idle_timeout); ofd->hard_timeout = htons(fs->hard_timeout); ofd->cookie = fs->cookie; memset(ofd->pad2, 0, sizeof ofd->pad2); ofd->pad = 0; ofd->importance = htons(fs->importance); ofd->flags = ofputil_encode_flow_mod_flags(fs->flags, version); } else if (raw == OFPRAW_OFPST11_FLOW_REPLY || raw == OFPRAW_OFPST13_FLOW_REPLY) { //處理------effect struct ofp11_flow_stats *ofs; VLOG_INFO("------------ofputil_append_flow_stats_reply-----OFPRAW_OFPST15_FLOW_REPLY----"); ofpbuf_put_uninit(reply, sizeof *ofs); oxm_put_match(reply, &fs->match, version); ofpacts_put_openflow_instructions(fs->ofpacts, fs->ofpacts_len, reply, version); ofs = ofpbuf_at_assert(reply, start_ofs, sizeof *ofs); ofs->length = htons(reply->size - start_ofs); ofs->table_id = fs->table_id; ofs->pad = 0; ofs->duration_sec = htonl(fs->duration_sec); ofs->duration_nsec = htonl(fs->duration_nsec); ofs->priority = htons(fs->priority); ofs->idle_timeout = htons(fs->idle_timeout); ofs->hard_timeout = htons(fs->hard_timeout); ofs->effect_sec = htonll(fs->effect_sec); //處理------effect ofs->effect_usec = htonll(fs->effect_usec); if (version >= OFP14_VERSION) { ofs->importance = htons(fs->importance); } else { ofs->importance = 0; } if (raw == OFPRAW_OFPST13_FLOW_REPLY) { ofs->flags = ofputil_encode_flow_mod_flags(fs->flags, version); } else { ofs->flags = 0; } memset(ofs->pad2, 0, sizeof ofs->pad2); ofs->cookie = fs->cookie; ofs->packet_count = htonll(unknown_to_zero(fs->packet_count)); ofs->byte_count = htonll(unknown_to_zero(fs->byte_count)); } else if (raw == OFPRAW_OFPST10_FLOW_REPLY) { struct ofp10_flow_stats *ofs; VLOG_INFO("------------ofputil_append_flow_stats_reply-----OFPRAW_OFPST10_FLOW_REPLY----"); ofpbuf_put_uninit(reply, sizeof *ofs); ofpacts_put_openflow_actions(fs->ofpacts, fs->ofpacts_len, reply, version); ofs = ofpbuf_at_assert(reply, start_ofs, sizeof *ofs); ofs->length = htons(reply->size - start_ofs); ofs->table_id = fs->table_id; ofs->pad = 0; ofputil_match_to_ofp10_match(&fs->match, &ofs->match); ofs->duration_sec = htonl(fs->duration_sec); ofs->duration_nsec = htonl(fs->duration_nsec); ofs->priority = htons(fs->priority); ofs->idle_timeout = htons(fs->idle_timeout); ofs->hard_timeout = htons(fs->hard_timeout); ofs->effect_sec = htonll(fs->effect_sec); ofs->effect_usec = htonll(fs->effect_usec); memset(ofs->pad2, 0, sizeof ofs->pad2); put_32aligned_be64(&ofs->cookie, fs->cookie); put_32aligned_be64(&ofs->packet_count, htonll(unknown_to_zero(fs->packet_count))); put_32aligned_be64(&ofs->byte_count, htonll(unknown_to_zero(fs->byte_count))); } else if (raw == OFPRAW_NXST_FLOW_REPLY) { struct nx_flow_stats *nfs; int match_len; VLOG_INFO("------------ofputil_append_flow_stats_reply-----OFPRAW_NXST_FLOW_REPLY----"); ofpbuf_put_uninit(reply, sizeof *nfs); match_len = nx_put_match(reply, &fs->match, 0, 0); ofpacts_put_openflow_actions(fs->ofpacts, fs->ofpacts_len, reply, version); nfs = ofpbuf_at_assert(reply, start_ofs, sizeof *nfs); nfs->length = htons(reply->size - start_ofs); nfs->table_id = fs->table_id; nfs->pad = 0; nfs->duration_sec = htonl(fs->duration_sec); nfs->duration_nsec = htonl(fs->duration_nsec); nfs->priority = htons(fs->priority); nfs->idle_timeout = htons(fs->idle_timeout); nfs->hard_timeout = htons(fs->hard_timeout); nfs->effect_sec = htonll(fs->effect_sec); nfs->effect_usec = htonll(fs->effect_usec); nfs->idle_age = htons(fs->idle_age < 0 ? 0 : fs->idle_age < UINT16_MAX ? fs->idle_age + 1 : UINT16_MAX); nfs->hard_age = htons(fs->hard_age < 0 ? 0 : fs->hard_age < UINT16_MAX ? fs->hard_age + 1 : UINT16_MAX); nfs->match_len = htons(match_len); nfs->cookie = fs->cookie; nfs->packet_count = htonll(fs->packet_count); nfs->byte_count = htonll(fs->byte_count); } else { VLOG_INFO("------------ofputil_append_flow_stats_reply-----OVS_NOT_REACHED----"); OVS_NOT_REACHED(); } ofpmp_postappend(replies, start_ofs); fs_->match.flow.tunnel.metadata.tab = orig_tun_table; }
六:ovs-vswitchd的啟動分析(見鏈接),是七的鋪墊
其中run方法后面會用到的!!!
七:數據面處理來自控制面(由四轉發而來)的openflow協議(重點)
(一)調用流程
Ovs-vswitchd會監聽socket,在ovs-vswitchd.c中
bridge_run --調用--> bridge_run__ --調用--> ofproto_run --調用--> connmgr_run(p->connmgr, handle_openflow); --調用--> ofconn_run(ofconn, handle_openflow) --調用--> handle_openflow(ofconn, &msgs);
當交換機有openflow調用的時候,handle_openflow會被調用去處理連接!!!!
handle_openflow--調用-->handle_flow_mod
分支(一):
--調用--> ofputil_decode_flow_mod解析各個版本的openflow協議到ofputil_flow_mod結構體中(包括基本字段、條件字段、action)
分支(二):調用分支一之后,所有信息都解析完成,存放在ofputil_flow_mod fm中,下面開始通過調用分支二下發到數據平面(datapath)
--調用--> handle_flow_mod__ --調用-->
(1) ofproto_flow_mod_init將規則初始化賦值給了ofm中的temp_rule --調用--> add_flow_init --調用--> ofproto_rule_create創建並初始化規則 --回調--> rule_construct初始化struct rule_dpif(內部保護規則rule)
(2) ofproto_flow_mod_start --調用--> add_flow_start 開始添加流表項 --調用--> replace_rule_start 開始替換規則
(3) ofproto_flow_mod_finish --調用--> add_flow_finish --調用--> replace_rule_finish 替換規則完成
(4) ofmonitor_flush 通知底層內核更新流表項
(二)修改ofproto_rule_create創建並初始化規則,為規則添加我們自定義的字段
1.handle_flow_mod__方法,保存后面需要的openflow_mod_requester請求數據到ofputil_flow_mod* fm中自定義字段中去
static enum ofperr handle_flow_mod__(struct ofproto *ofproto, const struct ofputil_flow_mod *fm, //協議公共數據結構,前面handle_flow_mod方法就是為了獲取他 const struct openflow_mod_requester *req) OVS_EXCLUDED(ofproto_mutex) //聲明鎖 { struct ofproto_flow_mod ofm; //具有執行上下文的流模式 enum ofperr error; error = ofproto_flow_mod_init(ofproto, &ofm, fm, NULL); //將規則賦值給了ofm中的temp_rules if (error) { return error; } ovs_mutex_lock(&ofproto_mutex); //加鎖----------加解鎖中間才是重要的東西 ofm.version = ofproto->tables_version + 1; ofm.omr = req; //將規則ofm中的temp_rules中,舊的放入old_rules;新的放入new_rules;中去 error = ofproto_flow_mod_start(ofproto, &ofm); //處理規則???----將規則加入ovs list if (!error) { VLOG_INFO("--------------------note ovs datapath----------start----"); ofproto_bump_tables_version(ofproto); //判斷版本,不用管 ofproto_flow_mod_finish(ofproto, &ofm, req); //這里處理了 //下面才是重點----------- ofmonitor_flush(ofproto->connmgr); //ofproto->connmgr是OpenFlow交換機的連接管理器 VLOG_INFO("--------------------note ovs datapath------------end--"); } ovs_mutex_unlock(&ofproto_mutex); //解鎖 return error; }
1.add_flow_init方法,fm參數中已經在分支一中賦值了
static enum ofperr add_flow_init(struct ofproto *ofproto, struct ofproto_flow_mod *ofm, const struct ofputil_flow_mod *fm) //ofputil_flow_mod中還是包含了openflow協議的所有信息的 OVS_EXCLUDED(ofproto_mutex) { struct oftable *table; struct cls_rule cr; uint8_t table_id; enum ofperr error; if (!check_table_id(ofproto, fm->table_id)) { return OFPERR_OFPBRC_BAD_TABLE_ID; } /* Pick table. */ //選擇表id if (fm->table_id == 0xff) { if (ofproto->ofproto_class->rule_choose_table) { error = ofproto->ofproto_class->rule_choose_table(ofproto, &fm->match, &table_id); if (error) { return error; } ovs_assert(table_id < ofproto->n_tables); } else { table_id = 0; } } else if (fm->table_id < ofproto->n_tables) { table_id = fm->table_id; } else { return OFPERR_OFPBRC_BAD_TABLE_ID; } table = &ofproto->tables[table_id]; //------------選擇對應的openflow表 if (table->flags & OFTABLE_READONLY && !(fm->flags & OFPUTIL_FF_NO_READONLY)) { //只讀表,不允許添加,返回錯誤 return OFPERR_OFPBRC_EPERM; } if (!(fm->flags & OFPUTIL_FF_HIDDEN_FIELDS) && !minimatch_has_default_hidden_fields(&fm->match)) { VLOG_WARN_RL(&rl, "%s: (add_flow) only internal flows can set " "non-default values to hidden fields", ofproto->name); return OFPERR_OFPBRC_EPERM; } if (!ofm->temp_rule) { //如果上下文中沒有臨時規則(新添加),則下面進行初始化 cls_rule_init_from_minimatch(&cr, &fm->match, fm->priority); //初始化對象,增加match到rule中 /* Allocate new rule. Destroys 'cr'. */ //分配新規則 //返回指示“flow”中存在哪些隧道元數據字段的位圖 uint64_t map = miniflow_get_tun_metadata_present_map(fm->match.flow); error = ofproto_rule_create(ofproto, &cr, table - ofproto->tables, //-----重點,將新規則結合到ofproto中去 fm->new_cookie, fm->idle_timeout, //回調rule_construct()初始化rule_dpif的參數 fm->hard_timeout, fm->effect_sec, fm->effect_usec,fm->flags, fm->importance, fm->ofpacts, fm->ofpacts_len, map, fm->ofpacts_tlv_bitmap, &ofm->temp_rule); //所有規則數據返回到&ofm->temp_rule中!!!-------重點!!! if (error) { return error; } get_conjunctions(fm, &ofm->conjs, &ofm->n_conjs); //重點---------------????? } return 0; }
2.ofproto_rule_create方法
static enum ofperr ofproto_rule_create(struct ofproto *ofproto, struct cls_rule *cr, uint8_t table_id, ovs_be64 new_cookie, uint16_t idle_timeout, uint16_t hard_timeout, uint64_t effect_sec,uint64_t effect_usec, enum ofputil_flow_mod_flags flags, uint16_t importance, const struct ofpact *ofpacts, size_t ofpacts_len, uint64_t match_tlv_bitmap, uint64_t ofpacts_tlv_bitmap, struct rule **new_rule) OVS_NO_THREAD_SAFETY_ANALYSIS { struct rule *rule; enum ofperr error; /* Allocate new rule. */ rule = ofproto->ofproto_class->rule_alloc(); //回調函數rule_alloc()分配rule_dpif和rule對象 if (!rule) { cls_rule_destroy(cr); VLOG_WARN_RL(&rl, "%s: failed to allocate a rule.", ofproto->name); return OFPERR_OFPFMFC_UNKNOWN; } /* Initialize base state. */ //接下來初始化rule對象中的相關參數…… *CONST_CAST(struct ofproto **, &rule->ofproto) = ofproto; cls_rule_move(CONST_CAST(struct cls_rule *, &rule->cr), cr); ovs_refcount_init(&rule->ref_count); ovs_mutex_init(&rule->mutex); ovs_mutex_lock(&rule->mutex); *CONST_CAST(ovs_be64 *, &rule->flow_cookie) = new_cookie; rule->created = rule->modified = time_msec(); rule->idle_timeout = idle_timeout; rule->hard_timeout = hard_timeout; rule->effect_sec = effect_sec; rule->effect_usec = effect_usec; *CONST_CAST(uint16_t *, &rule->importance) = importance; rule->removed_reason = OVS_OFPRR_NONE; *CONST_CAST(uint8_t *, &rule->table_id) = table_id; rule->flags = flags & OFPUTIL_FF_STATE; *CONST_CAST(const struct rule_actions **, &rule->actions) = rule_actions_create(ofpacts, ofpacts_len); //以上規則初始 //下面處理ovs_list ovs_list_init(&rule->meter_list_node); rule->eviction_group = NULL; rule->monitor_flags = 0; rule->add_seqno = 0; rule->modify_seqno = 0; ovs_list_init(&rule->expirable); ovs_list_init(&rule->effectable); //------修改effectable---------- ovs_mutex_unlock(&rule->mutex); /* Construct rule, initializing derived state. */ error = ofproto->ofproto_class->rule_construct(rule); //回調rule_construct()初始化rule_dpif的參數 if (error) { ofproto_rule_destroy__(rule); return error; } rule->state = RULE_INITIALIZED; rule->match_tlv_bitmap = match_tlv_bitmap; rule->ofpacts_tlv_bitmap = ofpacts_tlv_bitmap; mf_vl_mff_ref(&rule->ofproto->vl_mff_map, match_tlv_bitmap); mf_vl_mff_ref(&rule->ofproto->vl_mff_map, ofpacts_tlv_bitmap); *new_rule = rule; return 0; }
其中的rule->effectable鏈中存放的是所有還沒有生效的規則,會在后面run方法中遍歷和檢測,使之生效!!!
(三)修改replace_rule_start,開始替換規則
1.add_flow_start
/* ofm->temp_rule is consumed only in the successful case. */ //開始使用規則了!!! static enum ofperr add_flow_start(struct ofproto *ofproto, struct ofproto_flow_mod *ofm) OVS_REQUIRES(ofproto_mutex) { struct rule *old_rule = NULL; struct rule *new_rule = ofm->temp_rule; //------------使用 const struct rule_actions *actions = rule_get_actions(new_rule); //獲取規則actions,返回一個rule_actions指針 struct oftable *table = &ofproto->tables[new_rule->table_id]; //獲取對應的table enum ofperr error; /* Must check actions while holding ofproto_mutex to avoid a race. */ error = ofproto_check_ofpacts(ofproto, actions->ofpacts, //檢查鎖,避免競爭 actions->ofpacts_len); if (error) { return error; } /* Check for the existence of an identical rule. * This will not return rules earlier marked for removal. */ //檢查是否存在相同的規則(舊的) old_rule = rule_from_cls_rule(classifier_find_rule_exactly(&table->cls, &new_rule->cr, ofm->version)); if (!old_rule) { /* Check for overlap, if requested. 檢查重疊*/ if (new_rule->flags & OFPUTIL_FF_CHECK_OVERLAP && classifier_rule_overlaps(&table->cls, &new_rule->cr, ofm->version)) { return OFPERR_OFPFMFC_OVERLAP; } /* If necessary, evict an existing rule to clear out space. */ //如果流表項太多,就刪除表中的部分流表項 if (table->n_flows >= table->max_flows) { if (!choose_rule_to_evict(table, &old_rule)) { return OFPERR_OFPFMFC_TABLE_FULL; } eviction_group_remove_rule(old_rule); /* Marks 'old_rule' as an evicted rule rather than replaced rule. */ old_rule->removed_reason = OFPRR_EVICTION; } } else { ofm->modify_cookie = true; } if (old_rule) { //----------這里檢測出來是否有舊的流表項了 rule_collection_add(&ofm->old_rules, old_rule); }/* Take ownership of the temp_rule. */ //將去重處理后的新new_rule放入ofm->new_rules,並且清空ofm->temp_rule-------------重點!!!! rule_collection_add(&ofm->new_rules, new_rule); ofm->temp_rule = NULL; //開始真正插入操作!!!-----重點---------------開始將規則 /* replace_rule_start主要操作就是新的流表替換舊的流表的操作,如果存在舊流表,則調用ofproto_rule_remove__刪除, 然后調用ofproto_rule_insert__和classifier_insert添加流表。其中classifier_insert主要是將rule->cr添加到table->cls中 */ replace_rule_start(ofproto, ofm, old_rule, new_rule); return 0; }
2.replace_rule_start 處理包含自定義字段的流表項,放入effectable鏈中,不直接插入datapath。
static void replace_rule_start(struct ofproto *ofproto, struct ofproto_flow_mod *ofm, //---------插入cls struct rule *old_rule, struct rule *new_rule) { struct oftable *table = &ofproto->tables[new_rule->table_id]; //獲取表 /* 'old_rule' may be either an evicted rule or replaced rule. */ //“舊規則”可以是逐出規則或替換規則。 if (old_rule) {/* Copy values from old rule for modify semantics. */ if (old_rule->removed_reason != OFPRR_EVICTION) { //非驅逐--替換(用舊的替換新的部分信息) bool change_cookie = (ofm->modify_cookie && new_rule->flow_cookie != OVS_BE64_MAX && new_rule->flow_cookie != old_rule->flow_cookie); ovs_mutex_lock(&new_rule->mutex); ovs_mutex_lock(&old_rule->mutex); if (ofm->command != OFPFC_ADD) { new_rule->idle_timeout = old_rule->idle_timeout; new_rule->hard_timeout = old_rule->hard_timeout; *CONST_CAST(uint16_t *, &new_rule->importance) = old_rule->importance; new_rule->flags = old_rule->flags; new_rule->created = old_rule->created; } if (!change_cookie) { *CONST_CAST(ovs_be64 *, &new_rule->flow_cookie) = old_rule->flow_cookie; } ovs_mutex_unlock(&old_rule->mutex); ovs_mutex_unlock(&new_rule->mutex); } /* Remove the old rule from data structures. */ if(new_rule->effect_sec || new_rule->effect_usec){//把ofm中的old_rule剔除---------- rule_collection_remove(&ofm->old_rules, old_rule); }else{/* Mark the old rule for removal in the next version. */ //標記舊規則,以便后面刪除 cls_rule_make_invisible_in_version(&old_rule->cr, ofm->version); ofproto_rule_remove__(ofproto, old_rule); } } else { table->n_flows++; } /* Insert flow to ofproto data structures, so that later flow_mods may * relate to it. This is reversible, in case later errors require this to * be reverted. */ //將流插入到原始數據結構中,以便以后的流與之相關。這是可逆的,以防以后的錯誤需要恢復。 //先判斷是否應該插入,effectable if(new_rule->effect_sec || new_rule->effect_usec){ if(new_rule->state == RULE_EFFECTED || new_rule->state == RULE_INSERTED) return; struct ofproto_flow_mod* tofm = (struct ofproto_flow_mod*)xmalloc(sizeof(*ofm)); //用來一直存放 memcpy(tofm,ofm,sizeof(*ofm)); tofm->omr = (struct openflow_mod_requester*)xmalloc(sizeof(struct openflow_mod_requester)); memcpy(tofm->omr,ofm->omr,sizeof(struct openflow_mod_requester)); tofm->conjs = (struct cls_conjunction *)xmalloc(ofm->n_conjs*sizeof(struct cls_conjunction)); memcpy(tofm->conjs,ofm->conjs,ofm->n_conjs*sizeof(struct cls_conjunction)); rule_collection_add(&tofm->old_rules, old_rule); new_rule->ofm = tofm;
ofproto_rule_effect__(ofproto,new_rule); //進行插入操作 return; } ofproto_rule_insert__(ofproto, new_rule); /* Make the new rule visible for classifier lookups only from the next * version. */ classifier_insert(&table->cls, &new_rule->cr, ofm->version, ofm->conjs, ofm->n_conjs); }
分析代碼1:
//先判斷是否應該插入,effectable if(new_rule->effect_sec || new_rule->effect_usec){ //如果我們設置了自定義字段的值的話!!!,開始將這條規則通過ofproto_rule_effect__方法插入effectable鏈中,並且改變該規則的狀態 if(new_rule->state == RULE_EFFECTED || new_rule->state == RULE_INSERTED) return; struct ofproto_flow_mod* tofm = (struct ofproto_flow_mod*)xmalloc(sizeof(*ofm)); //我們需要將ofproto_flow_mod ofm存放到規則中的自定義字段中,因為規則生效時,進行插入datapath過程中需要使用到這些數據 memcpy(tofm,ofm,sizeof(*ofm)); //ofm在開始時是存放在棧中,不是堆中,生存周期沒有辦法達到我們需要的長度,所以我們變為堆中存放,並且把需要的數據拷貝過來 tofm->omr = (struct openflow_mod_requester*)xmalloc(sizeof(struct openflow_mod_requester)); //把請求數據一塊保存,后面插入也是需要這個數據的 memcpy(tofm->omr,ofm->omr,sizeof(struct openflow_mod_requester)); //ofm->omr在handle_flow_mod__方法中被賦值了,所以我們這里可以直接獲取openflow_mod_requester的數據 tofm->conjs = (struct cls_conjunction *)xmalloc(ofm->n_conjs*sizeof(struct cls_conjunction)); memcpy(tofm->conjs,ofm->conjs,ofm->n_conjs*sizeof(struct cls_conjunction)); rule_collection_add(&tofm->old_rules, old_rule); //把舊流表項添加到鏈中,當新的流表項生效時,再去把舊的流表項移除!!!!,注意我們后面分析的代碼2 new_rule->ofm = tofm; ofproto_rule_effect__(ofproto,new_rule); //調用自定義方法ofproto_rule_effect__進行插入操作,插入自定義的effectable鏈中去,並且修改流表規則的狀態 return; }
分析代碼2:
if(new_rule->effect_sec || new_rule->effect_usec){//把ofm中的old_rule剔除----------原本old_rule中是存在這個舊的規則,但是我們因為新規則還沒有生效,所以不能把舊規則刪除了,所以我們把這個規則先移除來,不被刪除掉 rule_collection_remove(&ofm->old_rules, old_rule); //如果effect字段被使用了,我們后面會添加到effectable鏈中單獨處理,對於當前正在運行的舊的流表項(相互沖突的),我們在新流表項生效之前,始終運行!!!,所以我們不能去移除他 }else{ //對於沒有使用自定義字段的規則,如果存在沖突的舊規則,則直接移除 //標記舊規則,以便后面刪除 cls_rule_make_invisible_in_version(&old_rule->cr, ofm->version); ofproto_rule_remove__(ofproto, old_rule); }
3.自定義方法ofproto_rule_effect__,添加規則到鏈表中去
static void ofproto_rule_effect__(struct ofproto *ofproto, struct rule *rule) OVS_REQUIRES(ofproto_mutex) { /* A rule may not be reinserted. */ ovs_assert(rule->state != RULE_EFFECTED); //生效系列操作 if(rule->effect_sec || rule->effect_usec){ VLOG_INFO("------!!!---------ofproto_rule_effect__-------effectable----start---%d-----%d",rule->ofm->version,rule->ofm->n_conjs); ovs_list_insert(&ofproto->effectable, &rule->effectable); //effectable入鏈 } rule->state = RULE_EFFECTED; //規則已經插入 }
(四)分析ofproto_flow_mod_finish方法(未修改)
1.add_flow_finish方法
static void add_flow_finish(struct ofproto *ofproto, struct ofproto_flow_mod *ofm, const struct openflow_mod_requester *req) OVS_REQUIRES(ofproto_mutex) {struct rule *old_rule = rule_collection_n(&ofm->old_rules) ? rule_collection_rules(&ofm->old_rules)[0] : NULL;struct rule *new_rule = rule_collection_rules(&ofm->new_rules)[0];struct ovs_list dead_cookies = OVS_LIST_INITIALIZER(&dead_cookies);//下面代替才是重點-----------------!!! replace_rule_finish(ofproto, ofm, req, old_rule, new_rule, &dead_cookies); learned_cookies_flush(ofproto, &dead_cookies);if (old_rule) { ovsrcu_postpone(remove_rule_rcu, old_rule); } else { ofmonitor_report(ofproto->connmgr, new_rule, NXFME_ADDED, 0, req ? req->ofconn : NULL, req ? req->request->xid : 0, NULL);/* Send Vacancy Events for OF1.4+. */ send_table_status(ofproto, new_rule->table_id); } }
2.replace_rule_finish方法
/* Adds the 'new_rule', replacing the 'old_rule'. */ static void replace_rule_finish(struct ofproto *ofproto, struct ofproto_flow_mod *ofm, const struct openflow_mod_requester *req, struct rule *old_rule, struct rule *new_rule, struct ovs_list *dead_cookies) OVS_REQUIRES(ofproto_mutex) { struct rule *replaced_rule; replaced_rule = (old_rule && old_rule->removed_reason != OFPRR_EVICTION) ? old_rule : NULL; ofproto->ofproto_class->rule_insert(new_rule, replaced_rule, //流表變化標識設置 ofm->modify_keep_counts); learned_cookies_inc(ofproto, rule_get_actions(new_rule));if (old_rule) {const struct rule_actions *old_actions = rule_get_actions(old_rule); const struct rule_actions *new_actions = rule_get_actions(new_rule); learned_cookies_dec(ofproto, old_actions, dead_cookies);if (replaced_rule) { enum nx_flow_update_event event = ofm->command == OFPFC_ADD ? NXFME_ADDED : NXFME_MODIFIED; bool changed_cookie = (new_rule->flow_cookie != old_rule->flow_cookie); bool changed_actions = !ofpacts_equal(new_actions->ofpacts, new_actions->ofpacts_len, old_actions->ofpacts, old_actions->ofpacts_len);if (event != NXFME_MODIFIED || changed_actions || changed_cookie) { //-------------------填充ofconn->updates用於消息發送 ofmonitor_report(ofproto->connmgr, new_rule, event, 0, req ? req->ofconn : NULL, req ? req->request->xid : 0, changed_actions ? old_actions : NULL); } } else { /* XXX: This is slight duplication with delete_flows_finish__() */ ofmonitor_report(ofproto->connmgr, old_rule, NXFME_DELETED, OFPRR_EVICTION, req ? req->ofconn : NULL, req ? req->request->xid : 0, NULL); } } }
(五)修改run方法,位於main的while循環中,循環處理effectable流表鏈!!!
在文件openvswitch-2.11.4/ofproto/ofproto-dpif.c中:
1.修改run方法,處理達到生效時間的流表規則,使之生效
#include <time.h>
static int run(struct ofproto *ofproto_) { struct ofproto_dpif *ofproto = ofproto_dpif_cast(ofproto_); uint64_t new_seq, new_dump_seq; struct timeval tv; //---------------獲取當前時間--------------- if (mbridge_need_revalidate(ofproto->mbridge)) { ofproto->backer->need_revalidate = REV_RECONFIGURE; ovs_rwlock_wrlock(&ofproto->ml->rwlock); mac_learning_flush(ofproto->ml); ovs_rwlock_unlock(&ofproto->ml->rwlock); mcast_snooping_mdb_flush(ofproto->ms); } /* Always updates the ofproto->ams_seqno to avoid frequent wakeup during * flow restore. Even though nothing is processed during flow restore, * all queued 'ams' will be handled immediately when flow restore * completes. */ ofproto->ams_seqno = seq_read(ofproto->ams_seq); /* Do not perform any periodic activity required by 'ofproto' while * waiting for flow restore to complete. */ if (!ofproto_get_flow_restore_wait()) { struct ofproto_async_msg *am; struct ovs_list ams; guarded_list_pop_all(&ofproto->ams, &ams); LIST_FOR_EACH_POP (am, list_node, &ams) { connmgr_send_async_msg(ofproto->up.connmgr, am); ofproto_async_msg_free(am); } } if (ofproto->netflow) { netflow_run(ofproto->netflow); } if (ofproto->sflow) { dpif_sflow_run(ofproto->sflow); } if (ofproto->ipfix) { dpif_ipfix_run (ofproto->ipfix); } new_seq = seq_read(connectivity_seq_get()); if (ofproto->change_seq != new_seq) { struct ofport_dpif *ofport; HMAP_FOR_EACH (ofport, up.hmap_node, &ofproto->up.ports) { port_run(ofport); } ofproto->change_seq = new_seq; } if (ofproto->lacp_enabled || ofproto->has_bonded_bundles) { struct ofbundle *bundle; HMAP_FOR_EACH (bundle, hmap_node, &ofproto->bundles) { bundle_run(bundle); } } stp_run(ofproto); rstp_run(ofproto); ovs_rwlock_wrlock(&ofproto->ml->rwlock); if (mac_learning_run(ofproto->ml)) { ofproto->backer->need_revalidate = REV_MAC_LEARNING; } ovs_rwlock_unlock(&ofproto->ml->rwlock); if (mcast_snooping_run(ofproto->ms)) { ofproto->backer->need_revalidate = REV_MCAST_SNOOPING; } new_dump_seq = seq_read(udpif_dump_seq(ofproto->backer->udpif)); if (ofproto->dump_seq != new_dump_seq) { struct rule *rule, *next_rule; ovs_mutex_lock(&ofproto_mutex); LIST_FOR_EACH_SAFE (rule, next_rule, effectable, &ofproto->up.effectable) { //對於小於effect生效時間的,不進行處理 if(rule->effect_sec!=0 || rule->effect_usec!=0){ gettimeofday(&tv, NULL); //獲取當前時間 if(!((tv.tv_sec < rule->effect_sec)||(tv.tv_sec == rule->effect_sec && tv.tv_usec < rule->effect_usec))){ //先對比秒,當前時刻小於生效時間,則return VLOG_INFO("-----------run--local_time--%d--%d->>>>>>-effect_time--%d--%d--",tv.tv_sec,tv.tv_usec,rule->effect_sec,rule->effect_usec); rule->effect_sec = 0; //歸0 rule->effect_usec = 0; replace_rule_effect(&ofproto->up,rule); //進行插入 ------- 沒有主動去調用handle_flow_mod函數,導致更新不及時!!!,不能進行處理 ovs_list_remove(&rule->effectable); //出鏈操作 } } } ovs_mutex_unlock(&ofproto_mutex); } if (ofproto->dump_seq != new_dump_seq) { //這里是用來遍歷expirable鏈表,用來檢測hard_timeout和idle_timeout是否到期的;effectable鏈就是模仿他實現的 struct rule *rule, *next_rule; long long now = time_msec(); /* We know stats are relatively fresh, so now is a good time to do some * periodic work. */ ofproto->dump_seq = new_dump_seq; /* Expire OpenFlow flows whose idle_timeout or hard_timeout * has passed. */ /* #define LIST_FOR_EACH_SAFE(ITER, NEXT, MEMBER, LIST) \ for (INIT_CONTAINER(ITER, (LIST)->next, MEMBER); \ (&(ITER)->MEMBER != (LIST) \ ? INIT_CONTAINER(NEXT, (ITER)->MEMBER.next, MEMBER), 1 \ : 0); \ (ITER) = (NEXT)) */ ovs_mutex_lock(&ofproto_mutex); LIST_FOR_EACH_SAFE (rule, next_rule, expirable, &ofproto->up.expirable) { rule_expire(rule_dpif_cast(rule), now); //-------這里校驗是否超時 } ovs_mutex_unlock(&ofproto_mutex); /* All outstanding data in existing flows has been accounted, so it's a * good time to do bond rebalancing. */ if (ofproto->has_bonded_bundles) { struct ofbundle *bundle; HMAP_FOR_EACH (bundle, hmap_node, &ofproto->bundles) { if (bundle->bond) { bond_rebalance(bundle->bond); } } } } return 0; }
代碼分析1:
if (ofproto->dump_seq != new_dump_seq) { struct rule *rule, *next_rule; ovs_mutex_lock(&ofproto_mutex); LIST_FOR_EACH_SAFE (rule, next_rule, effectable, &ofproto->up.effectable) { //遍歷交換機的effectable鏈表 //對於小於effect生效時間的,不進行處理 if(rule->effect_sec!=0 || rule->effect_usec!=0){ //其實這里的規則的effect_sec或者effect_usec都是有值的 gettimeofday(&tv, NULL); //獲取當前時間 if(!((tv.tv_sec < rule->effect_sec)||(tv.tv_sec == rule->effect_sec && tv.tv_usec < rule->effect_usec))){ //先對比秒和微秒,當前時刻大於生效時間,則當前流表項可以開始工作了,可以插入交換機進行生效處理!!!! rule->effect_sec = 0; //歸0,對於處理后的我們把值置為0,其實這里也可以不置為0,置為0之后后面不會在dump-flows中顯示設置的生效時間了 rule->effect_usec = 0; replace_rule_effect(&ofproto->up,rule); //調用自定義replace_rule_effect方法進行插入,在內部主動去調用ofmonitor_flush函數,使得datapath及時更新!!!否則的話,可能更新不及時,不能進行按照新的流表項處理 ovs_list_remove(&rule->effectable); //出鏈操作,后面就不會再重復處理了!!! } } } ovs_mutex_unlock(&ofproto_mutex); }
2.調用自定義方法replace_rule_effect,使得流表添加,並且被datapath及時處理(下面全部添加)
void replace_rule_effect(struct ofproto *ofproto,struct rule *new_rule){ ovs_assert(new_rule->state != RULE_INSERTED); //判斷是否生效---是否是沒有添加過的流表項const struct rule_actions *actions = rule_get_actions(new_rule); //獲取規則actions,返回一個rule_actions指針 rule結構體中有action
struct oftable *table = &ofproto->tables[new_rule->table_id]; //獲取表
struct ofproto_flow_mod* ofm = new_rule->ofm; //獲取對應規則的ofproto_flow_mod數據 struct rule* old_rule = NULL; enum ofperr error; /* Check for the existence of an identical rule. * This will not return rules earlier marked for removal. */
//檢查是否存在相同的規則(舊的)
/* Must check actions while holding ofproto_mutex to avoid a race. */ error = ofproto_check_ofpacts(ofproto, actions->ofpacts, //檢查鎖,避免競爭-----------------出錯!!!!
actions->ofpacts_len); if (error) {return ; } old_rule = rule_from_cls_rule(classifier_find_rule_exactly(&table->cls, //去流表中查找舊的規則(於新規則有沖突的表項) &new_rule->cr, ofm->version)); if (!old_rule) {/* Check for overlap, if requested. 檢查重疊*/
if (new_rule->flags & OFPUTIL_FF_CHECK_OVERLAP && classifier_rule_overlaps(&table->cls, &new_rule->cr, ofm->version)) {return ; } /* If necessary, evict an existing rule to clear out space. */
//如果流表項太多,就刪除表中的部分流表項
if (table->n_flows >= table->max_flows) { if (!choose_rule_to_evict(table, &old_rule)) {return ; } eviction_group_remove_rule(old_rule); /* Marks 'old_rule' as an evicted rule rather than replaced rule. */ old_rule->removed_reason = OFPRR_EVICTION; } } else { ofm->modify_cookie = true; } if (old_rule) { //如果存在舊規則,就放入ofm的old_rules鏈中 rule_collection_add(&ofm->old_rules, old_rule); } /* Take ownership of the temp_rule. */
//將去重處理后的新new_rule放入ofm->new_rules,並且清空ofm->temp_rule-------------重點!!!!//開始真正插入操作!!!-----重點---------------開始將規則 //“舊規則”可以是逐出規則或替換規則。
if (old_rule) { //開始刪除舊的規則/* Copy values from old rule for modify semantics. */
if (old_rule->removed_reason != OFPRR_EVICTION) { //非驅逐--替換(用舊的替換新的部分信息)
bool change_cookie = (ofm->modify_cookie && new_rule->flow_cookie != OVS_BE64_MAX && new_rule->flow_cookie != old_rule->flow_cookie); ovs_mutex_lock(&new_rule->mutex); ovs_mutex_lock(&old_rule->mutex); if (ofm->command != OFPFC_ADD) { new_rule->idle_timeout = old_rule->idle_timeout; new_rule->hard_timeout = old_rule->hard_timeout; *CONST_CAST(uint16_t *, &new_rule->importance) = old_rule->importance; new_rule->flags = old_rule->flags; new_rule->created = old_rule->created; } if (!change_cookie) {*CONST_CAST(ovs_be64 *, &new_rule->flow_cookie) = old_rule->flow_cookie; } ovs_mutex_unlock(&old_rule->mutex); ovs_mutex_unlock(&new_rule->mutex); }/* Mark the old rule for removal in the next version. */
//標記舊規則,以便后面刪除
cls_rule_make_invisible_in_version(&old_rule->cr, ofm->version);/* Remove the old rule from data structures. */ ofproto_rule_remove__(ofproto, old_rule); //移除舊的規則!!!! } else { table->n_flows++; } ofproto_rule_insert__(ofproto, new_rule); //插入新的規則(就是我們設置的生效時間下的流表項) /* Make the new rule visible for classifier lookups only from the next * version. */ classifier_insert(&table->cls, &new_rule->cr, new_rule->ofm->version, new_rule->ofm->conjs, new_rule->ofm->n_conjs); new_rule->state = RULE_INSERTED; //修改規則狀態為已安裝狀態!!!//-----------更新完成之后進行通知-
ofproto_bump_tables_version(ofproto); //判斷版本,不用管//ofproto_flow_mod_finish(ofproto, new_rule->ofm, new_rule->ofm->omr); //------------------這裏開始出錯!-------------------::::注釋之後可以正常開始了:::::://下面才是重點
ofmonitor_flush(ofproto->connmgr); //ofproto->connmgr是OpenFlow交換機的連接管理器-------------通知內核,及時修改流表 }
目前第一個較為穩定的版本已經實現完成!!!
對於修改源碼后,編譯程序的方法見:
openvswitch2.11.0修改源碼后重新編譯(2)