想嘗試多個agent來進行傳輸數據其中的一個配置文件如下:
al.sources = r1
al.sinks = k1
al.channels = c1
al.sources.r1.type = avro
al.sources.r1.channels = c1
al.sources.r1.bind = 0.0.0.0
al.sources.r1.port = 4141
al.sinks.k1.type = logger
al.channels.c1.type = memory
al.channels.c1.capacity = 1000
al.channels.c1.transactionCapacity = 100
al.sources.r1.channels = c1
al.sinks.k1.channel = c1
然后用命令執行:
bin/flume-ng agent -c conf -f conf/spool-logger2.conf -n al -Dflume.root.logger=INFO,console
卻發現一直報錯,部分錯如下 :
2018-09-09T19:11:03,230 INFO [lifecycleSupervisor-1-0] org.apache.flume.node.PollingPropertiesFileConfigurationProvider - Configuration provider starting
2018-09-09T19:11:03,265 INFO [conf-file-poller-0] org.apache.flume.node.PollingPropertiesFileConfigurationProvider - Reloading configuration file:conf/spool-logger.conf
2018-09-09T19:11:03,292 INFO [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Added sinks: k1 Agent: a1
2018-09-09T19:11:03,294 INFO [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Added sinks: k1 Agent: al
2018-09-09T19:11:03,294 INFO [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Processing:k1
2018-09-09T19:11:03,295 INFO [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Processing:k1
2018-09-09T19:11:03,295 INFO [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Processing:k1
2018-09-09T19:11:03,296 INFO [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Processing:k1
2018-09-09T19:11:03,296 INFO [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Processing:k1
2018-09-09T19:11:03,348 WARN [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Agent configuration for 'al' does not contain any channels. Marking it as invalid.
2018-09-09T19:11:03,354 WARN [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Agent configuration invalid for agent 'al'. It will be removed.
2018-09-09T19:11:03,355 INFO [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Post-validation flume configuration contains configuration for agents: [a1]
18-09-09T19:11:03,356 WARN [conf-file-poller-0] org.apache.flume.node.AbstractConfiguratio▽Provider - No configuration found for this host:al
2018-09-09T19:11:03,386 INFO [conf-file-poller-0] org.apache.flume.node.Application - Starting new configuration:{ sourceRunners:{} sinkRunners:{} channels:{} }
重點在於No configuration found for this host:al
這個意思是配置文件沒有al這個名字,但是我反復看了我的配置文件,我發現我的al沒有錯呀,所以一直沒有改
一直到后來,我嘗試把整個文件復制出來,把全部的al全部在改成al然后在嘗試執行命令,果然成功了。
這個結果真是令人抓狂因為英文的al和英文a+數字1很像,所以一直以為是對的
所以最后就是把al拿出來統一改一遍應該就不會錯了。