【原創】大數據基礎之Ambari(1)簡介、編譯安裝、使用


官方:http://ambari.apache.org/

The Apache Ambari project is aimed at making Hadoop management simpler by developing software for provisioning, managing, and monitoring Apache Hadoop clusters. Ambari provides an intuitive, easy-to-use Hadoop management web UI backed by its RESTful APIs.

ambari通過web UI使得Hadoop管理更加簡單,覆蓋安裝、管理和監控等方面;

Ambari enables System Administrators to:

  • Provision a Hadoop Cluster
    •   Ambari provides a step-by-step wizard for installing Hadoop services across any number of hosts.
    •   Ambari handles configuration of Hadoop services for the cluster.
  • Manage a Hadoop Cluster
    •   Ambari provides central management for starting, stopping, and reconfiguring Hadoop services across the entire cluster.
  • Monitor a Hadoop Cluster
    •   Ambari provides a dashboard for monitoring health and status of the Hadoop cluster.
    •   Ambari leverages Ambari Metrics System for metrics collection.
    •   Ambari leverages Ambari Alert Framework for system alerting and will notify you when your attention is needed (e.g., a node goes down, remaining disk space is low, etc).

功能包括:安裝向導、集群配置、各個組件的啟動停止和配置更新、監控頁面、指標收集、報警等;

Ambari enables Application Developers and System Integrators to:

  • Easily integrate Hadoop provisioning, management, and monitoring capabilities to their own applications with the Ambari REST APIs.

Ambari對HDP即Mpack的支持版本情況詳見:https://supportmatrix.hortonworks.com/ 

可見Ambari 2.7.3 只支持 HDP 3.0及以上版本;

 

 

一 安裝

官方安裝指南:https://cwiki.apache.org/confluence/display/AMBARI/Installation+Guide+for+Ambari+2.7.3

1 下載編譯打包

# wget http://www.apache.org/dist/ambari/ambari-2.7.3/apache-ambari-2.7.3-src.tar.gz
# tar xfvz apache-ambari-2.7.3-src.tar.gz
# cd apache-ambari-2.7.3-src
# mvn versions:set -DnewVersion=2.7.3.0.0

 

# pushd ambari-metrics
# mvn versions:set -DnewVersion=2.7.3.0.0
# popd

 

# mvn -B clean install rpm:rpm -DnewVersion=2.7.3.0.0 -DbuildNumber=4295bb16c439cbc8fb0e7362f19768dde1477868 -DskipTests -Dpython.ver="python >= 2.6"

最后一步要build很久(里邊的坑不少,還與網絡情況有關,要反復執行多遍,后續執行建議將clean去掉,節省時間,估計得幾天),並且可能出現多種報錯,詳見附錄;

成功build之后是這樣的:

[INFO] Ambari Main 2.7.3.0.0 .............................. SUCCESS [ 0.738 s]
[INFO] Apache Ambari Project POM 2.7.3.0.0 ................ SUCCESS [ 0.115 s]
[INFO] Ambari Web 2.7.3.0.0 ............................... SUCCESS [ 44.403 s]
[INFO] Ambari Views 2.7.3.0.0 ............................. SUCCESS [ 0.359 s]
[INFO] Ambari Admin View 2.7.3.0.0 ........................ SUCCESS [ 5.570 s]
[INFO] ambari-utility 1.0.0.0-SNAPSHOT .................... SUCCESS [ 0.541 s]
[INFO] ambari-metrics 2.7.3.0.0 ........................... SUCCESS [ 0.381 s]
[INFO] Ambari Metrics Common 2.7.3.0.0 .................... SUCCESS [ 4.635 s]
[INFO] Ambari Metrics Hadoop Sink 2.7.3.0.0 ............... SUCCESS [ 2.082 s]
[INFO] Ambari Metrics Flume Sink 2.7.3.0.0 ................ SUCCESS [ 1.214 s]
[INFO] Ambari Metrics Kafka Sink 2.7.3.0.0 ................ SUCCESS [ 1.307 s]
[INFO] Ambari Metrics Storm Sink 2.7.3.0.0 ................ SUCCESS [ 1.775 s]
[INFO] Ambari Metrics Storm Sink (Legacy) 2.7.3.0.0 ....... SUCCESS [ 1.546 s]
[INFO] Ambari Metrics Collector 2.7.3.0.0 ................. SUCCESS [02:03 min]
[INFO] Ambari Metrics Monitor 2.7.3.0.0 ................... SUCCESS [ 0.612 s]
[INFO] Ambari Metrics Grafana 2.1.0.0.0 ................... SUCCESS [ 2.052 s]
[INFO] Ambari Metrics Host Aggregator 2.7.3.0.0 ........... SUCCESS [ 44.355 s]
[INFO] Ambari Metrics Assembly 2.7.3.0.0 .................. SUCCESS [01:18 min]
[INFO] Ambari Service Advisor 1.0.0.0-SNAPSHOT ............ SUCCESS [ 0.175 s]
[INFO] Ambari Server 2.7.3.0.0 ............................ SUCCESS [11:39 min]
[INFO] Ambari Functional Tests 2.7.3.0.0 .................. SUCCESS [ 0.818 s]
[INFO] Ambari Agent 2.7.3.0.0 ............................. SUCCESS [04:35 min]
[INFO] ambari-logsearch 2.7.3.0.0 ......................... SUCCESS [ 1.587 s]
[INFO] Ambari Logsearch Appender 2.7.3.0.0 ................ SUCCESS [ 4.570 s]
[INFO] Ambari Logsearch Config Api 2.7.3.0.0 .............. SUCCESS [ 0.196 s]
[INFO] Ambari Logsearch Config JSON 2.7.3.0.0 ............. SUCCESS [ 0.209 s]
[INFO] Ambari Logsearch Config Solr 2.7.3.0.0 ............. SUCCESS [ 20.562 s]
[INFO] Ambari Logsearch Config Zookeeper 2.7.3.0.0 ........ SUCCESS [ 1.872 s]
[INFO] Ambari Logsearch Config Local 2.7.3.0.0 ............ SUCCESS [ 0.182 s]
[INFO] Ambari Logsearch Log Feeder Plugin Api 2.7.3.0.0 ... SUCCESS [02:45 min]
[INFO] Ambari Logsearch Log Feeder Container Registry 2.7.3.0.0 SUCCESS [ 22.523 s]
[INFO] Ambari Logsearch Log Feeder 2.7.3.0.0 .............. SUCCESS [01:45 min]
[INFO] Ambari Logsearch Web 2.7.3.0.0 ..................... SUCCESS [02:24 min]
[INFO] Ambari Logsearch Server 2.7.3.0.0 .................. SUCCESS [03:55 min]
[INFO] Ambari Logsearch Assembly 2.7.3.0.0 ................ SUCCESS [ 2.389 s]
[INFO] Ambari Logsearch Integration Test 2.7.3.0.0 ........ SUCCESS [01:20 min]
[INFO] ambari-infra 2.7.3.0.0 ............................. SUCCESS [ 4.214 s]
[INFO] Ambari Infra Solr Client 2.7.3.0.0 ................. SUCCESS [ 18.131 s]
[INFO] Ambari Infra Solr Plugin 2.7.3.0.0 ................. SUCCESS [05:11 min]
[INFO] Ambari Infra Manager 2.7.3.0.0 ..................... SUCCESS [02:14 min]
[INFO] Ambari Infra Assembly 2.7.3.0.0 .................... SUCCESS [ 7.399 s]
[INFO] Ambari Infra Manager Integration Tests 2.7.3.0.0 ... SUCCESS [ 13.159 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

 生成的rpm在這里

# ls ambari-server/target/rpm/ambari-server/RPMS/x86_64/
ambari-server-2.7.3.0-0.x86_64.rpm

ls ambari-agent/target/rpm/ambari-agent/RPMS/x86_64/
ambari-agent-2.7.3.0-0.x86_64.rpm

 

2 分發

將build完成的ambari-server-2.7.3.0-0.x86_64.rpm和ambari-agent-2.7.3.0-0.x86_64.rpm目錄拷貝到集群對應的服務器上,比如ansible 詳見 https://www.cnblogs.com/barneywill/p/10263278.html

3 啟動

3.1 server

安裝

yum install ambari-server*.rpm

Run the setup command to configure your Ambari Server, Database, JDK, LDAP, and other options:

ambari-server setup

具體過程為:

# ambari-server setup
Using python /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Custom JDK
==============================================================================
Enter choice (1): 2
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /data/jdk1.8.0_191/
Validating JDK on Ambari Server...done.
Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? y
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (1): 3
Hostname (localhost): $mysql_host
Port (3306):
Database name (ambari):
Username (ambari):
Enter Database Password (bigdata):
Re-enter password:
Configuring ambari database...
Should ambari use existing default jdbc /usr/share/java/mysql-connector-java.jar [y/n] (y)? y
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
ambari-admin-2.7.3.0.0.jar

Ambari repo file doesn't contain latest json url, skipping repoinfos modification
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

需要手工初始化數據庫:/var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql,導入之后表結構如下,導入詳見:https://www.cnblogs.com/barneywill/p/10271024.html

mysql> show tables;
+-------------------------------+
| Tables_in_ambari |
+-------------------------------+
| ClusterHostMapping |
| QRTZ_BLOB_TRIGGERS |
| QRTZ_CALENDARS |
| QRTZ_CRON_TRIGGERS |
| QRTZ_FIRED_TRIGGERS |
| QRTZ_JOB_DETAILS |
| QRTZ_LOCKS |
| QRTZ_PAUSED_TRIGGER_GRPS |
| QRTZ_SCHEDULER_STATE |
| QRTZ_SIMPLE_TRIGGERS |
| QRTZ_SIMPROP_TRIGGERS |
| QRTZ_TRIGGERS |
| adminpermission |
| adminprincipal |
| adminprincipaltype |
| adminprivilege |
| adminresource |
| adminresourcetype |
| alert_current |
| alert_definition |
| alert_group |
| alert_group_target |
| alert_grouping |
| alert_history |
| alert_notice |
| alert_target |
| alert_target_states |
| ambari_configuration |
| ambari_operation_history |
| ambari_sequences |
| artifact |
| blueprint |
| blueprint_configuration |
| blueprint_setting |
| clusterconfig |
| clusters |
| clusterservices |
| clusterstate |
| confgroupclusterconfigmapping |
| configgroup |
| configgrouphostmapping |
| execution_command |
| extension |
| extensionlink |
| groups |
| host_role_command |
| host_version |
| hostcomponentdesiredstate |
| hostcomponentstate |
| hostconfigmapping |
| hostgroup |
| hostgroup_component |
| hostgroup_configuration |
| hosts |
| hoststate |
| kerberos_descriptor |
| kerberos_keytab |
| kerberos_keytab_principal |
| kerberos_principal |
| key_value_store |
| kkp_mapping_service |
| members |
| metainfo |
| permission_roleauthorization |
| remoteambaricluster |
| remoteambariclusterservice |
| repo_applicable_services |
| repo_definition |
| repo_os |
| repo_tags |
| repo_version |
| request |
| requestoperationlevel |
| requestresourcefilter |
| requestschedule |
| requestschedulebatchrequest |
| role_success_criteria |
| roleauthorization |
| servicecomponent_version |
| servicecomponentdesiredstate |
| serviceconfig |
| serviceconfighosts |
| serviceconfigmapping |
| servicedesiredstate |
| setting |
| stack |
| stage |
| topology_host_info |
| topology_host_request |
| topology_host_task |
| topology_hostgroup |
| topology_logical_request |
| topology_logical_task |
| topology_request |
| upgrade |
| upgrade_group |
| upgrade_history |
| upgrade_item |
| user_authentication |
| users |
| viewentity |
| viewinstance |
| viewinstancedata |
| viewinstanceproperty |
| viewmain |
| viewparameter |
| viewresource |
| viewurl |
| widget |
| widget_layout |
| widget_layout_user_widget |
+-------------------------------+
111 rows in set (0.00 sec)

后續想修改數據庫可以看這里

/etc/ambari-server/conf/ambari.properties

啟動

ambari-server start

輸出

Using python /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start................
Server started listening on 8080

3.2 agent

安裝

yum install ambari-agent*.rpm

配置

/etc/ambari-agent/conf/ambari-agent.ini

將其中的hostname修改為master的host或ip

[server]
hostname=localhost

啟動

ambari-agent start

輸出

Verifying Python version compatibility...
Using python /usr/bin/python
Checking for previously running Ambari Agent...
Checking ambari-common dir...
Starting ambari-agent
Verifying ambari-agent process status...
Ambari Agent successfully started
Agent PID at: /run/ambari-agent/ambari-agent.pid
Agent out at: /var/log/ambari-agent/ambari-agent.out
Agent log at: /var/log/ambari-agent/ambari-agent.log

4 打開Web

http://<ambari-server-host>:8080

Log in with username admin and password admin and follow on-screen instructions. 

當你看到以下頁面的時候,就說明你成功了一半

 

安裝頁面:

使用過程中也可能遇到問題,詳見這里:https://www.cnblogs.com/barneywill/p/10273687.html

 

如果某個節點掛掉重裝系統,需要手工操作:

# yum install ambari-agent
# ambari-agent restart <master hostname>

然后在ambari server上進行Recover Host操作;

 

附:build過程錯誤

1)MojoExecutionException

[ERROR] Failed to execute goal org.codehaus.mojo:rpm-maven-plugin:2.0.1:rpm (default-cli) on project ambari: RPM build execution returned: '127' executing '/bin/sh -c cd /data/apache-ambari-2.7.3-src/target/rpm/ambari/SPECS && rpmbuild -bb --buildroot /data/apache-ambari-2.7.3-src/target/rpm/ambari/buildroot --define '_topdir /data/apache-ambari-2.7.3-src/target/rpm/ambari' --target noarch-redhat-linux ambari.spec' -> [Help 1]

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:rpm-maven-plugin:2.0.1:rpm (default-cli) on project ambari: RPM build execution returned: '127' executing '/bin/sh -c cd /data/apache-ambari-2.7.3-src/target/rpm/ambari/SPECS && rpmbuild -bb --buildroot /data/apache-ambari-2.7.3-src/target/rpm/ambari/buildroot --define '_topdir /data/apache-ambari-2.7.3-src/target/rpm/ambari' --target noarch-redhat-linux ambari.spec'

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)

    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)

    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)

    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)

    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)

    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)

    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)

    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)

    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)

    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)

    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)

    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)

    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)

    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke (Method.java:566)

    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)

    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)

    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)

    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)

Caused by: org.apache.maven.plugin.MojoExecutionException: RPM build execution returned: '127' executing '/bin/sh -c cd /data/apache-ambari-2.7.3-src/target/rpm/ambari/SPECS && rpmbuild -bb --buildroot /data/apache-ambari-2.7.3-src/target/rpm/ambari/buildroot --define '_topdir /data/apache-ambari-2.7.3-src/target/rpm/ambari' --target noarch-redhat-linux ambari.spec'

    at org.codehaus.mojo.rpm.RPMHelper.buildPackage (RPMHelper.java:150)

    at org.codehaus.mojo.rpm.AbstractRPMMojo.execute (AbstractRPMMojo.java:736)

    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)

    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)

    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)

    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)

    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)

    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)

    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)

    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)

    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)

    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)

    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)

    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)

    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)

    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke (Method.java:566)

    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)

    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)

    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)

    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)

原因是缺少rpm-build

yum install -y rpm-build

2)超時

[INFO] Downloading https://github.com/yarnpkg/yarn/releases/download/v0.23.2/yarn-v0.23.2.tar.gz to /root/.m2/repository/com/github/eirslett/yarn/0.23.2/yarn-0.23.2./yarn-v0.23.2.tar.gz

 

[get] Getting: http://dev.hortonworks.com.s3.amazonaws.com/HDP/centos7/3.x/BUILDS/3.0.0.0-1634/tars/hbase/hbase-2.0.0.3.0.0.0-1634-bin.tar.gz
[get] To: /data/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/hbase.tar.gz

有好幾個jar下載過程可能很慢或者報超時錯誤和連接被重置,這時要么換個環境手工下載后放到maven local repository,要么直接重試;

3)編譯錯誤

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (default-compile) on project ambari-metrics-common: Compilation failure: Compilation failure: 
[ERROR] /data/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetrics.java:[24,33] package javax.xml.bind.annotation does not exist

這是因為你使用了jdk11,因為java.xml.bind (JAXB) 在11中 REMOVED,改用11之前的jdk版本即可;

Java 8 - OK
Java 9 - DEPRECATED
Java 10 - DEPRECATED
Java 11 - REMOVED

4)RatCheckException

[ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.12:check (default) on project ambari: Too many files with unapproved license: 1 See RAT report in: /data/apache-ambari-2.7.3-src/target/rat.txt -> [Help 1]

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.rat:apache-rat-plugin:0.12:check (default) on project ambari: Too many files with unapproved license: 1 See RAT report in: /data/apache-ambari-2.7.3-src/target/rat.txt

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)

    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)

    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)

    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)

    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)

    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)

    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)

    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)

    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)

    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)

    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)

    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)

    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke (Method.java:498)

    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)

    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)

    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)

    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)

Caused by: org.apache.rat.mp.RatCheckException: Too many files with unapproved license: 1 See RAT report in: /data/apache-ambari-2.7.3-src/target/rat.txt

    at org.apache.rat.mp.RatCheckMojo.check (RatCheckMojo.java:183)

    at org.apache.rat.mp.RatCheckMojo.execute (RatCheckMojo.java:156)

    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)

    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)

    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)

    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)

    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)

    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)

    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)

    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)

    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)

    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)

    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)

    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)

    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke (Method.java:498)

    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)

    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)

    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)

    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)

mvn命令中增加 

-Drat.skip=true

5)MojoExecutionException

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (psutils-compile) on project ambari-metrics-host-monitoring: An Ant BuildException has occured: exec returned: 1

[ERROR] around Ant part ...<exec failonerror="true" dir="/data/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/psutil" executable="/data/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/../../ambari-common/src/main/unix/ambari-python-wrap">... @ 4:275 in /data/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/target/antrun/build-psutils-compile.xml

[ERROR] -> [Help 1]

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (psutils-compile) on project ambari-metrics-host-monitoring: An Ant BuildException has occured: exec returned: 1

around Ant part ...<exec failonerror="true" dir="/data/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/psutil" executable="/data/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/../../ambari-common/src/main/unix/ambari-python-wrap">... @ 4:275 in /data/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/target/antrun/build-psutils-compile.xml

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)

    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)

    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)

    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)

    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)

    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)

    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)

    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)

    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)

    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)

    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)

    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)

    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke (Method.java:498)

    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)

    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)

    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)

    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)

Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: exec returned: 1

around Ant part ...<exec failonerror="true" dir="/data/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/psutil" executable="/data/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/../../ambari-common/src/main/unix/ambari-python-wrap">... @ 4:275 in /data/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/target/antrun/build-psutils-compile.xml

    at org.apache.maven.plugin.antrun.AntRunMojo.execute (AntRunMojo.java:355)

    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)

    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)

    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)

    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)

    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)

    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)

    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)

    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)

    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)

    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)

    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)

    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)

    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke (Method.java:498)

    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)

    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)

    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)

    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)

Caused by: org.apache.tools.ant.BuildException: exec returned: 1

    at org.apache.tools.ant.taskdefs.ExecTask.runExecute (ExecTask.java:646)

    at org.apache.tools.ant.taskdefs.ExecTask.runExec (ExecTask.java:672)

    at org.apache.tools.ant.taskdefs.ExecTask.execute (ExecTask.java:498)

    at org.apache.tools.ant.UnknownElement.execute (UnknownElement.java:291)

    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)

    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke (Method.java:498)

    at org.apache.tools.ant.dispatch.DispatchUtils.execute (DispatchUtils.java:106)

    at org.apache.tools.ant.Task.perform (Task.java:348)

    at org.apache.tools.ant.Target.execute (Target.java:390)

    at org.apache.tools.ant.Target.performTasks (Target.java:411)

    at org.apache.tools.ant.Project.executeSortedTargets (Project.java:1399)

    at org.apache.tools.ant.Project.executeTarget (Project.java:1368)

    at org.apache.maven.plugin.antrun.AntRunMojo.execute (AntRunMojo.java:327)

    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)

    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)

    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)

    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)

    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)

    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)

    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)

    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)

    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)

    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)

    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)

    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)

    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke (Method.java:498)

    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)

    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)

    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)

    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)

這個看不出直接原因,錯誤是因為執行python返回1,具體的錯誤沒報出來,只能手工重現問題;

首先看報錯的腳本:

# cat /data/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/target/antrun/build-psutils-compile.xml

<?xml version="1.0" encoding="UTF-8" ?>

<project name="maven-antrun-" default="psutils-compile"  >

<target name="psutils-compile">

  <exec failonerror="true" dir="/data/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/psutil" executable="/data/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/../../ambari-common/src/main/unix/ambari-python-wrap">

    <arg value="setup.py"/>

    <arg value="build"/>

    <arg value="--build-platlib"/>

    <arg value="/data/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/target/psutil_build"/>

  </exec>

</target>

</project>

然后手工重現:

# cd /data/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/psutil

# /data/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/../../ambari-common/src/main/unix/ambari-python-wrap setup.py build --build-platlib /data/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/target/psutil_build
running build
running build_py
running build_ext
building '_psutil_linux' extension
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c psutil/_psutil_linux.c -o build/temp.linux-x86_64-2.7/psutil/_psutil_linux.o
unable to execute gcc: No such file or directory
error: command 'gcc' failed with exit status 1

終於看到直接原因了,缺少gcc(你的原因可能不同,手工執行可以看到)

yum install gcc

6)RuntimeError

如果在后續部署ambari-metrics-collector的時候報錯

RuntimeError: Failed to execute command '/usr/bin/yum -y install ambari-metrics-collector', exited with code '1', message: 'Error: Nothing to do'

 這是因為ambari.repo沒有加到yum中

# wget http://s3.amazonaws.com/public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0/ambari.repo

# mv ambari.repo /etc/yum.repos.d/

可以改為你需要的版本;

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM