解決方案：
在conf后面加上. set("spark.testing.memory", "471859201")
bug報錯：
Exception in thread "main" java.lang.IllegalArgumentException: System memory 259522560 must be at 。。。
解決方案：
hadoop-common-bin-2.7.3-x64.zip解壓到任意目錄。
bug報錯：
org.apache.hadoop.util.Shell$ExitCodeException: /bin/bash: line
bug原因：
win環境運行hadoop任務，需要下載相應的運行環境。
解決方案：
for (x <- glomTest(arrayTwo).collect()) {
println(x.toList)
}
bug報錯：
array（array（），array（）） 這種rdd打印不出來
解決方案：
bin/kafka-console-consumer.sh --bootstrap-server ambari02:9092 --from-beginning --topic first
bug報錯：
zooker ，bootstrap
bug原因：
之前連接zookeeper是老版本，新版本要用 --boot..，同時端口號改為9092
解決方案：
hive-site.xml
1 將Type改成dbcp
2 將mysql-connect-java 的連接驅動 放到spark/jars目錄下
x
1
// 數據庫連接
2
<property>
3
<name>datanucleus.connectionPoolingType</name>
4
<value>dbcp</value>
5
<description>
6
Expects one of [bonecp, dbcp, hikaricp, none].
7
Specify connection pool library for datanucleus
8
</description>
9
</property>
10
//版本校驗
11
<property>
12
<name>hive.metastore.schema.verification</name>
13
<value>false</value>
14
<description>
15
Enforce metastore schema version consistency.
16
True: Verify that version information stored in is compatible with one from Hive jars. Also disable automatic
17
schema migration attempt. Users are required to manually migrate schema after Hive upgrade which ensures
18
proper metastore schema migration. (Default)
19
False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.
20
</description>
21
</property>
bug報錯：
org.datanucleus.exceptions.NucleusUserException: The connection pool plugin of type "HikariCP" was not found in the CLASSPATH! at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:234)
bug原因：
hive-site.xml默認配置的數據庫連接方式就是HikariCP。
bug報錯：
org.apache.hadoop.hive.metastore.ObjectStore (line: 6684) : Version information found in metastore differs 3.1.0 from expected schema version 1.2.0. Schema verififcation is disabled hive.metastore.schema.verification so setting version.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/sql/catalyst/catalog/HiveTableRelation