外業數據采集平台(GPS+Android Studio+Arcgis for android 100.2.1)

本文轉載自查看原文 2018-07-05 22:31 1016 GIS/ LocationManager/ Arcgis for android/ 土地確權/ MapView/ 海域監測/ Arcgis for android 100/ GPS/ Android Studio/ 地圖繪圖/ 外業數據采集平台/ Arcgis

外業數據采集平台

1. 綜述

在室外，通過平板或者手機接收GPS坐標，實時繪制點、線、面數據，以便為后續進行海域監測、土地確權、地圖繪圖提供有效數據和依據。

2. 技術路線

Android studio3.0.1+Arcgis for android 100.2.1+GPS

2.1 Android studio工具:

2.2 Android studio工具下載地址：

http://www.android-studio.org/index.php/download/hisversion

2.2.1 Android studio安裝與配置

參考博客地址：https://www.cnblogs.com/xiadewang/p/7820377.html

2.3 Arcgis for android

Arcgis for android是ESRI公司專門為Android手機開發GIS地圖軟件的一套API，整合廣泛的地圖和GIS能力在線或離線，包括編輯，分析，地理編碼，路由，網絡地圖管理，數據可視化，移動地圖包和矢量平鋪層。

2.3.1 Arcgis for android SDK

https://developers.arcgis.com/android/latest/

2.3.2 Arcgis for android配置

http://www.cnblogs.com/gis-luq/p/4760370.html

2.4 技術架構

外業數據采集平台采用的是Android最原生也是最基礎的架構，可以理解為MVC，Controller即是Activity和Fragment，但是這兩者掌握了Android系統中絕大多數的資源，並且在內部直接控制View，因此傳統的Android App一般是以Activity和Fragment為核心，將網絡模塊，數據庫管理模塊，文件管理模塊，常用工具類等分離成若干工具類包，供Activity和Fragment使用。由於項目不是很大，最后決定采用Android默認的架構，而沒有采用MVP或者MVVM架構。

3. 平台展示與實例代碼

3.1 登錄與主界面

3.1.1 登錄

3.1.2 主界面

包含最基本的：放大、縮小、旋轉、指北針、比例尺等基本功能。

3.2 基礎功能

3.2.1 導入矢量

支持導入內置SD卡和外置SD卡中.shp格式的數據。

 1 private ShapefileFeatureTable featureLayerShapefile(String filePath, String fileType, boolean
 2         isFullExtent, String dapFilePath, boolean isVisible, float opacity, boolean isCheckLayerName) {
 3     if (!check(filePath, "SHP", isCheckLayerName)) {
 4         return null;
 5     }
 6     ShapefileFeatureTable shapefileFeatureTable = new ShapefileFeatureTable(filePath);
 7     shapefileFeatureTable.loadAsync();
 8     ShapefileFeatureTable finalShapefileFeatureTable = shapefileFeatureTable;
 9     shapefileFeatureTable.addDoneLoadingListener(() -> {
10         if (finalShapefileFeatureTable.getLoadStatus() == LoadStatus.LOADED) {
11             // create a feature layer to display the shapefile
12             FeatureLayer featureLayer = new FeatureLayer(finalShapefileFeatureTable);
13             if ("".equals(dapFilePath)) {
14                 featureLayer.setDescription(filePath);
15             } else {
16                 featureLayer.setDescription(dapFilePath);
17             }
18             featureLayer.setVisible(isVisible);
19             mMapView.getMap().getOperationalLayers().add(featureLayer);
20             if (isFullExtent) {
21                 Envelope envelope = LayerUtil.GetLayerFullExtend(featureLayer);
22                 if (envelope != null && !envelope.isEmpty()) {
23                     mMapView.setViewpointGeometryAsync(envelope
24                             , LayerUtil.FullExtendPadding);
25                 }
26             }
27         } else {
28             String error = "Shapefile feature table failed to load: " +
29                     finalShapefileFeatureTable
30                             .getLoadError().toString();
31             Log.e(TAG, error);
32         }
33     });
34 
35     return shapefileFeatureTable;
36 }

3.2.2 導入影像

支持導入內置SD卡和外置SD卡中.tif影像格式的數據。

 1 private Layer loadLocalTif(ArcGISMap arcGISMap, String filePath, String extendName, boolean
 2             isVisible, float opacity, boolean isCheckLayerName) {
 3         if (!check(filePath, extendName, isCheckLayerName)) {
 4             return null;
 5         }
 6         Raster raster = new Raster(filePath);
 7         if (raster == null)
 8             return null;
 9 
10         final RasterLayer rasterLayer = new RasterLayer(raster);
11         rasterLayer.setName("基礎底圖");
12         rasterLayer.setDescription(filePath);
13         rasterLayer.setVisible(isVisible);
14         rasterLayer.setOpacity(opacity);
15         Basemap basemap = new Basemap(rasterLayer);
16         mMapView.getMap().setBasemap(basemap);
17         rasterLayer.addDoneLoadingListener(new Runnable() {
18             @Override
19             public void run() {
20                 mMapView.setViewpointGeometryAsync(rasterLayer.getFullExtent(), LayerUtil
21                         .FullExtendPadding);
22                 mMapView.setViewpointScaleAsync(MyConfig.initialScale);
23             }
24         });
25         mMapView.setViewpointScaleAsync(MyConfig.initialScale);
26 
27         return rasterLayer;
28     }

3.2.3 全圖

將所有圖層中所有要素范圍作為地圖的顯示范圍。

1  Envelope envelope = LayerUtil.GetFullExtend(mMapView);
2  if (envelope != null) {
3      mMapView.setViewpointGeometryAsync(envelope, LayerUtil.FullExtendPadding);
4  }

3.2.4 屬性識別

識別最上層的點、線、面要素。

 1 @Override
 2     public boolean onSingleTapConfirmed(MotionEvent e) {
 3         android.graphics.Point screenPoint = new android.graphics.Point((int) e.getX(), (int) e.getY());
 4         final ListenableFuture<List<IdentifyLayerResult>> identifyFuture = mMapView
 5                 .identifyLayersAsync(
 6                         screenPoint, 20, false, 25);
 7         identifyFuture.addDoneListener(new Runnable() {
 8             @Override
 9             public void run() {
10                 try {
11                     List<IdentifyLayerResult> identifyLayersResults = identifyFuture.get();
12                     for (IdentifyLayerResult identifyLayerResult : identifyLayersResults) {
13                         if (identifyLayerResult.getElements().size() > 0) {
14                             GeoElement topmostElement = identifyLayerResult.getElements().get(0);
15                             if (topmostElement instanceof Feature) {
16                                 Feature identifiedFeature = (Feature) topmostElement;
17                                 LayerContent layerContent = identifyLayerResult.getLayerContent();
18                                 String layerName = "";
19                                 if (layerContent != null)
20                                     layerName = layerContent.getName();
21                                 Map<String, Object> attrs = identifiedFeature.getAttributes();
22                                 String[] itemArr = new String[items.size()];
23                                 itemArr = items.toArray(itemArr);
24                                 showIdentifyInfo(itemArr, layerName);
25                                 break;
26                             }
27                         }
28                     }
29                 } catch (InterruptedException | ExecutionException ex) {
30                 }
31             }
32         });
33 
34         return true;
35     }

3.2.5 圖層管理

包括圖層順序（置頂、置底、上移、下移）、圖層標注（能標注該圖層中多個字段）、圖層樣式（能改變圖層中要素的顏色、寬度等信息）、圖層移除、圖層縮放到（縮放到選擇圖層的所有要素的范圍）等功能；

 1 private void loadLayers(boolean isToast,List<String> listChecked) {
 2         if (mapView == null) {
 3             return;
 4         }
 5         linearLayoutContent.removeAllViews();
 6         LayerList layers = mapView.getMap().getOperationalLayers();
 7         boolean layersLoaded = initialLayers(layers, 2,listChecked);
 8         if(layersLoaded)
 9         {
10             linearLayoutContent.addView(ViewUtil.AddDividerView(this));
11         }
12         Basemap basemap = mapView.getMap().getBasemap();
13         LayerList baseLayers = basemap.getBaseLayers();
14         boolean baseLayersLoaded = initialLayers(baseLayers, 1,listChecked);
15     }
16 
17     /**
18      * @param layers
19      * @param layerType //1:底圖  2：操作類
20      * @return
21      */
22     private boolean initialLayers(LayerList layers, int layerType,List<String> listChecked) {
23         if (layers == null || layers.size() == 0) {
24             return false;
25         }
26         for (int j = layers.size() - 1; j > -1; j--) {
27             Layer layer = layers.get(j);
28             boolean isChecked=false;
29             if(listChecked!=null) {
30                 isChecked= listChecked.contains(layer.getName());
31             }
32             addItem(layer, layerType,isChecked);
33         }
34         return true;
35     }

 1 private void initialLineSymbol(SimpleLineSymbol lineSymbol)
 2     {
 3         rowFillColor.setVisibility(View.INVISIBLE);
 4         rowIsFill.setVisibility(View.INVISIBLE);
 5         txtSymbolColor.setText("線顏色：");
 6         txtSymbolSize.setText("線寬度：");
 7         txtType.setText("線樣式：");
 8 
 9         btnSymbolColor.setBackgroundColor(lineSymbol.getColor());
10         seekBarSymbolSize.seekBar.setProgress((int)lineSymbol.getWidth());
11         initialColorPick(lineSymbol.getColor(),btnSymbolColor);
12     }
13 
14 private SimpleLineSymbol setLineSymbol(SimpleLineSymbol lineSymbol)
15     {
16         lineSymbol.setColor(ViewUtil.GetButtonBackgoundColor(btnSymbolColor));
17         lineSymbol.setWidth(seekBarSymbolSize.seekBar.getProgress());
18         return lineSymbol;
19     }

 1 private void initialListView()
 2     {
 3             FeatureLayer featureLayer=(FeatureLayer)layer;
 4             List<Field> fieldList=  featureLayer.getFeatureTable().getFields();
 5             List<LabelDefinition> labelDefinitionList=  featureLayer.getLabelDefinitions();
 6             boolean isLabel=featureLayer.isLabelsEnabled();
 7            for (Field field :fieldList)
 8            {
 9                CheckBox checkBox=new CheckBox(this);
10                String fileName=field.getAlias();
11                if(TextUtils.isEmpty(fileName))
12                {
13                    fileName=field.getName();
14                }
15                checkBox.setText(fileName);
16                checkBox.setTag(field);
17                if(isLabel)
18                {
19                    for (LabelDefinition labelDefinition:labelDefinitionList)
20                    {
21                      String jsonString =labelDefinition.toJson();
22                        try {
23                            JSONObject jObject = new JSONObject(jsonString);
24                            if(jObject!=null)
25                            {
26                            JSONObject jObjectItem=    jObject.getJSONObject("labelExpressionInfo");
27                                if(jObjectItem!=null)
28                                {
29                                    String expression=jObjectItem.getString("expression");
30                                    String[] expressionContent=expression.split("\\.");
31                                    if(expressionContent.length>1)
32                                    {
33                                     String[] expressionField=   expressionContent[1].split(";");
34                                     if(expressionField.length>0)
35                                     {
36                                        if( expressionField[0].equals(field.getName()))
37                                        {
38                                            checkBox.setChecked(true);
39                                        }
40                                     }
41                                    }
42                                }
43                            }
44                        } catch (JSONException e) {
45 
46                        }
47                    }
48                }
49 
50                listViewLabel.addView(checkBox);
51            }
52     }
53 
54     public void btnLabelOk_Click(View view)
55     {
56             FeatureLayer featureLayer = (FeatureLayer) layer;
57             List<Field> checkedFieldList = getCheckedFields();
58             featureLayer.getLabelDefinitions().clear();
59             if(checkedFieldList!=null&&checkedFieldList.size()>0) {
60                 for (Field field : checkedFieldList) {
61                     LabelDefinition labelDefinition = LayerUtil.CreateFillLabelDefinition(field.getName());
62                     featureLayer.getLabelDefinitions().add(labelDefinition);
63                 }
64                 featureLayer.setLabelsEnabled(true);
65             }
66             else
67             {
68                 featureLayer.setLabelsEnabled(false);
69             }
70             finish();
71     }

3.2.6 距離測量

通過手在屏幕中點擊開始，雙擊停止，支持動態顯示每段線的距離（以米為單位）。

 1 public boolean onDoubleTap(MotionEvent point) {
 2 
 3 if (geoType == GeometryType.POLYLINE)//繪制線
 4         {
 5             if (!lineGeometry.isSketchValid()) {
 6                 removeGraphic(pCurrGraphic);
 7             } else {
 8                 updateGraphic(lineGeometry.toGeometry());
 9             }
10             PartCollection partCollection = lineGeometry.getParts();
11             if (partCollection.size() == 0) {
12                 return false;
13             }
14             Part part = partCollection.get(partCollection.size() - 1);
15             int count = part.getPointCount();
16             if (count <= 2) {
17                 return false;
18             }
19             double length = GeometryEngine.lengthGeodetic(lineGeometry.toGeometry(), new
20                     LinearUnit(LinearUnitId.METERS), GeodeticCurveType.GEODESIC);
21             Graphic lengthGriphic = new Graphic(ptCurrent, getTextSymbol(getFormatString(length,
22                     2, "米"), TextSymbol.HorizontalAlignment.RIGHT, TextSymbol.VerticalAlignment
23                     .TOP));
24             drawLayer.getGraphics().add(lengthGriphic);
25 
26    }
27 
28  }

3.2.7 面積測量

通過手在屏幕中點擊開始，雙擊停止，會形成一個面要素，並顯示長度和面積（以米為單位）。

 1 public boolean onDoubleTap(MotionEvent point) {
 2 
 3 if (!polygonGeometry.isSketchValid()) {
 4     removeGraphic(pCurrGraphic);
 5     return true;
 6 } else {
 7     updateGraphic(polygonGeometry.toGeometry());
 8 }
 9 
10 double area = GeometryEngine.areaGeodetic(polygonGeometry.toGeometry(), new AreaUnit
11         (AreaUnitId.SQUARE_METERS), GeodeticCurveType.GEODESIC);
12 Graphic areaGriphic = new Graphic(ptCurrent, getTextSymbol(getFormatString(area, 2,
13         "平方米"), TextSymbol.HorizontalAlignment.LEFT, TextSymbol.VerticalAlignment
14         .BOTTOM));
15 drawLayer.getGraphics().add(areaGriphic);
16 
17 double length = GeometryEngine.lengthGeodetic(polygonGeometry.toPolyline(), new
18         LinearUnit(LinearUnitId.METERS), GeodeticCurveType.GEODESIC);
19 Point ptShift = mapView.screenToLocation(new android.graphics.Point(Math.round(point
20         .getX()), Math.round(point.getY()) + 30));
21 Graphic lengthGriphic = new Graphic(ptShift, getTextSymbol(getFormatString(length,
22         2, "米"), TextSymbol.HorizontalAlignment.RIGHT, TextSymbol.VerticalAlignment
23         .TOP));
24 drawLayer.getGraphics().add(lengthGriphic);
25 
26  }

3.2.8 清除覆蓋物

清除上面距離測量或者面積測量的圖形要素。

1 if (mearsureGraphicsOveray != null) {
2     mearsureGraphicsOveray.getGraphics().clear();
3 }

3.2.9 當前位置

點擊此按鈕，將GPS最近一次接受點的位置置於地圖中心位置，並增加圖標。

 1 public boolean startGPSCurrent(boolean isRequestUpdate) {
 2     if (GPSLocationManager.gpsLocationManager == null) {
 3         GPSLocationManager.gpsLocationManager = GPSLocationManager.getInstances
 4                 (MainActivity
 5                         .this);
 6     }
 7     if (currentLocationListener == null) {
 8         currentLocationListener = new MyGPSCurrentLocationListener();
 9     }
10     if (isRequestUpdate)
11         GPSLocationManager.gpsLocationManager.stop(2);
12     //開啟定位
13     GPSLocationManager.gpsLocationManager.start(currentLocationListener,
14             isRequestUpdate, 100, 2);
15 
16     return true;
17 }
18 
19  
20 
21 class MyGPSCurrentLocationListener implements GPSLocationListener {
22     @Override
23     public void UpdateLocation(Location location) {
24         if (location != null) {
25             LocationUtil.AddOrUpdateLocation(MainActivity.this, location,
26                     locationGraphicsOveray,
27                     spatialReference, mMapView);
28             if (GPSLocationManager.gpsLocationManager != null) {
29                 GPSLocationManager.gpsLocationManager.stop(2);
30             }
31         }
32     }
33 
34 }

3.2.10 數據導出

將采集到的點、線、面數據導出成.shp格式數據，並支持刪除要素功能。

 1 public void initialGridData(ShapefileFeatureTable shapefileFeatureTable, GridView gridView,
 2                                 int gridType, Context context, boolean isDisplayToastInf) {
 3         List<Field> fileds = shapefileFeatureTable.getFields();
 4         Map<String, Object> gridTitle = new HashMap<String, Object>();
 5 
 6         for (String field : LayerUtil.mFields) {
 7             String key = field;
 8             keysList.add(key);
 9             String alias = LayerUtil.GetAliasNameByFieldName(key);
10             gridTitle.put(key, alias);
11         }
12         gridDataList.add(gridTitle);
13         QueryParameters qParameters = new QueryParameters();
14         String whereClause = "1=1";
15         qParameters.setWhereClause(whereClause);
16         final ListenableFuture<FeatureQueryResult> future = shapefileFeatureTable
17                 .queryFeaturesAsync(qParameters);
18         FeatureQueryResult result = null;
19         try {
20             result = future.get();
21             Iterator<Feature> features = result.iterator();
22             while (features.hasNext()) {
23                 Feature feature = features.next();
24                 Map<String, Object> attrs = feature.getAttributes();
25 
26                 Map<String, Object> dd = new HashMap<String, Object>();
27                 for (String field : LayerUtil.mFields) {
28                     dd.put(field, String.valueOf(attrs.get(field)));
29                 }
30                 gridDataList.add(dd);
31             }
32         } catch  {
33 
34         }
35         String[] keys = keysList.toArray(new String[keysList.size()]);
36         GridViewAdapter adapter = new GridViewAdapter(context, gridDataList);
37         gridView.setAdapter(adapter);
38     }

3.2.11 系統設置

設置GPS接收坐標間隔以及地圖經緯度顯示格式。

 1 @Override
 2         public boolean onPreferenceChange(Preference preference, Object value) {
 3             String stringValue = value.toString();
 4             if (preference instanceof ListPreference) {
 5                 ListPreference listPreference = (ListPreference) preference;
 6                 if (listPreference.getKey().equals(updateMapFormatKey)) {
 7                     PreferencesUtil.setPrefString(this.getActivity(), updateMapFormatKey, String.valueOf(stringValue));
 8                 } else if (listPreference.getKey().equals(updateCollectIntervalKey)) {
 9                     PreferencesUtil.setPrefString(this.getActivity(), updateCollectIntervalKey, String.valueOf(stringValue));
10                     MyConfig.mMinTime=Long.parseLong(stringValue);
11                 }
12             }
13 
14             return true;
15         }

3.3 高級功能

3.3.1 拍照

拍取外業測量點的實況，並在照片上增加經緯度、備注信息，信息錄入完畢后，保存即可，為后續數據處理增加依據。

 1 public void takePhoto(Activity activity) {
 2         int currentapiVersion = android.os.Build.VERSION.SDK_INT;
 3         Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
 4         if (hasSdcard()) {
 5             SimpleDateFormat timeStampFormat = new SimpleDateFormat(
 6                     "yyyy_MM_dd_HH_mm_ss");
 7             String filename = timeStampFormat.format(new Date());
 8             File tempFile = new File(FileUtils.GetDefaultPath(this) + "/" + MyConfig.OutCameraDir,
 9                     filename + ".jpg");
10             imageView.setTag(tempFile);
11             if (currentapiVersion < 24) {
12                 imageUri = Uri.fromFile(tempFile);
13                 intent.putExtra(MediaStore.EXTRA_OUTPUT, imageUri);
14             } else {
15                 ContentValues contentValues = new ContentValues(1);
16                 contentValues.put(MediaStore.Images.Media.DATA, tempFile.getAbsolutePath());
17                 imageUri = activity.getContentResolver().insert(MediaStore.Images.Media
18                         .EXTERNAL_CONTENT_URI, contentValues);
19                 intent.putExtra(MediaStore.EXTRA_OUTPUT, imageUri);
20             }
21         }
22     }

3.3.2 點采集

根據GPS坐標，實時生成點要素，並隨位置變化而變化地圖中心點，點擊 “結束”會彈出屬性保存界面；

3.3.3 線采集

根據GPS坐標，實時生成線要素，並隨位置變化而變化地圖中心點，點擊 “結束”會彈出屬性保存界面；

3.3.4 面采集

根據GPS坐標，實時生成面要素，並隨位置變化而變化地圖中心點，點擊 “結束”會彈出屬性保存界面；

3.3.5 停止編輯

 1 private void addOrUpdateGeometry(Location point ) {
 2         if (point == null || point.isEmpty() || point.getX() == 0 || point.getY() == 0)
 3             return;
 4 
 5 
 6         switch (collectGeometryType) {
 7             case POINT:
 8                 if (collectPointTable == null) {
 9                     return;
10                 }
11                 collectFeatures.add(pointF);
12                 toastGpsInfo("點采集數據", "", collectFeatures.size());
13                 break;
14             case POLYLINE:
15                 if (collectLineTable == null) {
16                     return;
17                 }
18                 lineGeometry.addPoint(point);
19                 collectPoints.add(point);
20                 if (lineGeometry.isSketchValid()) {
21                     if (polylineFeature == null) {
22                         polylineFeature = collectLineTable.createFeature();
23                         polylineFeature.setGeometry(lineGeometry.toGeometry());
24                         collectLineTable.addFeatureAsync(polylineFeature);
25                     } else {
26                         polylineFeature.setGeometry(lineGeometry.toGeometry());
27                         collectLineTable.updateFeatureAsync(polylineFeature);
28                     }
29                 }
30                 toastGpsInfo("線采集數據", "", collectPoints.size());
31                 break;
32             case POLYGON:
33                 if (collectAreaTable == null) {
34                     return;
35                 }
36                 polygonGeometry.addPoint(point);
37                 collectPoints.add(point);
38                 if (polygonGeometry.isSketchValid()) {
39                     if (polygonFeature == null) {
40                         polygonFeature = collectAreaTable.createFeature();
41                         polygonFeature.setGeometry(polygonGeometry.toGeometry());
42                         collectAreaTable.addFeatureAsync(polygonFeature);
43                     } else {
44                         polygonFeature.setGeometry(polygonGeometry.toGeometry());
45                         collectAreaTable.updateFeatureAsync(polygonFeature);
46                     }
47                 }
48                 toastGpsInfo("面采集數據", "", collectPoints.size());
49                 break;
50         }
51     }

3.3.6 撤銷

若有GPS坐標發生偏移，可以根據此功能進行撤銷，一直可以撤銷到第一個采集到的點。

 1 private boolean deleteFirstFeature(ShapefileFeatureTable shapefileFeatureTable, Feature
 2             defaultFeature) {
 3         if (shapefileFeatureTable == null)
 4             return false;
 5         if (shapefileFeatureTable.getTotalFeatureCount() == 1) {
 6             if (defaultFeature == null)
 7                 return false;
 8             String delFid = "";
 9             Map<String, Object> item = defaultFeature.getAttributes();
10             if (item != null) {
11                 delFid = String.valueOf(item.get(CollectEidtActivity.attrFID));
12             }
13             QueryParameters qParameters = new QueryParameters();
14             String whereClause = CollectEidtActivity.attrFID + "=" + delFid;
15             qParameters.setReturnGeometry(true);
16             qParameters.setWhereClause(whereClause);
17 
18             final ListenableFuture<FeatureQueryResult> featuresResult = shapefileFeatureTable
19                     .queryFeaturesAsync(qParameters);
20 
21             FeatureQueryResult features;
22             try {
23                 features = featuresResult.get();
24                 if (!features.iterator().hasNext()) {
25                     return false;
26                 }
27                 shapefileFeatureTable
28                         .deleteFeaturesAsync(features).get();
29                 return true;
30             } catch (Exception ex) {
31                 return false;
32             }
33         }
34 
35         return false;
36     }

免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。

猜您在找 高清影像地圖+完全免費使用！分享一款移動端外業數據采集神器采集的GPS數據如何正確顯示在arcgis和cad中剖析大數據平台的數據采集宜信數據采集平台DBus概述「數據采集」實驗一 Beats數據采集 [開源 .NET 跨平台 Crawler 數據采集爬蟲框架: DotnetSpider] [五] 如何做全站采集? ETL數據采集方法美團數據采集 Hadoop的數據采集框架