AmazonS3-Java對接
1、Amazons3？
S3 是一個全球存儲區域網絡(SAN)，它表現為一個超大的硬盤，可以存儲AWS用戶上傳的資源文件。S3可根據AWS用戶需求不同創建不同區域、不同名稱的Bucket（桶）,代碼調用的時候可直接根據BucketNamed對Bucket進行訪問（前提是AWS憑證認證通過）。
2、基礎配置
1、AWS用戶憑證設置
AWS用戶憑證有一下集中方式可配置：
1：LocalFile（本地配置文件）
2：Java.profile/.java文件中設置變量
3：配置環境變量（暫未測試）
1、LoaclFile
1、AWS官網查看用戶Id/Key:
2、Windows設置：
指定目錄下創建文件夾（.aws)及下屬文件（config/credentials)
config文件：
區域可根據自己需求（以及服務所在區域）設置
credentials文件：
3：CLI設置憑證
2、pom.xml配置
只需要引入如下依賴即可，版本可根據自行需求設定
<dependency> <groupId>com.amazonaws</groupId> <artifactId>aws-java-sdk-s3</artifactId> <version>1.11.759</version> </dependency>
3、代碼示例
1、AmazonS3連接（正常連接）
package com.stefanie.sun.bean.AWS.Rekognition; import com.amazonaws.AmazonClientException; import com.amazonaws.AmazonServiceException; import com.amazonaws.ClientConfiguration; import com.amazonaws.Protocol; import com.amazonaws.auth.AWSCredentials; import com.amazonaws.auth.AWSStaticCredentialsProvider; import com.amazonaws.auth.BasicAWSCredentials; import com.amazonaws.auth.DefaultAWSCredentialsProviderChain; import com.amazonaws.auth.profile.ProfileCredentialsProvider; import com.amazonaws.regions.Regions; import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.AmazonS3Client; import com.amazonaws.services.s3.AmazonS3ClientBuilder; import com.amazonaws.services.s3.model.*; import com.amazonaws.util.IOUtils; import lombok.extern.slf4j.Slf4j; import org.springframework.mock.web.MockMultipartFile; import org.springframework.web.multipart.MultipartFile; import java.io.File; import java.io.FileInputStream; import java.io.IOException; import java.util.List; /** * Created with IntelliJ IDEA * StefanieSun * * @Description: AWS S3桶連接 * @Author: zy * @Date: 2020-06-28 15:51 * @Version: 1.0.0 */ @Slf4j public class S3client { private final static String REGION = "us-east-2"; private final static String ACCESSKEY = "your id"; private final static String SECREYKEY = "your key"; public static void main(String[] s) throws Exception { AmazonS3 s3 = amazonS31(); try { //遍歷bucket信息 for (Bucket bucket : s3.listBuckets()) { System.out.println("bucker名稱:" + bucket.getName() + "/bucker創建時間：" + bucket.getCreationDate() + "/bucker-owner:" + bucket.getOwner()); } } catch (AmazonClientException ace) { log.error(ace.getMessage()); } } //1-讀取默認配置連接AmazonS3 public static AmazonS3 amazonS31() { AWSCredentials credentials = null; try { credentials = new ProfileCredentialsProvider("default").getCredentials(); } catch (Exception e) { throw new AmazonClientException("Exception on credentials.", e); } AmazonS3 s3 = new AmazonS3Client(credentials); return s3; } //2-默認連接方式 public static AmazonS3 amazonS32() { AmazonS3 s3 = AmazonS3ClientBuilder.defaultClient(); return s3; } //3-手動設置區域連接 public static AmazonS3 amazonS33(){ AWSCredentials credentials = null; try { //獲取憑證信息 credentials = new ProfileCredentialsProvider().getCredentials(); } catch (Exception e) { throw new AmazonClientException( "Cannot load the credentials from the credential profiles file. " + "Please make sure that your credentials file is at the correct " + "location (~/.aws/credentials), and is in valid format.", e); } //通過美國區域設置獲取 AmazonS3 s3 = AmazonS3ClientBuilder.standard() .withCredentials(new AWSStaticCredentialsProvider(credentials)) .withRegion(REGION) .build(); return s3; } //4-使用S3客戶端創建連接 public static AmazonS3 amazonS34(){ AmazonS3 s3 = new AmazonS3Client(new DefaultAWSCredentialsProviderChain()); return s3; } //5-使用Java.profile/.Java變量連接 public static AmazonS3 amazonS35(){ BasicAWSCredentials awsCreds= new BashicAWSCredentials(ACCESSKEY,SECRETKEY); AmazonS3 s3 = AmazonS3ClientBuilder.standard() .withCredentials(new AWSStaticCredentialsProvider(awsCreds)) .withRegion(Region.US-EASE-2) .build(); return s3; } }
2、異常區域無法訪問
1、錯誤信息
場景：在使用 "us-east-1"區域訪問的時候無法連接，其他區域均可正常連接
java.lang.reflect.InvocationTargetException: null at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.amazonaws.http.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:76) at com.amazonaws.http.conn.$Proxy2.connect(Unknown Source) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1323) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1139) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:796) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:764) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:738) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:698) at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:680) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:544) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:524) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5054) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5000) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4994) at com.amazonaws.services.s3.AmazonS3Client.listBuckets(AmazonS3Client.java:993) at com.amazonaws.services.s3.AmazonS3Client.listBuckets(AmazonS3Client.java:999) at com.stefanie.sun.bean.AWS.Rekognition.S3client.main(S3client.java:49) Caused by: java.net.UnknownHostException: s3.amazonaws.com at java.net.InetAddress.getAllByName0(InetAddress.java:1280) at java.net.InetAddress.getAllByName(InetAddress.java:1192) at java.net.InetAddress.getAllByName(InetAddress.java:1126) at com.amazonaws.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:27) at com.amazonaws.http.DelegatingDnsResolver.resolve(DelegatingDnsResolver.java:38) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) ... 28 common frames omitted 10:14:49.175 [main] DEBUG org.apache.http.impl.conn.DefaultManagedHttpClientConnection - http-outgoing-3: Shutdown connection 10:14:49.175 [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Connection discarded 10:14:49.175 [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection released: [id: 3][route: {s}->https://s3.amazonaws.com:443][total available: 0; route allocated: 0 of 50; total allocated: 0 of 50] 10:14:49.175 [main] ERROR com.stefanie.sun.bean.AWS.Rekognition.S3client - Unable to execute HTTP request: s3.amazonaws.com Process finished with exit code 0
2、原因分析
1：需要調用的服務在US-EAST-1區，Bucket也在US-EAST-1區，但是連接時無法訪問，提示如上所示為請求有問題
2：通過日志比對訪問成功時與訪問失敗的日志可以看出https://------后面的連接有問題，如下圖所示：
由上圖可以看出https請求地址不同。
3：在找出時http請求地址有問題的情況下，查看源代碼，找出問題，但是並未找到哪里設置有問題。
4：在源碼中查出AmazonS3.endpoint屬性存儲http請求地址，故更改屬性即可。
5：使用原有生成AmazonS3的方式無法更改屬性（在構建的時候無法更改屬性）。
3、解決辦法
public static AmazonS3 amazonS36() { AWSCredentials credentials = new BasicAWSCredentials(ACCESSKEY, SECRETKEY); ClientConfiguration clientConfig = new ClientConfiguration(); clientConfig.setProtocol(Protocol.HTTP); clientConfig.withSignerOverride("S3SignerType"); //noinspection deprecation AmazonS3 s3 = new AmazonS3Client(credentials, clientConfig); s3.setEndpoint("https://s3.us-east-1.amazonaws.com");//訪問地址類似，只是區別與用戶憑證的驗證 return s3; }
4、上傳圖片
/** * 上傳文件 * * @param filePath 本地文件路徑 * @param bucketName 桶名稱 * @param keyName 桶路徑以及名稱 實例："assets/test/1513132153/1.png"; * @throws IOException */ private static void uploadFileToBucket(String filePath, String bucketName, String keyName) throws IOException { AmazonS3 s3Client = amazonS36(); File imageFile = new File(filePath); FileInputStream in = new FileInputStream(imageFile); MultipartFile file = new MockMultipartFile("file", imageFile.getName(), "text/plain", IOUtils.toByteArray (in)); ObjectMetadata metadata = new ObjectMetadata(); metadata.setContentType(file.getContentType()); metadata.setContentLength(file.getSize()); try { s3Client.putObject(new PutObjectRequest( bucketName, keyName, file.getInputStream(), metadata)); } catch (AmazonServiceException ase) { ase.getMessage(); } }
5、查看Bucket中資源
/** * 獲取s3桶中數據 */ public static void getS3Object(String bucketName) { AmazonS3 amazonS3 = amazonS36(); ListObjectsV2Result result = amazonS3.listObjectsV2(bucketName); List<S3ObjectSummary> objects = result.getObjectSummaries(); for (S3ObjectSummary os : objects) { System.out.println("* " + os.getKey()); } }