Skip to main content

How to upload files to Amazon S3 Bucket with Java

This is going to be a piece of code really useful for many purposes for uploading anything you want to your S3 Bucket, as MySql or PostgreSql backups, or anything else.


the first thing we need to do after created the bucket you are going to set the data, go to AWS Identity and Access Management (IAM) is a web service that helps you securely control access to AWS resources.

Let’s create such user. Go to Services -> IAM. In the navigation pane, choose Users and then choose Add user.




Enter user’s name and check Access type ‘Programatic access’. Press next button. We need to add permissions to this user. Press ‘Attach existing policy directly’, in the search field enter ‘s3’ and among found permissions choose AmazonS3FullAccess.



Then press next and ‘Create User’. If you did everything right then you should see an Access key ID and a Secret access key for your user. There is also ‘Download .csv’ button for downloading these keys, so please click on it in order not to loose keys.

Our S3 Bucket configuration is done so let’s proceed to our Maven Java application.



AWS S3 PutObject – In this tutorial, we will learn about how to upload an object to Amazon S3 bucket using java language.

Project Setup

Create a simple maven project in your favorite IDE and add below mentioned dependency in your pom.xml file.
<dependency>
  <groupId>com.amazonaws</groupId>
  <artifactId>aws-java-sdk-s3</artifactId>
  <version>1.11.533</version>
</dependency>
For latest version of aws library, check this page.

S3 Connection

Create an object of AmazonS3 ( com.amazonaws.services.s3.AmazonS3 ) class for sending a client request to S3. To get instance of this class, we will use AmazonS3ClientBuilder builder class. It requires three important parameters :- 
  1. Region :- It is a region where S3 table will be stored.
  2. ACCESS_KEY :- It is a access key for using S3. You can generate this key, using aws management console.
  3. SECRET_KEY :- It is a secret key of above mentioned access key.
Here is a code example :-
AmazonS3 s3 = AmazonS3ClientBuilder.standard()
    .withRegion(Regions.AP_SOUTH_1)
    .withCredentials(new AWSStaticCredentialsProvider
                                                    (new BasicAWSCredentials("ACCESS_KEY","SECRET_KEY")))
    .build();

Put Object

An AmazonS3.putObject method uploads a new Object to the specified Amazon S3 bucket. The specified bucket must be present in the Amazon S3 and the caller must have Permission.Write permission on the bucket.
The PutObjectRequest object can be used to create and send the client request to Amazon S3. A bucket name, object key, and file or input stream are only information required for uploading the object. This object can optionally upload an object metadata and can also apply a Canned ACL to the new object. Depending on whether a file or input stream is being uploaded, this request has slightly different behavior:-
Uploading File
  1. The AmazonS3 client automatically computes and validate a checksum of the file.
  2. AmazonS3 determines the correct content type and content disposition to use for the new object by using the file extension.
Uploading Input Stream
  1. The content length must be specified before data is uploaded to Amazon S3. If not provided, the library will have to buffer the contents of the input stream in order to calculate it.
Amazon S3 never stores partial objects, if an exception is thrown during an upload operation, the partial objects will get deleted.


package com.AWS_S3.AwsS3_Upload;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;


import com.amazonaws.AmazonServiceException;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.CannedAccessControlList;
import com.amazonaws.services.s3.model.ObjectMetadata;
import com.amazonaws.services.s3.model.PutObjectRequest;
import com.amazonaws.services.s3.model.PutObjectResult;
import com.amazonaws.services.s3.model.StorageClass;
import java.net.URISyntaxException;
import java.security.CodeSource;
import java.util.Properties;
import java.util.logging.Level;
import java.util.logging.Logger;

public class App
{
    public static void main( String[] args )
    {
   
   
    CodeSource codeSource = App.class.getProtectionDomain().getCodeSource();
         File jarFile = null;
         try {
             jarFile = new File(codeSource.getLocation().toURI().getPath());
         } catch (URISyntaxException ex) {
             Logger.getLogger(App.class.getName()).log(Level.SEVERE, null, ex);
         }
         String jarDir = jarFile.getParentFile().getPath();
         System.out.println("PATH " + jarDir);

         Properties props = new Properties();
         FileInputStream in = null;
         try {
             in = new FileInputStream(jarDir+"/config.properties");
         } catch (FileNotFoundException ex) {
             Logger.getLogger(App.class.getName()).log(Level.SEVERE, null, ex);
         }
         try {
             props.load(in);
         } catch (IOException ex) {
             Logger.getLogger(App.class.getName()).log(Level.SEVERE, null, ex);
         }
         try {
             in.close();
         } catch (IOException ex) {
             Logger.getLogger(App.class.getName()).log(Level.SEVERE, null, ex);
         }

        
        String accesKey = props.getProperty("accessKey");
         String secretKey = props.getProperty("secretKey");
   
       

        /* Create S3 Client Object */
                    AmazonS3 s3 = AmazonS3ClientBuilder
                            .standard()
                            .withRegion(Regions.US_EAST_1)
                            .withCredentials(new AWSStaticCredentialsProvider(
                                                new BasicAWSCredentials(accesKey,secretKey)))
                            .build();
                   
                    /* Set Bucket Name */
                    String bucketName = args[0];
                    //bucketname/db1";
                   
                    InputStream inputStream = null;
                           
                    try {          
                       
                        /* Create File Object */
                        File file = new File(args[1]);
                       
                        /* Set Object Key */
                        String objectKey = file.getName();
                       
                        /* Send Put Object Request */
                        PutObjectResult result = s3.putObject(bucketName, objectKey, file);
                       
                        /* Second Way of Putting Object */
                       
                        /* Create File Object */
                        //file = new File("C:/images/S4.png");
                       
                        /* Create InputStream Object */
                        try {
            inputStream = new FileInputStream(file);
            } catch (FileNotFoundException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
            }
                       
                        /* Set Object Key */
                        objectKey = file.getName();
                     
                     
                        ObjectMetadata metadata = new ObjectMetadata();
                      /*  Map<String,String> map = new HashMap<>();
                        map.put("ImageType", "PNG");
                       
                        metadata.setUserMetadata(map);*/
                       
                        /* Set Content Length */
                        metadata.setContentLength(file.length());
                       
                        /* Create PutObjectRequest Object */
                        PutObjectRequest request = new PutObjectRequest(bucketName, objectKey, inputStream, metadata);
                       
                        /* Set StorageClass as Standard Infrequent Access */
                        request.setStorageClass(StorageClass.StandardInfrequentAccess);
                       
                        /* Set Canned ACL as BucketOwnerFullControl */
                        request.setCannedAcl(CannedAccessControlList.BucketOwnerFullControl);
                       
                        /* Send Put Object Request */
                        result = s3.putObject(request);
                     
                       
                    } catch (AmazonServiceException  e) {
                       
                        System.out.println(e.getMessage());           
                    }
                   
                    if (inputStream != null) {
                        try {
            inputStream.close();
            } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
            }               
                    }
                    if (s3 != null) {
                        s3.shutdown();
                    }
                   
       
    System.out.println("Process ended.....[OK] ");
       
     
    }
}

Do not forget to set the config.properties close to the final jar AwsS3-Upload-0.0.1-SNAPSHOT-jar-with-dependencies.jar

remember to replace inside the config.properties your credentials:

accessKey=
secretKey=

those are the credentiales which were created before in order to have permision over the bucket 

to tun this, just run
mvn clean compile package
go to the target folder and paste the config.properties, or move it to the folder you want to run it, but do not forget that config.properties file

now the way to use it:


1: first, execute the the jar, as java -jar AwsS3-Upload-0.0.1-SNAPSHOT-jar-with-dependencies.jar
2: give it the first parameter as the bucket name, it does not matter if you have more subfolders inside the bucket just keep in mind that the way to especify more subfolders in is with the character /
3: give it the path of the file you want to upload, if you are using windwos that's the way to do it

D:\\filename or
C:\\

if you are using linux systems it would be like /yourpath/yourfile

example:
java -jar AwsS3-Upload-0.0.1-SNAPSHOT-jar-with-dependencies.jar backupsdbmedics/db3 D:\\zpl-zbi2-pm-en-PROGRAMING_GUIDE_ZM400.pdf

you can download the whole project from GitHub at from https://github.com/juandavidmarin368/Upload-AwsS3Bucket-Java

Labels:

Comments

Post a Comment

Popular posts from this blog

How to deploy a VueJS App using Nginx on Ubuntu

There are thousands of blogs and websites out there explaining how to do a hello world and how to start with VueJS, but in this little post, I’m just going to be explaining how to do deploy a VueJs app after you have run the command through the CLI npm run build . So when you run the command npm run build a dist folder is created and that folder’s got the essential .js files to run our app, when we want to run our app on an Nginx server by default the vue-router is not going to work well so that Nginx does not come ready to work by default with a VueJs app This is basically for a Linux Ubuntu distribution if you’ve got any other Linux distribution just pay attention where is going to be set the www/html folder and the Nginx settings so that this is useful for any Linux distribution  Install and configure nginx sudo apt install nginx Double check to make sure the nginx service is running with command service nginx status, then open your browser and enter url
1 comment
Read more

How to do pagination SpringBoot with Jbctemplate, MySQL

We are going to be working on a topic which is a basic need when doing any app, and it is Pagination. let's get started creating a product table at https://mockaroo.com/ create table products ( id INT, name VARCHAR(50), code VARCHAR(50) ); insert into products (id, name, code) values (1, 'Hettinger-Goyette', '42549-680'); insert into products (id, name, code) values (2, 'Konopelski-Klein', '49527-724'); insert into products (id, name, code) values (3, 'Smitham, Kuhlman and Balistreri', '53238-003'); insert into products (id, name, code) values (4, 'Hettinger, Weissnat and Goodwin', '0143-9916'); insert into products (id, name, code) values (5, 'Rowe Inc', '42291-898'); insert into products (id, name, code) values (6, 'Ernser-Hauck', '10544-617'); insert into products (id, name, code) values (7, 'Maggio and Sons', '68788-9087'); insert into products (id, name,
Post a Comment
Read more

How to secure SpringBoot with SSL and Tomcat or Undertow

when we are going to take our applications to production mode, we must have an SSL certificate for the FrontEnd and   BackEnd too, in this case, our backend is a  SpringBoot which is using a tomcat embedded server. Terminology TLS vs SSL TLS is the successor to SSL. It is a protocol that ensures privacy between communicating applications. Unless otherwise stated, in this document consider TLS and SSL as interchangable. Certificate (cert) The public half of a public/private key pair with some additional metadata about who issued it etc. It may be freely given to anyone. Private Key A private key can verify that its corresponding certificate/public key was used to encrypt data. It is never given out publicly. Certificate Authority (CA) A company that issues digital certificates. For SSL/TLS certificates, there are a small number of providers (e.g. Symantec/Versign/Thawte, Comodo, GoDaddy, LetsEncrypt) whose certificates are included by most browsers and Operating System
2 comments
Read more