Skip to main content

Using Custom S3 Resource in Spring Batch Application - Part 2

In the last article titled Simple S3 ItemReader for Spring Batch Application, I explained how to write a custom FileItemReader for S3. You might have noticed, the solution works great but you will have to restart your application/server for any changes in the S3 file to take effect.


Here, we will try another approach where we will extend org.springframework.core.io.AbstractResource class and implment a S3 resource provider. For this, we will have to implement the below methods:

  • public String getDescription()
  • public InputStream getInputStream()
  • public boolean exists()
  • public long contentLength()
  • public long lastModified()
  • public String getFilename()
  • public URL getURL()

Let's implement these methods.

Method: getDescription()

This should return a short description on the aws resource including the bucket name and the object name.
StringBuilder builder = new StringBuilder("S3 resource [bucket='");
        builder.append(this.bucketName);
        builder.append("' and key='");
        builder.append(this.key);
        builder.append("']");
        return builder.toString();

Method: getInputStream()

This method should return a S3ObjectInputStream as given below.
GetObjectRequest getObjectRequest = new GetObjectRequest(this.bucketName, this.key);
        return this.amazonS3.getObject(getObjectRequest)
                .getObjectContent();

Method: exists()

To check whether the specified resource exists or not. We can use AmazonS3.getObjectMetadata() to decide whether the resource exists or not.
GetObjectMetadataRequest metadataRequest = new GetObjectMetadataRequest(
                        this.bucketName, this.key);
        return this.amazonS3.getObjectMetadata(metadataRequest);

Method: contentLength()

We can derive this from the above returned ObjectMetadata.
return objectMetadata.getContentLength();

Method: lastModified()

Similar to the above method.
return objectMetadata.getLastModified().getTime();

Method: getFilename()

This is nothing but the resource name.
return this.key;

Method: getURL()

The url can be constructed using the bucket and resource name.
Region region = this.amazonS3.getRegion()
                .toAWSRegion();
        return new URL("https", region.getServiceEndpoint(AmazonS3Client.S3_SERVICE_NAME),
                "/" + this.bucketName + "/" + this.key);

Integrating S3 Resource with FileItemReader

We call the setResource() method of FlatFileItemReader and set the S3 resource instance.
public ItemReader reader() throws IOException {
        FlatFileItemReader reader = new FlatFileItemReader<>();
        reader.setResource(new S3Resource(s3Client(), "bucketName", "fileName"));
        lineMapper.setLineTokenizer(your tokenizer);
        lineMapper.setFieldSetMapper(your field mapper);
        reader.setLineMapper(your line mapper);
        return reader;
    }

Conclusion

So, we built a resource class for aws S3 and set it as the resource provider for FlatFileItemReader. Similarly you can implement a S3 writer as well, let me know if you need any help on that.

Note: In Spring Cloud, there is in-built support available for reading and writing files from aws S3. Refer this for details.

Comments

Popular posts from this blog

HDFC Bank introduces Missed Call Service to know Account Balance

Missed call is a powerful business tool in developing countries like India where customers give a miss call to specific phone numbers for getting account details, providing feedback, voting etc. On receiving a missed call from a registered phone number, the underlying app performs a phone number lookup and sends the data to the caller via text message(SMS) or records the call details for future processing.HDFC Bank recently introduced missed call service for its retail customers which allows to retrieve bank account details, mini statement etc. by simply giving a miss call to their toll free numbers.Following services are now available: 1800 270 3333 - Account Balance 1800 270 3355 - Mini Statement 1800 270 3366 - Request for new Cheque Book 1800 270 3377 - Request for Bank account statement Also you can download HDFC Mobile Banking Application by giving a missed call to : 1800 270 3344. Other banks providing missed call serviceAxis bank(known as Axis Dial) - 09225892258Bank of India…

Induction Cooker Showing an Error Code? Induction Cooker Error Codes Explained

Are you searching for Induction Cook-top error codes? Here you can find the error codes of all popular induction cooktops and how to troubleshoot it.

These are for your reference only, do not try to open your cooktop without proper safety measures, we advise you to call the service person if any servicing is needed.

If you want to know how Induction Cook-top works, read our previous article titled What is Induction Cooker? How Induction Cooker Works?.

Whirlpool Induction Cooker Error CodesError CodeErrorSolutionF0An internal error was detected.Disconnect power. Wait 5 seconds before reconnecting power. If the symbol appears again, call for service.F2The surface cooking area is too hot and has turned off.Remove the pans from the surface cooking area. "F2" will disappear when the surface cooking area has cooled. If you turn the surface cooking area back on and "F2" reappears, the cooktop is still too hot. Turn off the surface cooking area and allow it to cool.F4The po…

LICHFL - Generating Home Loan Statements Online

Generating an online statement from LIC Housing Finance Ltd is very easy, simply follow the below steps to create an online account with LICHFL and generate statements online! You may use the online generated statement as a proof for principal paid for a housing loan(under section 80C) and interest paid(under section 24) while filing income tax returns.* Want to know how to save maximum income tax? Read our most read article how to save maximum income tax (opens in new tab).* Paying high interest to LICHFL? Learn how to reduce interest on your existing home loan (opens in new tab).
Before reading further, make sure you have the following information with you: Your Loan Account NumberSanctioned AmountStep 1 - Open LICHFL websiteVisit LICHFL website and click on the 'New Customers? Click Here' link(refer the below screenshot). Step 2 - Enter your loan account detailsFill in the following details: New Loan NumberSanctioned AmountDate of BirthSecurity Codeand click on the Submit b…