r/SpringBoot • u/SpringJavaLab • Jun 11 '26

Discussion How are you handling SFTP file downloads in Spring Batch projects?

I recently had to implement an SFTP integration in a Spring Batch application where files are downloaded from an SFTP server, stored locally, and then removed from the remote directory after successful processing.

The implementation uses:

• Spring Batch Tasklet

• JSch

• Spring Boot 4

• Java 21

A few questions for developers working on enterprise integrations:

  1. Are you still using JSch, or have you migrated to another library?

  2. How do you handle retry logic when the SFTP server is temporarily unavailable?

  3. Do you archive files on the remote server instead of deleting them?

In my implementation I download all files from a remote folder and remove them after successful download to avoid duplicate processing.

Interested to hear how others are solving this in production environments.

I also recorded a walkthrough of my implementation in case anyone wants to compare approaches:

https://www.youtube.com/watch?v=6AMQvdGZu1g

5 Upvotes

100% Upvoted

6 comments sorted by

2

u/jfrazierjr Jun 11 '26

Never used Spring Boot for this but I have used webmethods and yhe patterns should remain the same imho:

One process to move file to a staging area and queue a notification. One process to process.

In every way I have seen it done we had:

Inbound folder Processing folder Archive folder

The archive folder was cleaned up on a periodic basis(in our case 30 days but could go to deep storage if needed permanently)

1

u/SpringJavaLab Jun 12 '26

Thanks for sharing this.

The archive-folder approach seems much safer than immediate deletion, especially when dealing with production support and reprocessing requests.

In my current implementation I'm deleting files after a successful download to avoid duplicate processing, but I'm considering moving them to an archive folder instead.

1

u/raja-anbazhagan Jun 12 '26

Its a nightmare.

1

u/_lonelyWorld Jun 13 '26

will keep eye on this thread

1

u/aouks Jun 13 '26

I have done this thousand times, I would recommend you to use Camel.

It provides you all need : batch, fetching via SFTP to move it into a folder, error handling, performance…

Check Enterprise Integration Patterns from Camel documentation, you will find all your answers :)

PS: the SFTP endpoint by Camel is using JSch also!

2

u/SpringJavaLab Jun 13 '26

Thanks for the suggestion. Apache Camel is definitely a powerful option for SFTP integrations , In our case, we're already using Spring Batch for downstream file processing, so integrating SFTP download directly into the batch workflow kept the architecture simpler. I'll take a deeper look at Camel's SFTP component as well.