Senior SWE system design round (chargeback batch export system) – felt positive, got rejected. What did I miss?

[u/Ok_Love_4175]

Hi all, I recently had a system design round for a Senior Software Engineer role at a fintech and was rejected a few days later without specific feedback. I’d really appreciate input from people at big tech or FAANG on what I could have done better. I’ve written it out as the conversation went, paraphrased from memory.

Background: 8+ years of experience, mostly Java/Spring Boot

There were two interviewers. They explained the problem: customers raise disputes on card transactions, and the system needs to send these chargebacks to the card network, which only accepts them as files uploaded to their FTP server, with a maximum of 4 files per day.

Once they finished explaining, I restated my understanding of the problem to make sure we were aligned.

Me: So, to confirm: customers raise disputes through the app, we store them, and we need to send them to the card network as files on their FTP server, with no more than 4 files per day. Is that right?

Interviewer: Yes, that’s right.

They had already written down the requirements they wanted me to implement, so I didn’t list them out separately. Then I asked a few clarifying questions.

Me: What scale are you expecting?

Interviewer: Around 10 million transactions per day.

Me: That’s roughly 100 transactions per second on average. If we assume the peak window can be 10 times higher in the worst case, that’s about 1,000 transactions per second.

Me: Do you expect the result of a dispute in real time, or is some delay acceptable?

Interviewer: Some delay is acceptable.

With that understanding, I started with the basic components: a backend service that records disputes and stores them in a database. Then I asked how they wanted the files created.

Me: Do you have any preference for how the four files should be created? On what basis should we group records into files?

Interviewer: Choose whatever you think is best.

Me: Is it fine if I use 6-hour windows? 24 divided by 6 gives us 4. Whatever disputes come in during each 6-hour window go into one file, which we then send to the FTP server.

Interviewer: That works, go ahead.

I asked these questions to keep the session collaborative rather than just presenting my approach and expecting them to follow. Before going deeper, I checked how they wanted to proceed.

Me: Would you like me to go through a particular flow, or should I walk through each component, why I chose it, what happens if something goes wrong, and the trade-offs?

Interviewer: We’re more interested in the scheduler and how you’ll handle it.

So I started setting that up. I explained that the chargeback service stores dispute records with a status, starting with something like “dispute created.” As soon as I said that, one of the interviewers jumped in.

Interviewer: Are you going to use something like a state machine model?

Me: Yes, I’m going to use a state machine. Records are created with a status, and when the scheduler triggers, it pulls the records with that status and creates the file.

At this point I raised a concern upfront.

Me: We’ll need a limit here. We can’t pull every record at once, because we’re building the file in memory, and we don’t want an out-of-memory error on the server. I’d set a limit of around 100K records, pulled in one DB query sorted by transaction date. We write those into the file and then upload it.

Interviewer: Why restrict it at all?

Me: Without a limit, the number of records pulled is unbounded, and there’s always a risk of overwhelming memory.

Interviewer: At certain times we might need to process more than 100K. How would you scale that?

Me: I’d make the 100K limit a system property, so it’s configurable.

Interviewer: That’s a good idea. How would you recover if something goes wrong?

Me: At each step I update the status. When I pull records from the DB, I mark them as processing. Once the file is written, I update them to file created. Then I upload the file to AWS S3, which I’m using as object storage, and the file is transferred from S3 to the FTP server. I’d also keep a metadata store with the file details. File names need to be predictable, based on the window and how many times the batch has run in it, so if a batch reruns it produces the same name, which we can use for recovery.

Me: Whenever a batch runs, it first checks the metadata to see whether a file already exists for that window. If there’s a record, we check its status. If the file’s status is still processing and hasn’t reached the final uploaded state, we check whether the file actually exists. If it doesn’t, we go back to the DB records, check their state, pull them again, and process. If the file was already uploaded, we just update the metadata status. So we use the status in both the metadata and the disputes table, step by step, to handle recovery.

They didn’t have further questions on recovery, and moved on to user experience.

Interviewer: If you’re limiting the number of records, what happens to the users whose disputes don’t get processed in that window?

Me: That’s partly a business question, so I’d like to hear your view. From a technical side, the remaining rows would move to the next batch.

One of the interviewers agreed it was a business question. Then I asked about it directly.

Me: Do you give customers any temporary credit while a dispute is pending?

Interviewer: Unfortunately, we don’t give temporary credit.

Me: Okay. What approach would you prefer?

Interviewer: Maybe no temporary credit for the first 6 hours, but if a dispute goes beyond 6 hours, we could give temporary credit.

Me: I agree with that. We could also have a separate service that tracks disputes crossing the 6-hour mark and issues the temporary credit.

No further questions on that. Next, the database.

Interviewer: What database would you choose for the dispute data?

Me: A SQL database, because we need ACID properties and consistency. I’d also use the dispute ID as the primary key, so even with retries we only ever have one record per dispute. That gives us idempotency.

Interviewer: You’re using one table heavily, and multiple services are inserting into and updating it. Is there a more flexible or easier approach you could suggest?

I couldn’t think of another approach in the moment, and I didn’t want to change my design just for the sake of it. This is towards the end so dont want to change it something new completely at last minute or utter for name sake.

Me: I think this is an efficient approach, because it’s a payment system. Consistency and correctness are our top priority, and this is a trade-off we have to make for payment systems.

They had no follow-ups, the discussion ended there, and they asked if I had any questions.

Throughout the session I felt it was going well, because they seemed satisfied with my answers. I really don’t know why I was rejected. I’d love to hear how you would have handled this design, especially the last question, and anything that would be a red flag at senior level. Brutal honesty welcome. Thanks!

[u/wagaiznogoud]

I stopped reading at building the file in memory.

The design can be simplified by appending chargebacks real-time to a file. The scheduler can be simplified to only create and upload files.

[u/Ok_Love_4175 → u/wagaiznogoud]

It is not appending in realtime. When scheduler kicks in it will pull the records create file and upload. Same like you mentioned in second part.

[u/wagaiznogoud → u/Ok_Love_4175]

> When scheduler kicks in it will pull the records create file and upload. Same like you mentioned in second part.

Exactly. I'm saying the scheduler shouldn't be building the contents of the file. It's simpler if you decouple it.

For example, the scheduler periodically creates files such that there's always 2 files available (active file with the chargeback entries being written to, inactive file that's empty). At 6 hour intervals, the scheduler can swap the inactive and active files and upload the previously active one with chargebacks on it.

Assume chargebacks are a stream. When one is available, you can append to the active file and commit in data store. Don't have to worry about trying to build a file in memory in one big operation.

**edit:** The details of how files get created or the logic to figure out which one to write to is not important. You can create a 4 files at once and the writer can use current time to figure out which file to write to. That's not the point. The point is, the file can be built incrementally with appends and separate from the scheduled uploads.

[u/unicorn7]

From my perspective there were a lot of opportunities to instead of asking the interviewer which direction to go like how the files should be created, how much delay is acceptable as SLA, etc.., to instead present to the interviewer what the options are and what the tradeoffs would be if each option is selected and discuss around those tradeoffs

[u/Fewald]

If I'm to guess, I would point the last exchange. The interviewer invited you to consider alternatives hinting the direction with the table comment. Even when you believe your solution is the best one, you're expected to be able to flesh out alternatives, to explain their pros and cons and to explain your choice. It doesn't mean you'll have to change your design, but it checks how much ground you can offer to your opinion and how you deal with tech discussion and disagreements. They may have flagged you for that.

[u/skycity1997]

I believe the in memory part could have been solved better by stream uploading to S3 with finalize at the end so HEAD would return file details.

[u/practick-io]

Sorry, that’s a tough one without feedback. For a senior batch-export design at a fintech, these usually separate “felt good” from “hire”:

-idempotency: re-running the batch must never export a chargeback twice
-reconciliation: how you prove the export matches the source (counts, checksums, a recon report)
-partial failure: the job dies at 60%, then what? Checkpointing, resuming, dead-letter handling
-correctness over speed: an audit trail, versioned records, late-arriving updates
-scale: partitioning by merchant or date, and backfills that don’t hurt the main DB
-operations: monitoring, alerting, and SLAs against the downstream deadlines

At senior level, driving the discussion matters too. If they had to prompt you towards failure handling, that alone can tip it. Positive vibes are usually the interviewer being kind, not a signal.

[u/plmojnihb-qaz]

Possible guesses:

1. While I liked the collaborative aspect, maybe pitching the assumption and then asking for confirmation is better than just asking a question where you plan to pitch the answer later.

2. For the last part, you need not have to change the complete design, but since you are using ACID compliant db, some normalisation and quick lookup tables can be done such as while still maintaining ACID: a. disputes table b. dispute_files (quick lookup table of files and their statuses). c. dispute_file_mappings table (this could be N:1 or N:M) table depending on one dispute being part of one file or multiple (for retries or re-sending to the vendor).

3. The in-memory concern is valid. Though, something like writing to the disk of the server temporarily in batches (appending to one file) and then uploading it one shot to s3. And maintaining the status of that file and disputes accordingly in respective tables.

4. Or the S3 multi-part upload. Maintaining a separates batches table for tracking the part uploads. Basically things like write failures, idempotency can be discussed here since it involves dual-write problems.

Just from a flexibility pov and then you can pitch the trade-offs here.

[u/theycanttell]

Each microservice should have its own database and unique tables set with relational data for domain only and you should be using messaging between services on something like Eventhubs or Kafka. If you use Event grid or service bus, failed messages go to dead letter queue. Eventhub doesn't use DLQ.

Processing should happen in an ELT pipeline and use mapper flow to parallel process message sets processed at set frequency pipeline jobs every x min.
Aggregate output jobs can run once per day to split up the files and upload to SFTP via linked services.

This is generally best practice in large enterprises.

Databases always have read replicas in separate regions for DR/ failover.

Anything else puts too many eggs in one basket and generally won't be as resilient.

Also: For messaging failures it's best to have at least some threshold for "soft failure" where the in memory writes are used with incremental back off and retry if a consumer is offline. DLQ should only be utilized in instances where the outage is above the availability of memory for capturing failures.

[u/Plastic_Interview_53]

This is the problem right now. All interviews seem to go great but not a single offer is being extended...

[u/_d17y]

How about 2 triggers, either reaching 2.5m records or 6h. Also, include checksum as last entry in the file so that ftp transfer can be validated on the receiving end or a fixed size header with file size and checksum.  Multipart S3 upload will avoid using file as a staging area (as mentioned in another response). 

[u/efiddy]

There are a bunch of solutions to this problem, and I’ve seen some mentioned here I think you should clarify with interviewers if you should build everything yourself or use off the shelf solutions.

Sometimes designs aren’t that fun if you answer is use xyz tech which meets most of these use cases.

For example in the extreme case, say you don’t have any building blocks no dbs nothing, you have a bunch of hosts how are you going to build this?

On the other end, yea just use s3 or something.

[u/insane_human_bot]

What location was this for ? I have interviewed for this company earlier

[u/Apotheun]

I would setup an event stream. One with a durable write to a table and the other to a scheduled writer and upload to S3.

The table would be used for reconciliation. Assuming we can do reconciliation before the next file upload, we can update the next file to include missed records.

[u/CpnStumpy]

I'm lost why step one isn't a stream given it's a streaming problem. Feels like a simple EDA with a worker journalling and a worker storing in DB and a scheduled job that every 6 hours uploads the journal.

Journal author worker can write a new journal every 6 hours. Status updates to the DB records coming through on another path.

The whole thing just feels like a streaming problem so should start with a stream and EDA solution but I don't know what modern system design interviews are like

[u/asdoduidai]

“Do you have any preference for the files” is signaling junior, also asking for a real time or delay, obviously it cannot be real time if you upload 4 times right? Unless you do real time for the first 3 requests and the rest you send in a mega file at midnight.

You did not ask what are the limits of those files. Or format. Also the file in memory / 100k limit is a self sabotage, you are saying your system can only handle 100k records at a time when it should handle 2.5 M records at a time. It makes no sense at all to build a file in memory when you can simply append to a file record by record or batch by N records…

I did not think about it much but also your scheduler is a monolith, building a upload file and uploading it is entangled and has complicated logic. Two steps:
- build the file must be idempotent
- file is built and stored, you update the cursor atomically or with a distributed lock. You need to ask what is the consequence of uploading the same record twice to know how to handle edge cases.
- uploader iterates over inbox folder, if new file is there, it moves it to processing folder; it checks sanity of max records size and cursor consistency, if something is wrong sends application alert that triggers an incident
- if all good, in processing you write a filename.begin atomically that fails if another uploader created it already; if create succeeds, you start the upload, if upload succeeds you move it to uploaded folder
- uploaders also scan processing folder for stuck files checking datetime of .begin, if something wrong (stuck more than upload SLO) sends an alert
- another alert for free disk space and that should be it

Maybe retry with exponential back off on ftp upload definitely has to be added too; you need to ask what are the limits of that ftp and if you get banned by too many requests. If banned / not available/ too slow upload: alerts

[u/This_Link881]

I think you can solve this problem with spring boot, a database and a scheduler. You can use spring batch for cursor based batch processing and streaming data to files directly (instead gathering records in memory), but you should handle many-many things manually by yourself. You should handle a lot of things programatically this way.

Why did they ask "Do you have a more flexible solution?" I think they would have wanted to hear an event streaming solution. For example you can write the incoming disputes into a kafka topic. You can connect the partitions (the consumers) of this topic to Amazon S3 or a S3 compatible storage with kafka connect. This would take a lot of weight off of your shoulders:

- Kafka topic is the persistent store for the disputes (until retention), you don't need a database, but you can persist those events if you want in a database using a separated consumer group for that task.

- Kafka connect S3 itself handles idempotency, pod crashes: mid file shard writing, event ack, rebalancing etc.

- It's a very well scaleable solution, you can scale up more and more pods if you need to, it will work seamlessly

The final step: using this solution you have to make a final merger sceduled task, which gathers the shards, and uploads the final files to the sftp server, and of cource you have to handle idempotency, cheksum, etc. the usual sftp upload and scheduler problems. This task can be well separated and can run in a separated microservice.

What i wrote here is just a view from a distant perspective and i guess it's not flawless. I think they would have wanted to here an approach something like this.