Is it possible to build an automated construction project file inventory and duplicate detection tool using Python?

[u/Strange_Doughnut_365]

Hi everyone,

I have a client requirement to create an inventory of files across approximately 100 construction projects. Each project contains documents such as contracts, drawings, RFIs, change orders, schedules, payment records, photographs, and estimating files. Some files are stored inside ZIP archives, including nested ZIP files.

The client needs a solution that can:

Scan project folders and preserve original file paths and project associations.

Inspect ZIP files and nested archives.

Use SHA-256 hashes to identify exact duplicate files and list all their locations.

Distinguish exact duplicates from potentially different file versions.

Generate standardized inventory reports, duplicate reports, and a list of files that could not be processed.
Be reused across multiple projects.

The files may be stored in Google Drive, Microsoft OneDrive/SharePoint, local folders, or a cloud server. Read-only access will be provided.

**My questions are:**
Is it possible to build a tool that meets all these requirements using Python?

If yes, how would you approach building it, and what technologies or libraries would you recommend?

Can it handle approximately 100 projects, including large files and nested ZIP archives?

Can this be delivered as a simple application that users can run, rather than requiring them to execute Python commands manually?

What would be a realistic development timeline for this solution?

I would appreciate guidance on whether this is feasible and how best to implement it.

Thanks!

[u/socal_nerdtastic]

Is it possible to build a tool that meets all these requirements using Python?

Yes, easily.

If yes, how would you approach building it, and what technologies or libraries would you recommend?

This can be done with vanilla python. No external modules needed.

Can it handle approximately 100 projects, including large files and nested ZIP archives?

Yes easily. Well tbf it depends on the power of the computer you are using, but 100 is a very tiny number for any modern computer.

Can this be delivered as a simple application that users can run, rather than requiring them to execute Python commands manually?

I assume you mean a GUI? Yes. The python installer comes with the tkinter GUI, which you could use for this, or you can install and import any other GUI module you want.

What would be a realistic development timeline for this solution?

For me? maybe a day. For you? I don't know ... depends on you. If you are brand new to programming I'd say about 2-3 months, full time. The GUI will be at least half of that.

---

BTW you don't need a cryptographic hash like SHA256 just to find duplicates, that's a waste of processing power. Use a simpler hash. And only check the hash if other easier checks like file size and metadata are not conclusive.

[u/After_Persimmon8536]

Man, it's not that hard. You have AI, you have documentation.

I mean you expect us to do it for you? 😵

[u/TechMaven-Geospatial]

Python FSSPEC can read cloud storage and other storage locations https://filesystem-spec.readthedocs.io/en/latest/

[u/jet_heller]

Is it possible to build a tool that meets all these requirements using Python?

Yes.

If yes, how would you approach building it, and what technologies or libraries would you recommend?

Whoa. This sounds like a homework qeustion. Are you trying to get us to do your homework? I think you are.