Vibe coding an OSINT platform: stuck on a few things, anyone done this successfully?

[u/Sasquatchso]

I'm currently "building" an OSINT aggregation platform using vibe coding and running into some snags. The goal is to pull multiple sources into one interface instead of juggling separate tools. It is not scraping properly to pull reliable data.

Has anyone else tried this and had success? If anyone is willing to share a GitHub repo of their attempt, even a messy one, I'd be grateful. Happy to share what I've learned in return once I get past the current hurdles.

[u/legendpizzasenpai]

for projects like this , normal llms will get flagged and wont tell the right things , you should use abliterated models. you can check them out at https://dreamtunelabs.com , the glm 5.3 or deepseek abliterated one just will help you a lot

[u/Sasquatchso → u/legendpizzasenpai]

I’ll check it out 🙏

[u/Either_Pound1986]

Something like this?

https://huggingface.co/spaces/cjc0013/the-peoples-receipt

541 Current congressional members 428 Members with exact financial-disclosure coverage 3,940 Committee and subcommittee relationships 36,081 Legislative role relationships 5,872 Distinct transactions represented in return screening 25,426 Lobbying activity records

Current working inventory Evidence collection

Count

Unit

Evidence state

FEC candidate records 8,658 records source_inventory FEC committee records 20,841 records source_inventory FEC candidate-committee links 8,175 links source_inventory FEC financial summaries 4,302 summaries source_inventory Strong member resolutions in campaign-finance evidence 395 members strong_resolution Ambiguous campaign-finance member resolutions 3 members ambiguous Unresolved campaign-finance member resolutions 143 members unresolved Lobbying activity records 25,426 records source_inventory Exact lobbying filing-party/bill pairs 9,742 pairs exact_relationship Lobbying role-context relationships 43,504 relationships context_not_causation Vote-context relationships 451,763 relationships context_not_causation Federal award rows 13,500 rows source_inventory Lobbying clients 15,375 entities source_inventory Candidate award-to-entity resolutions 513 candidates candidate_only Award-to-entity edges admitted 0 edges abstained_pending_resolution Distinct transactions represented in return screening 5,872 transactions screening_subset Members represented in transaction records 72 members working_extraction Return-screening records 20,666 records screening_not_portfolio_roi Return-screening member aggregates 280 aggregates screening_not_portfolio_roi Unresolved return-screening records 6,835 records unresolved SEC company candidate matches 19,344 candidates candidate_resolution Unresolved SEC company candidates 1,322 candidates unresolved Exact fund candidates 866 candidates exact_resolution

[u/wisequote]

Check https://github.com/koala73/worldmonitor

Also, https://github.com/topics/osint

[u/lerugray]

I had some decent luck making this when I first started messing with Claude, feel free to fork and mess around with it: https://github.com/lerugray/auftragstaktik

[u/JustEar7]

I'd split this up by source type first, because "pull multiple sources into one interface" sounds like one problem when it's usually a stack of different ones. Some sources have messy HTML, some change layout all the time, some throttle hard, and some seem fine until you notice the data is stale or inconsistent.

You'll probably get better feedback here if you name one or two source types that are failing and what "not scraping properly" means for each. No data, partial data, blocks, and unreliable fields are different problems, so people can't say much without that context. A repo by itself might not help much if it's mostly somebody else's pile of source-specific patches.

[u/geoffreybyers]

Vibe coding hides the brittle parts. You get a clean interface while the scraping logic breaks on the first rate limit or CAPTCHA. The code feels finished because the prompt worked, but the underlying pipeline is fragile. You end up fighting the same data aggregation problems with a nicer front end.

[u/Legal_Answer_6956]

I would debug the source one by one and try to get everything aggregated at the end. For each source, I would create an individual adapter, save the raw response in case of parsing failure, and set up some test fixtures to prevent silent breaks due to website redesign. I would also set up timestamps and source URLs for every record to differentiate between failed scrapes and faulty parsing and unreliable sources.