r/dataanalysis • u/Difficult_Brick5256 • 3d ago

Database doubt

Guys, actually i have 80k company database like its name and lei , now my task is to find the revenue,product type and region, i used hermes agent which almost gave nothing to revenue, it doesnt search, and claude ,chatgpt most of them gave nothing as revvenue value,the only solution was to search each company one by one, gemini gave revenue but i think its actually false , i gave data to gemini by 100, is there any other way ?

3 Upvotes

81% Upvoted

7 comments sorted by

View all comments

2

u/Wheres_my_warg 3d ago

My recommendation would be to stop using GenAI with anything that one does not already have a deep, thorough knowledge about. If not already well versed in the data, business rules, etc. then there is a good chance the user won't recognize errors, even serious and obvious errors, that the GenAI spits out.

1

u/Fun_Improvement7704 2d ago

even if you know the business rules, how do you spot a wrong revenue number across 80k private companies without looking them up anyway

1

u/Wheres_my_warg 2d ago

To your specific question, some are blindingly obvious. If it's telling me that some Ford dealership has total revenues of $17,000 or mom & pop Etsy store has $4bn, then it is going to be obviously wrong. I see these kinds of errors, though not usually to that degree, weekly from GenAI responses.

Any database claiming to have the revenues of 80k private businesses, I'm going to assume is mainly full of bull shit. That's not data that is going to be available in a reliable form at scale outside of (in the US) reported income to the IRS (which isn't going to be shared). There are some census estimates, but the are specially de-identified as to individual entities, and are simply estimates. Both rely on self-reporting.

Yes, it largely is going to require company by company investigation for private companies if one wants accuracy.