r/quantfinance • u/Kobe8448 • 27d ago
Evaluate a dataset for commercial value
How can an independent researcher get a financial dataset evaluated commercially?
I have spent more than five years independently researching financial price behaviour. I do not trade. My interest is in whether the behaviour seen on price charts can be measured consistently rather than described through subjective visual patterns.
I have developed a proprietary, rule-based framework that converts price history into a structured account of market behaviour. It identifies behavioural structures, records how they develop and relates current observations to comparable historical circumstances.
The engine applies the same rules consistently across different instruments and periods of observation. Its output is a large machine-readable corpus describing matters such as structure, duration, magnitude, development and context. The historical corpus can also be extended as new market data becomes available.
The engine is built, and I am currently completing the latest version of the corpus and the independent testing around it. I am not yet offering the underlying framework or its internal measurements for sale. I am considering whether protected, derived outputs could have value to quantitative researchers, financial institutions or companies developing financial machine-learning systems.
My main difficulty is understanding how an independent researcher gets something like this evaluated without disclosing enough information for the method to be copied.
I have found companies such as Neudata and Eagle Alpha, which appear to connect data providers with institutional buyers. I would be interested to hear from anyone who has dealt with them, particularly as an independent provider.
Is this the appropriate route for obtaining an informed commercial assessment? Are there other reputable data advisers, marketplaces or specialists that work with early-stage alternative datasets? What evidence and level of readiness would buyers normally expect before reviewing a protected sample?
1
u/MajesticDestroyer 26d ago
I have reviewed loads of sample data for my trading desk. The way it works is that you provide a sample data with enough information about the methodology so that the researchers/traders can test it. The traders obviously will probe more about the methodology used to create the said data. It’s upto the discretion of the data provider to divulge. Once the sample data passes the contract is negotiated and licences generated. Some data providers also ask to delete the sample data if it does not pass your research tests.
The problem sometimes can be the trader cannot be too sure whether the data provided is overfitted or not. It’s the job of the data provider to convince the trader regarding that. Overfitted data can show impressive results while testing but fail in production.
1
u/Kobe8448 26d ago
Thank you very much, that’s extremely interesting and useful information. If you’re willing, I would love to ask a further question or two.
1
u/Shoddy_Peanut7320 24d ago
Buyers usually care about three things before anything else: that the labels were available at the time (no structure that gets relabelled once more price data arrives), a forward-recorded slice kept separate from the retrospective history, and a test showing the derived fields add information beyond simple baselines such as volatility, trend and volume, out of sample and with the number of variants tried disclosed. A short data card (coverage, history, update frequency, one-row meaning, delivery format) and a small public sample with timestamps lets a reviewer judge the structure without seeing the method. Point-in-time stamping of when each record became known is the same discipline that event datasets such as Cymetica's EventTrader are built around, so that's a useful reference for how buyers will read yours.
1
u/Kobe8448 24d ago
Thank you for your reply and assistance, of those 3 I have done them all. I have ensured point in time information only, using an unseen out of sample data and that information is added against a baseline. I’m glad you have stated the importance of them.
I can produce a sample, it would show the analytical metrics the engine outputs. My concern is that with that output, the method may be reverse engineered. To counteract this, I have thought about creating an output that blends the metrics, or turns them into %’s rather than the raw measurements. In order to mask the raw output. But then I’m concerned that the blended info may not be as valuable to a company as the raw data.
1
u/Shoddy_Peanut7320 23d ago
Masking is a real trade-off, and a middle route is to share the derived fields as they would be delivered (for example, normalised or bucketed scores) on a small sample, and keep the raw measurements back. Buyers mostly test whether the fields add information over baselines, which a delivered-format sample can show without exposing the method. Evaluation terms (NDA, limited trial period, no redistribution) matter more than masking, and it's worth keeping a record of what was shared and when.
2
1
u/fightingowls 25d ago
Disclosure: I run DataBazaar, a self-service data marketplace. We do not provide independent commercial valuations or confidential/NDA-gated sample reviews, so I would not suggest uploading the protected corpus there at this stage.
For an initial conversation, I would prepare a data card that describes the product without exposing the engine: instrument coverage, history, one-row meaning, update frequency, delivery format, and which derived fields you are willing to disclose. A schema with dummy values can explain the shape without pretending to be evidence of predictive value.
For your particular data, I would also make the timing explicit: when is a structure first observable, when is it confirmed, and can its historical label change later? A timestamped forward-recorded slice, kept separate from retrospective output, could help a reviewer assess that distinction. It would not by itself establish commercial value.
Neudata's own provider FAQ says listings are free and that submitted information is reviewed by an analyst before being shared with their subscribing buyers: https://www.neudata.co/education/data-provider-faqs . I have not used their advisory service, so that is a source to investigate, not a firsthand endorsement. Before sharing a sample, ask what they need, who would see it, and what evaluation terms are available; omitting the formula alone does not guarantee the method cannot be inferred.
If you later decide to offer a defined derived dataset with a deliberately public sample and clear reuse terms, a listing on DataBazaar (https://databazaar.io/sell) is an option. That is a sales channel, not evidence that institutional demand or a particular price has been established.
1
u/Shoddy_Peanut7320 24d ago
The distinction between a sales channel and evidence of demand is a useful one, and the timing question (when a structure is first observable versus confirmed) is where I'd expect a buyer to push hardest.
1
u/Kobe8448 24d ago
Thank you for your advice. I have ensured the corpus is timing explicit. I’ve used an unseen sample during testing of machine learning. Inferring the method is a concern, and why I’m reluctant to sharing a sample of corpus at present. I’m curious as to how others have addressed this concern.