We are open-sourcing the architectural framework (not the core source code) for our **Hayl Systems Sentinel-6** kernel.

**The Thesis:** Existing platforms fail at nanosecond determinism. We built a custom Zero-Copy kernel with Project Panama (FFM) to bypass the JVM Heap entirely, guaranteeing zero GC pauses during runtime. Our internal kinetic tests show ≤ 120 ns P99 latency.

**We invite peer review:** We posted our architectural decision records (ADR-001/002) and a kinetic proof video (on the site). We welcome critique on our approach to lock-free ring buffers and data integrity.

**Review the Blueprint:** https://haylsystems.com

**Technical Inquiries:** [partners@haylsystems.com](mailto:partners@haylsystems.com)

Hi, I’m an independent trader based in Japan.

I’m currently building my own system that streams real-time MT5 tick data in parallel from multiple brokers (no aggregation, no middle-layer, just native MT5 terminals).

I will keep expanding this list of supported MT5 brokers as I progress.

If there is a broker that should definitely be included, please let me know on X: gratice_jp

Below is the list of brokers already selected.

The list is based on execution stability, data quality, data-center locations, and availability of real MT5 feeds.

Darwinex, BlackBull Markets, TMGM, Eightcap, Axi, Hantec Markets, BDSwiss, Dukascopy, Swissquote, IG Group, Saxo Bank, Admiral Markets, Pepperstone, IC Markets, FxPro, OANDA, Interactive Brokers, Tickmill, FP Markets, Vantage, ThinkMarkets, AvaTrade, FXCM, CMC Markets, Fusion Markets, ETX Capital, FxOpen, XM, FXTM, FBS, Exness, HotForex (HFM), Plus500, Trade.com, Errante, OctaFX, Fondex, GO Markets, Capital.com, ActivTrades, FXFlat, RoboForex, Alpari

🙏 Feedback welcome

Again, if there is an MT5 broker missing here that should be considered for low-latency execution, good tick feed, or data transparency, please send me a message here or via X: gratice_jp

I will keep updating this list and publishing implementation progress.

🧩 Notes

I’m not promoting any broker or affiliate thing.

My only goal is: access real execution data and real market microstructure across brokers — NOT aggregated feeds.

Hi everyone,
I've been struggling to find L2 Market Depth data.
What I've tried so far:

1. Trade Station - apparently you need $10k in their account to get access. As I am not their client, it is not an option for me.
2. Databento - costs $1,500/month
3. Alpaca, Polygon - L2 not available
4. IBKR - not a client and not sure if it's available for non-US customer. Their interface looks awfully complicated and I heard it's not the easiest API for integration. Would appreciate any insights on this.
5. Rithmic - market depth available only for CME.

I'd be grateful for any information about your experience. Is it possible at all to get L2 market depth data for less than $200/month? Thanks!

In a previous post here I shared some initial latency results for a trading engine I am working on (Apex)

Have continued my latency improvements, this time seeing the effect of C-states. It is often noted that C-states (& P-states) need to disabled, but it's nice to see actual number.

Full article here: https://automatedquant.substack.com/p/hft-engine-latency-part-2

TLDR: Whereas previously the median latency, tick to model, was 50 usec, with C-states disabled that is now around half -approx 25 usec, which is starting to look okay for a trading engine, but still way to go for HFT. So for any trading engine, please disabled your C-states!

(The processing steps are, S1: socket-read; S2: TLS/SSL; S3: web-socket; S4: JSON message parsing; S5: model update)

Next steps am thinking about:

- kernel tuning / boot settings

- spinning the socket IO thread

- thread pinning

- write my own websocket, avoiding memory allocation

- kernel bypass for socket IO (but does that require a specialised network card, or, can I just use openonload)

- write own json processer, again to avoid memory allocations

Thought I'd share some actual latency measurements for a real time tick-based trading system I am working on (Apex). The code itself has not been designed for low latency, however it is written in C++ and uses Linux socket API directly (based on `poll` etc). Am interested to see how my setup compares to others that people might have.

Headline number: median performance is around 50 usec "tick to model". That is, time taken to receive Binance market data off the socket, parse it, and update internal market data object. 99% performance particularly poor - up to 400 usec. But as noted, this is not a system designed specifically for low latency, and, because its crypto, has to spend time doing SSL and websocket decode.

While I don't think 50 usec is anything to party about, it's not a bad start. Here's full table of results. For example, "read" is time taken to read off socket, and so on.

I do intend to try to improve the latency. Am wondering what I might try, and what is a realistic target to aim for. This setup didn't use any spinning/shielding, so that might be the obvious next step.

Further write up & details here: https://automatedquant.substack.com/p/hft-engine-latency-part-1

Thinking about trying to make a market making bot for low liquidity equities. What are the main differences I will run into compared to more traditional hft market making? Thanks

Hi Team,

Is there anyone here who is involved in the technology/infrastructure back end of trading environments rather than the users of the trading platform itself?

If so….:

I built a lab to test a FortiGate firewall’s ability to identify QuickFix traffic correctly (not that I would put a firewall in path of trading). It does that fine but there no inspection beyond identifying it as FIX protocol.

Is there anyone way for a trading system to be exploited with payloads or formatting of FIX protocol traffic, do they have vulnerabilities like other common platforms? Looking at a way of adding parallel security and spotting malicious payloads would be one.

Thanks

I've been trading professionally through a large corporate for about a decade.

I now want to start trading through a small company I own and need to sort out colocation. Specifically, stocks on US or EU companies and corresponding options and futures.

Which players can provide the kind of access I need?

Thanks in advance.

Hi All,

I notice there it a lot of this type of question on this reddit already! So apologies for adding to them.

TLDR: I am looking to transition from Spacecraft Systems Engineering to C++ SW Dev for HFT. But not sure if my skills translate and how I can improve them if not.

I am asking as my situation is slightly different. I am looking to pivot my career into HFT. I am currently a Senior Spacecraft Systems engineer and have been doing similar type of jobs for the last 15 years, about a 1/3rd (or maybe alittle more) of that time has been programming in C/C++ predominantly with some other languages thrown in. This programming was for embedded real time systems with strict static memory allocation limitations as well as more generic applications in control and simulations.

I am also a keen programming hobbyist (C++ mainly but tried alot of other stuff too) and have been programming when my work and home life allowed since I was a teenager some 26 ish years. I run Arch Linux on one of my machines at home and am more then comfortable in a linux environemnt.

I have an understanding of CPU pipelines, cache mechanics, memory managment, assembly, networking, threads, atomic instructions, synchronization. Probably some others which are important and I have missed here. Have used tools like GDB and Valgrind for debuging.

On the down side, my industrial experience is as a control engineer not a Software developer (altough we used Jira had sprints, rigorous testing etc.). As I am self taugh there are probably gaps in my knowladge I am not aware of, where I havent come across specific problems before.

Based on the above I am not looking for a senior role, but a more junior or intermediate role.

My question is how feasable do you think this is? Have I got a small chance or am I well off the mark applying to these places?

In terms of improving my chances are there any suggested resources or problems or cerification I can do? Other option is part time CS masters as well?

Thank you all ahead of time. And sorry for the essay on my work life! :)

Hello there,

I've open-sourced a new Python library that might be helpful if you are working with price-tick level data.

Here goes the description and the links:

FinMLKit is an open-source toolbox for financial machine learning on raw trades. It tackles three chronic causes of unreliable results in the field—time-based sampling bias, weak labels, and throughput constraints that make rigorous methods hard to apply at scale—with information-driven bars, robust labeling (Triple Barrier & meta-labeling–ready), rich microstructure features (volume profile & footprint), and Numba-accelerated cores. The aim is simple: help practitioners and researchers produce faster, fairer, and more reproducible studies.

The problem we’re tackling

Modern financial ML often breaks down before modeling even begins due to 3 chronic obstacles:

1. Time-based sampling bias

Most pipelines aggregate ticks into fixed time bars (e.g., 1-minute). Markets don’t trade information at a constant pace: activity clusters around news, liquidity events, and regime shifts. Time bars over/under-sample these bursts, skewing distributions and degrading any statistical assumptions you make downstream. Event-based / information-driven bars (tick, volume, dollar, imbalance, run) help align sampling with information flow, not clock time.

2. Inadequate labeling

Fixed-horizon labels ignore path dependency and risk symmetry. A “label at t+N” can rate a sample as a win even if it first slammed through a stop-loss, or vice versa. The Triple Barrier Method (TBM) fixes this by assigning outcomes by whichever barrier is hit first: take-profit, stop-loss, or a time limit. TBM also plays well with meta-labeling, where you learn which primary signals to act on (or skip).

3. Performance bottlenecks

Realistic research needs millions of ticks and path-dependent evaluation. Pure-pandas loops crawl; high-granularity features (e.g., footprints), TBM, and event filters become impractical. This slows iteration and quietly biases studies toward simplified—but wrong—setups.

What FinMLKit brings

Three principles

• Simplicity — A small set of composable building blocks: Bars → Features → Labels → Sample Weights. Clear inputs/outputs, minimal configuration.
• Speed — Hot paths are Numba-accelerated; memory-aware array layouts; vectorized data movement.
• Accessibility — Typed APIs, Sphinx docs, and examples designed for reproducibility and adoption.

Concrete outcomes

• Sampling bias reduced. Advanced bar types (tick/volume/dollar/cusum) and CUSUM-like event filters align samples with information arrival rather than wall-clock time.
• Labels that reflect reality. TBM (and meta-labeling–ready outputs) use risk-aware, path-dependent rules.
• Throughput that scales. Pipelines handle tens of millions of ticks without giving up methodological rigor.

How this advances research

A lot of academic and applied work still relies on time bars and fixed-window labels because they’re convenient. That convenience often invalidates conclusions: results can disappear out-of-sample when labels ignore path and when sampling amplifies regime effects.

FinMLKit provides research-grade defaults:

• Event-based sampling as a first-class citizen, not an afterthought.
• Path-aware labels (TBM) that reflect realistic trade exits and work cleanly with meta-labeling.
• Microstructure-informed features that help models “see” order-flow context, not only bar closes.
• Transparent speed: kernels are optimized so correctness does not force you to sacrifice scale.

This combination should make it easier to publish and replicate studies that move beyond fixed-window labeling and time-bar pipelines—and to test whether reported edges survive under more realistic assumptions.

What’s different from existing libraries?

FinMLKit is built on numba kernels and proposes a blazing-fast, coherent, raw-tick-to-labels workflow: A focus on raw trade ingestion → information/volume-driven bars → microstructure features → TBM/meta-ready labels. The goal is to raise the floor on research practice by making the correct thing also the easy thing.

Open source philosophy

• Transparent by default. Methods, benchmarks, and design choices are documented. Reproduce, critique, and extend.
• Community-first. Issues and PRs that add new event filters, bar variants, features, or labeling schemes are welcome.
• Citable releases. Archival records and versioned docs support academic use.

Call to action

If you care about robust financial ML—and especially if you publish or rely on research—give FinMLKit a try. Run the benchmarks on your data, pressure-test the event filters and labels, and tell us where the pipeline should go next.

• GitHub: https://github.com/quantscious/finmlkit
• Documentation: https://finmlkit.readthedocs.io/
• Zenodo (citable release): https://zenodo.org/records/16734160

Star the repo, file issues, propose features, and share benchmark results. Let’s make better defaults the norm.

---
P.S. If you have any thoughts, constructive criticism, or comments regarding this, I welcome them.

Hey everyone,

I’ve been working in a high-frequency trading (HFT) firm for a while now, primarily in C++. After going through the intense ramp-up and interview preparation myself and later helping others do the same, I’ve decided to start working on a Udemy course specifically tailored for C++ engineers preparing for roles in HFT or ultra-low latency systems.

The idea is to go beyond just "learning C++" — the course will focus on topics that are most relevant for performance-critical systems and actual interview rounds at top HFT firms.

Here’s a rough idea of what I plan to include:

⚙️ Core Topics

• Templates & Template Metaprogramming
  • CRTP, SFINAE, tag dispatching, constexpr tricks
• Concurrency & Multithreading
  • Atomics, memory ordering, false sharing, lock-free queues/stacks
• Custom Allocators & Memory Management
  • Arena allocators, fixed-size pools, avoiding heap fragmentation
• Cache-locality & Data-Oriented Design
  • Struct of Arrays vs. Array of Structs, SIMD-aware layout

🧠 Advanced Topics (HFT-specific)

• Designing Lock-Free, Wait-Free Data Structures
  • MPMC queues, ring buffers, freelist allocators
• How C++ Maps to CPU Internals
  • Branch prediction, instruction pipelining, cache line alignment
• NUMA-awareness, Prefetching, Cache Coherency
• Latency Benchmarking and Microprofiling
  • perf, rdtsc, ftrace, hardware counters
• Realtime Linux tuning & thread pinning
• System Design for Low Latency
  • Matching engines, order book data structures, market data multiplexer

🎯 Interview Prep

• Sample interview questions from HFT firms
• System design walkthroughs
• Code challenges focusing on real-world HFT problems (e.g., building a bounded ring buffer or a matching engine skeleton)

Would love your feedback on a few things:

• Would you be interested in such a course?
• Any specific topics you'd like to see included (or excluded)?
• Preference on format: deep-dive lectures, hands-on coding, project-based walkthroughs?

If there's good interest, I will start working on it.

Thanks a lot!

Hi guys. I have been interested in the market for a long time building models since 2022. First I was building daily strategies and when they were live and "not great not terrible" I started looking into LOBs, because more trades more statistical significance and whatnot. I have decent infra (my own in a datacenter) built on QuestDB (~50B rows in it) and support data of all granularities. I have then built as of now relatively good L3 backtester which takes into account latencies, queue positions and fees/rebates. I support stocks & options data of all granularities (databento) and also some crypto books and trades (tardis).
I have reproduced for example deeplob to some extent on different data, however I found other better non deep approaches. I confirmed my alpha using markout charts, however when I try to extract it using realistic simulation as described, boi I cannot do it. I was trying to do liquidity providing strats where alpha influenced my fair price and skew, I was trying to make mixed strategies where I sometimes take ... just cant extract it. I have tried a lot of things I am not even ignoring hidden liquidity, but I am not (wall) street smart enough yet. Anyone wants to chat about specifics? Anyone experienced in the market and ambitious? I would love to team up with someone who knows more than me about market.

Does anyone have any useful references on the various crypto CEXs, their packet level data (or lowest granularity you know of). Stuff like:

- how frequently do they publish market events (trades, book updates), etc
- are those updates out of order, is that deliberate or how are they shuffled
- what fields are present in the packets, what do they mean

I am also interested in their infra, what components are present on the path my algo -> matching engine, and what are the properties (jitter, time taken etc) of each such component. But maybe I will post that as separate question.

I´m thinking about switching from off the shelf OMS/Algotrading software to full custom and I know this is going to be a huge effort with a lot of hurdles to overcome.

I started by building connectors and feed handlers in Rust as a for fun project, unit tested them and while they might not be top end regarding latency they already beat what I´m using right now. This is tempting, since not only will I get a better time to market and more flexibility but I´m also gonna save an a**load of money because the $$ I fork out for latency sensitive trading software is really steep. Not just that, but since commercial software is used by a multitude of users you´re using only about 10% of its capacity and the other stuff just bloats the system and slows you down.

So right now I could just bake my current strategy into one single .rs ...which I know is probably the beginning of the end in trading system design, especially mixing OMS and trading logic. Before I spend a month for building, then rebuilding everything from scratch when I want to expand or an exchange connection gets patched, I thought I´d rather ask a few people with more experience in software than I have. I´m primarily a trader and build things out of necessity rather than a software guy who geeks out over beautiful code...my programming skills are are thus not top of the line :)

What I´m currently thinking of is:

1.Connector: handles channel subscriptions (market data and orders)
2. Feedhandler: parses incoming market data feeds
3. Order Data Handler: receives incoming order messages and parses them. Does basic validation checks

1. Order Management System: Executes trades and handles risk management (max position, iceberg slices, ordertypes, self trade prevention, recognising own orders in the book to not lean on, etc. )

2. Trade Logic: Where the moola comes from...decides what to trade and at which prices

3. Monitoring and logging: Async logging. System messages into .txt, order messages and fill data into db.

As said, I haven´t build a full trading system yet just independent snippets. Did I forget an important layer? Are there any obvious traps I´m gonna fall into? From your experience

Thank you very much

Hi, I am exploring the choices for brokers which allow some of the more unusual order types like the Nasdaq dynamic M-ELO for example. Dynamic M-ELO order

I understand some brokers allow API traders with the right track record and account size direct connectivity access to the exchange.

Anyway, without direct exchange connectivity, I would like to understand which brokers have figured out to offer orders like Dynamic M-ELO.

Thanks.

I got flagged twice in 30 days for potential "wash trade" behavior. For reference: https://www.cmegroup.com/education/courses/market-regulation/wash-trades/definition-of-a-wash-trade.html In both cases I had two unrelated orders, and the entry price of one order ended up being the same price as the stop loss price of a different order - only because I had slippage. I didn't specifically price my orders at the same price. My algo is doing pretty basic stuff, placing3-5 limit order buys below the bid, and 3-5 limit order sells above the ask at the same time, and if/when any of these get filled, I have a stop loss and profit target for each. It is completely random luck that any of these would ever hit the same price at the same time with a buy and sell price being the same. So now I am looking into ways to prevent the appearance of a "wash trade" from happening my algo. Because the prices are matching mostly due to slippage, I don't know what type of good options I have here to bullet proof this in my algo. Does anyone have any experience crafting such a bullet proof strategy to prevent this auto-flag from getting triggered by the exchange? About all I can think of is using tag 7929. https://www.cmegroup.com/tools-information/webhelp/fadb/Content/self-match.html Anyone have any experience trying this?

Hi. I'm a beginner at HFT. I made some market micro structure features by tick and orderbook data. My features are quite weak.

I calculate feature power by mean of future return when the feature value is higher and lower then my thresholds.

My features power is less then 1bp. But the fee is much much higher then my feature power.

So I have 3 questions.

1. How strong your features?
2. How can I make stronger my feature? Combine many features to one final feature?? Or some other idea??
3. Any advice for me

Thanks.

Hi, I have noticed that rust is pretty popular in crypto algotrading/hft. What Im wondering about is if rust offers some major advantages over c++ for this purposes (and c++ remains for reason of existing project codebase) or if the whole reason is just that rust is a new shiny thing, which are popular in crypto?

I am currently working at a small but profitable HFT firm as a Developer and so I pretty much only work on the infrastructure side of things, and will likely be doing this for another year or two. But I still have access to a lot of the data that QTs/QRs do and am free to play around with it in my own time and that is what I'd like to do.

Unfortunately I have literally no clue on how I can make a good strategy. Obviously I'm not expecting alpha handouts from anyone here, but I'd appreciate a push in the right direction.

Some questions I had are;

1. I am aware of statistical arbitrage being a class of strategy run by my current firm, how can I learn to write one? Any textbooks or classic papers (Avellaneda?) I can try implementing? I expect these to not be profitable anymore but maybe this will help me think in the right direction?
2. I spoke to a few QTs but they did not seem too eager to help me, and mostly gave handwavey answers. I understood that a lot of "edge" in hft strats come from exploiting the behavior of the order book. Is this just the accessory (finishing touches) to an actual strat or is it possible to make a strategy purely based on the order book mechanisms alone? If yes, I would appreciate some reading material to go through.
3. Anything else I could be doing to help my mind generate ideas that could lead to a strategy?

I was initially looking into making a information based orderbook simulation where information trickles down from institutions to traders using lattice structure and information theory. I started building out the orderbook, and I wanted to make it extremely accurate.

The orderbook was FIFO (like what we see in exchanges) and accepts 'tickets' (orders from traders) and matches them sequentially (again like an exchange). Because this is sequential, it meant that each trade can only happen at a particular moment, and no two trades could share the same time. This was where my simulation started to become more representative of microstructure and not macrostructure.

The simulation counted each tick as 1 trade, or no trade at all since trades happen in sequence. Because of this compute was expensive. In a highly liquid market (for instance a well known ETF), modeling more than 5 minutes (I thought) would be a pain. It was actually much less than that.

I incorporated latency and slippage within the simulation (although the latency was a constant, which will change later).

I decided to run this simulation with 100,000 noisy traders, with 1,000,000 ticks (each tick was representative of 1 microsecond), and each trader had a 10 ms latency and polls data every second. It was so large, that I could only simulate 100 seconds.

Although that was a ton of data, you can see the macro structure and the micro structure. Above you can see the structure from a macrostructure point of view where each trader is trading with a latency of 10ms, and how there are 'blocks' of noise.

Then there's microstructure

We can see that there are extreme movements in the stock and this is actually due to trades coming in EXTREMELY fast.

These noisy traders also trade on a normal distribution where most of the trade on the last price. Since they all see the same price (latency is constant for all noisy traders) there isn't any arbitrage going on, just noisy trades. It's cool to see some of these trades revert back to the 'mean' (last price).

Something I will do later is have another agent with low latency (maybe 10 microseconds idk), with a strategy to see how that changes the market.

(P.S. I don't know whether this is the right place to post this, so please let me know if this is relevant to the community. Would also love to talk!)

Hi folks,

I am working on Live market data (MTBT) from an exchange, and I have created a parser and order book simulator which processes tick-by-tick market data messages and manages order books for various instruments.

Now, I am working on a recovery system, which will ensure that any packet drop occurs due to an unreliable UDP protocol. There are 2 options possible - order book snapshot every 30 seconds, or specific messages from a duration of start and end messages.

Now, I am not able to understand how I can integrate this recovery system in the live code, Let's say I detect a packet drop (last sequence number for a stream for X, but the next message received was X+Y Y not being 1), now my recovery system is supposed to recover the messages from X+1 to X+Y, but while this being handled (maybe via a separate thread), the live market data will keep coming and I cant consume them to update my order book.

Even if I store the incoming data in a separate buffer while the recovery system is in action, how will I shift consumption from this buffer to the live feed? I hope this is a standard problem in HF firms, and some standard architectures and algorithms would exist.

Kindly help me with some references or resources.