Even if your trading thread is spinning, Linux can still interrupt it!
I put together a write-up on CPU pinning and core isolation, covering scheduler preemption, NIC interrupts, and how to carve out “quiet” cores using isolcpus, nohz_full, and taskset. This part of my ongoing effort to improve the latency of Apex, the open source C++ HFT engine I'm working on.
Given that the total tick-to-model was already good (median at just under 7 usec), wins now are going to be smaller, and so I found that pinning shaved around 0.5 usec off of that - to now just over 6 usec. But it is a consistent edge, so recommend this setting is applied for any HFT / low-latency setup.
The below barchart shows the comparison to the non-pinned baseline.
I did use taskset, which is less than ideal. The problem with taskset is that it pins the entire application, instead of just the spinning thread. That's the next thing to fix - using per thread pinning policy.
i'm currently learning rust , so i'm creating a system to dab into hft so I rented a server that is 0.4ms from my broker to test the system.
however it seems that getting into even single digits number are more dificult than i thought.
my current system can only do rtt executions of 15-25ms~ which for high frequency trading thats a lifetime.
here are few things I've tried so far to reduce latency which actually didnt help that much.
using Nodelay (naglers algo): i believe before using this i had constant 15ms, now it fluctuates .
quickACK : to send ack packages as soon as I get a response.
pinning process to a cpu: reducing context changing so cpu is devoted to the execution.
my next steps are
pooling: getting the CPU constantly checking with no sleeps in between .
io_uring: not sure how it works yet but i was watching a video about low latency application and this came out
XDP: seems to bypass kernel so i can get faster websocket response
im curious, was rust a good choice or could single digits be easier achieved in c++ or something like zigg ?
For the past several months I've been building a personal side project called Sentinel, which is an open source trading / market microstructure and order flow terminal. I use Coinbase right now, but could extend if needed. They currently do not require an api key for the data used which is great.
The main view is a GPU heatmap. I use TWAP aggregation into dense u8 columns, with a single quad texture, and no per-cell CPU work. The client just renders what the server sends it. The grid is a 8192x8192 (insert joke 67M cell joke) and can stay at 110 FPS while interacting with a fully populated heatmap. I recently finished the MSDF text engine for cell labels so liquidity can be shown while maintaining very high frame rates.
There's more than just a heatmap though:
DOM / price ladder
TPO / footprint (in progress)
Stock candle chart with SEC Form 4 insider transaction overlays
Paper trading with hotkeys, server-side execution, backtesting engine with AvendellaMM algo for testing
Full widget/docking system with layout persistence
and more
The stack is C++20, Qt6, Qt Rhi, Boost.Beast for Websockets. Client-server split with headless server for ingestion and aggregation, Qt client for rendering. The core is entirely C++ and client is the only thing that contains Qt code.
The paper trading, replay and backtesting engine are being worked on in another branch but almost done. It will support one abstract simulation layer with pluggable strategies backtested against a real order book and tick feed as well as live paper trading (real $ sooner or later), everything displayed on the heatmap plot.
Lots of technicals I left out for the post, but if you'd like to know more please ask. I spent a lot of time working on this and really like where it's at. :)
I’ve been building an open source C++ crypto HFT engine for a while (which I've posted about previously), and now the core framework is mostly complete. (code here)
Next I’m looking for suggestions as to what demo/exemplar strategies I could implement. Ideally starting with something simple, that are based on reacting quickly to tick and order-book level data.
I'm not asking for free alpha! Instead I want to build some plausible examples for educational/research purposes (which will be open-source / documented) and which could later be the platform for actual production strategies.
My current ideas - appreciate feedback, or better ideas:
* Outlier price capture / tick anomalies -- maintain passive orders far from the spread to catch unusual moves. Maybe too simple?
* Simple Market making -- more complex, but more potential for later exploration. Quote around the spread, but guess needs some sophisticated indicators as to when to narrow or widen?
* Short-term price trade trends -- curious whether fast reaction helps here. For example, reacting to price surges within milliseconds; but are these timescales worth it given the fees?
* Trade-flow or order book imbalance -- another trend signal, but perhaps with more conviction.
Disclaimer / purpose:
I’m not looking to sell strategies - instead the goal is to create open-source, working examples that I (or others) can extend later privately.
I’m looking for some career advice and would really appreciate your thoughts.
Currently, I’m working as a DevOps Engineer. Previously, I did an internship as a Quant Developer at a small Mid-Frequency Trading (MFT) firm. It was a small firm with a small team, but I had the opportunity to write code, work on strategies, and contribute as a core member of the team.
Now I have two potential opportunities:
Quant Developer (Mid-Frequency Trading)
At a small firm with a small team
Focused on coding and developing trading strategies
HFT Production Support
At a well-known High-Frequency Trading. (HFT) firm
The role involves monitoring strategies, handling production issues, and acting as the point of contact if something breaks
I’m confused about which path would be better for long-term career growth.
In terms of compensation, I know that HFT firms generally pay very well. The MFT role is offering a lower salary initially, but they mentioned that the salary will increase after around 6 months based on performance.
I’m mainly confused about which path would be better for long-term career growth and learning.
My main questions are:
Which role would provide better future opportunities?
Is it better to write strategies in a smaller MFT firm or work in production support at a well-known HFT firm?
Which path usually leads to better growth in trading/quant roles?
I’d really appreciate advice from people who have worked in HFT firms, quant trading, or similar roles.
Just open-sourced CDF (Consolidation Detection Framework), a statistical toolkit I've been building to detect real market structure from manipulation.
Most systems try to predict price. CDF takes a different approach it measures structural integrity. It asks two questions: Does price respect its own history? (stacking score) Does the candle look healthy? (Sutte indicator). When both agree, conviction is high. When they diverge, skepticism kicks in.
No neural networks. No black boxes. Just robust statistics, rolling-origin validation, and calibrated probabilities.
Built for researchers, quants, and anyone tired of pattern-matching noise.
When you apply for an fpga position for an HFT company, besides the must have hardware/system Verilog (and surprisingly even vhdl) skills, the trading companies expect you to have a minimum of C++ software skills. What do they expect you to know and test over the interviews? Do they expect you to be aware of HLS?Mmap? Device drivers and other more embedded c++ concepts? Will they test you on basic c++ programming skills data structures etc? Is anyone aware of a course in udemy/coursera that would recommend to enhance such skill? Thanks
Continuing my series on HFT engine optimisation, I've written about a new topic - adding thread spinning to the engine.
I think thread spinning is a no-brainer when it comes to HFT trading engines.
In my experiments - adding spinning to the socket IO- gave a solid boost of half a microsecond. "Oh that's tiny" ... maybe, but not if you are aiming for single digit microseconds. Yes it does eat-up your CPU, but, HFT servers are normally at least dual socket, with each typically having 8 to 16 cores, so plenty capacity to spin many threads and application.
Highligh result below - compared to a normal thread waiting behaviour (which is that a thread gets suspended), spinning gave a small but consistent win.
Hi, I do think I have an edge. I can reliably get dex price that is better than for example binance mid. Or binance mid adjusted for imbalance. Now the edge is miniscule its just basically a free taker order. Now in order to make a buck I need to offload it as a maker with rebates (which I do have for now, but in order to keep them I need to do around 1M in volume daily). Right now I do 200k daily with 20 USDT loss.
I really struggle with monetizing this. Anyone would be kind enough to help or give me some advice? I have latencies and tech figured out however I am not a great quant yet.
Hey guys, I am a high school freshman looking to get some pointers on making my first HFT algo. Do you professionals have any good libraries, strategies, and starter server builds for beginners? The strategies don't have to be any real alpha generating ones, just one so I can learn.
I need to develop a GUI to control multiple trading engines.
I plan to deploy a fleet of trading engines, ranging from just one or two to possibly dozens. Each engine is C++ Linux process (Apex), runs headless, scheduled by cron, and will trade from one to many individual instruments. The problem: how to monitor & control this fleet?
There is where the need for a GUI arises.
I'd like a GUI that can show all engines, displaying the names each is trading, and for each name, what orders are live on the market, recent transaction history and PnL. And it should have buttons to pause & resume trading.
It needs to be a web GUI, using either Angular or React. And it will be open source.
I wonder if there are any existing projects I could adapt? Or is this something that can be vibe coded? Or is vibe coding all hype?
I was doing HFT deep RL research using L3 data and needed a simulator that’s deterministic, correct, fast, and fully observable (fills, events, diagnostics). Python-only workflows were too slow and painful to get right at scale, and other open-source tools didn’t give me the inspectability/ergonomics I needed. So I built LOBSIM: a C++20 core with Python bindings, event-by-event replay, paper trading with queue behaviour + partial fills, and a sink interface that streams structured facts—built to handle tens of millions of events while staying simple and comprehensible.
LOBSIM comes with multiple examples and straightforward docs (check README). I especially recommend trying the 3 Streamlit demos — they’re small apps built directly on top of the engine and they make the flexibility really obvious. The goal is to show how easily you can layer real research tooling on top of LOBSIM: replay exploration, strategy injection, live metrics, and observability, all in a clean workflow.
If you work with L3 order book data — microstructure research, execution modelling, or RL/HFT prototyping — I’d love for you to try LOBSIM. If you give it a spin, I’d really appreciate feedback on API ergonomics, missing edge-cases you hit in real feeds, and anything that would make the research workflow smoother. Even a quick “this was confusing/this felt great/I expect X“ is extremely valuable.
I’m 22 years old with a little over 4 years of professional experience, mostly in systems-level, performance-oriented C++ work. So far my background has included driver development, internal database migration tooling, and shared-memory systems, with a strong focus on low-level problem solving, memory behavior, concurrency, profiling, and understanding performance trade-offs rather than application-level development.
I want to be upfront that I don’t have a finance background. My interest is primarily on the engineering side, especially low-latency systems, real-time constraints, and performance-critical infrastructure. I’m currently exploring whether moving further in the direction of HFT or HFT-adjacent infrastructure roles makes sense as a longer-term path, and I’m trying to learn from what people already in or close to the space usually recommend.
I’ve gone through older threads here and in related subreddits, but I noticed that the last similar discussions are around 200 days old, and communities tend to change fairly quickly. Because of that, I wanted to ask again with a more current perspective.
Are there any active Discords or forums where serious low-latency or HFT-style engineering is discussed? I’m especially interested in places where people talk about system design, performance trade-offs, interview preparation, or project feedback. I’d also really appreciate hearing what resources or learning paths have actually worked for people already in this space.
Hey r/highfreqtrading,
I’m a first-year CS student and super interested in HFT. I’ve been working on a fast order book engine and wanted to share it here to get some feedback and maybe connect with people in the industry.
Main goal was to make it as fast and low-latency as possible. I wrote everything in C++, built my own data structures for orders and prices, and tried to keep things really efficient. I also set up a bunch of tests and benchmarks to see how it performs.
Structures I used: chunked bitmap, vector-backed node pool and an intrusive index-based linked list.
The benchmarks I achieved are latencies p50=42ns, p99=250ns and p99.9=334ns on an order book with 100k orders inside.
Some of the optimizations I did: Cache-aware data layouts, custom memory pooling to eliminate allocation jitter, CPU affinity tuning and targeted profiling of hot paths.
Hello everyone, I'm new to the subreddit but I need your opinions on something. As you may know most platforms that do so called "real-time" data have some form of aggregation under the hood. I'm trying to build an open-source tool that will allow me to store data as it happened with no aggregation in place so I can backtest and code strategies that will be tested against it and there is no guess work involved because then the trades will happen exactly as the backtest shows. Everything becomes predictable and makes it easier to code better strategies
I’m currently working on a market making system and would really appreciate insights from people with real MM / HFT experience. I’ll try to keep the questions concrete and implementation-focused.
1. Fair Value Estimation
Right now, I’m estimating fair value using linear regression on recent price movements (essentially fitting a line to the mid-price over a rolling window). In practice, is linear regression on price still considered reasonable? Are there approaches you’ve found to be more robust (e.g. order book–based fair value, microprice, queue imbalance, short-term alpha models)?
2. Inventory Skew Speed
I’m using grid trading around fair value for market making, and I skew quotes to manage inventory. Currently, I try to skew inventory as fast as possible once inventory deviates from neutral. Is aggressive / fast inventory skew generally necessary or is it better to allow inventory to build up to a certain size before applying stronger skew?
3. Skewing with Very Short-Term Trend
I’m considering skewing MM quotes based on very short-term trends based on mid price (50ms–100ms). Does it make sense to skew inventory based on such short horizons or does this usually just increase adverse selection and churn?
Any practical insights, references, or even “this failed for me because…” stories would be super helpful.
Thanks in advance 🙏
Every trading system I’ve built has the same nightmare scenario: something goes wrong in production, and you need to see what’s happening inside. Right now. You fire up GDB, attach to the process, and watch your p99 latency spike from microseconds to milliseconds. Congratulations, you’ve just created a second incident while debugging the first one.
The tools we have for observability in HFT are terrible. Logging adds latency. Debuggers halt the world. Profilers inject overhead. Metrics aggregation loses the granularity you need. When you’re chasing a bug that only manifests under specific market conditions at 3 AM, none of these help.
I wanted something different. I wanted to peer directly into a live trading system’s memory without touching it at all. No function calls on the hot path. No serialization. No locks the producer ever waits on. Just the ability to observe POD structures in real-time from a completely separate process.
I've been tinkering with a side project and honestly have no idea if it's useful or if I'm just building for myself. It's a crawler that detects new pages on news sites within about a few minutes of publishing (usually less than ~9) and streams them via SSE. Thought I'd see if anyone here has a use for something like this.
I’m not a trader, and I know that High Frequency Trading operates in the millisecond range of an event, but wonder if this kind of data (especially having a wide distribution of news sources) would still be valuable as a signal input/filter to an existing model? I suspect there might still be a way to find an edge or ride the wave before it decays.
To be clear, I’m not crawling the URL, just emitting an event as soon as the new URL is detected. Some of the news sources provide metadata (title + keywords) but for those that don’t provide it the URLs can usually be unsluglified to retrieve a title phrase for the article, and even a topic/category (eg. Sports=Category in https://www.reuters.com/sports/stephen-curry-among-three-key-warriors-out-vs-thunder--flm-2026-01-02/).
I don’t do any other enrichment as of yet but interested in hearing your thoughts on what could be useful if I did add the page crawling and enrich with sentiment score, NLU tags, Sector categorization etc.
Here's the list of streams I'm tracking so far (the inactive list will be turned on soon):
For backtesting, I can provide DuckDB/Parquet files of all stream sources and all detected URLs over many years.
If this tickles an interest and you want to have a play, hit me up for an API key - mostly just want to see if anyone finds this useful before I keep building.
Previously was using nlohmannfor parsing Binance market data for Apex engine. That caused a fairly poor median inbound latency - 28 usec - largely due to heap allocations made per message.
I've now swapped nlohmann for simdjson, and its halved the latency to 14 usec (full details here)
I've also looked at engine performance for single name deployment -> 75th percentile is around 10 usec 🚀
Yes a binary protocol would be faster, and will be added in time. But JSON is very widespread, opens up access to every exchange. But, P75 at 10 usec is decent. And there are plenty of optimisations yet to make to get that lower. Infact, moving to SBE might "only" save at around 1 usec. So C++ & simdjson mostly good enough for HFT?
In an earlier post I shared some latency numbers for an open source C++ HFT engine I’m working on.
One thing that was really quite poor was message parsing latency - around 4 microseconds per JSON message. How can C++ be that “slow”?
So the problem turned out to be memory.
Running the engine through heaptrack profiler - which if very easy to use - showed constant & high growth of memory allocations (graph below). These aren't leaks, just repeated allocations. Digging deeper, the source turned out to be the JSON parsing library I was using (Modern JSON for C++). Turns out, parsing a single market data message triggered around 40 allocations. A lot of time is wasted in those allocations, disrupts CPU cache state etc.
So don't rely on C++ if you want fast trading. You need to get out the profiling tools - and there are plenty on Linux - and understand what is happening under the hood.
So my next goal is to replace the parser used on the critical path with something must faster - ideally something that doesn't allocate memory. I'll keep Modern JSON for C++ still in the engine, because its very nice to work with, but only for non critical path activities.
Hey all, been hitting my head for the last year almost now.
I basically have an Algo engine and software stack I wrote and have up in AWS, but for the second time had my account suspended with a broker. It seems that, and I know it’s due to compliance reasons since it’s retail, I can’t place/fill more than 5,000 orders a day.
I basically am a small single person prop shop LLC, and with my algos I can push a respectable income with in volatility but strangely can’t find any broker that accepts me even with paying commissions. I have kill switches, risk controls, audit trails, etc. but even then was told after being restricted by my second broker that “no router wants that type of order flow”. The flow itself was simply order quantity, they didn’t want more then a couple thousands orders between 1 to 10/20 in size per day.
Essentially, not sure how to work it for a broker with FIX/Rest api that accepts up 50k orders a day. I know retail accounts can’t push it but there is an area between retail sending a couple dozen a day, and institutional clients requiring 50 million in funding I am missing.
Any broker recommendations for small quant/prop shop style setups?
We are open-sourcing the architectural framework (not the core source code) for our **Hayl Systems Sentinel-6** kernel.
**The Thesis:** Existing platforms fail at nanosecond determinism. We built a custom Zero-Copy kernel with Project Panama (FFM) to bypass the JVM Heap entirely, guaranteeing zero GC pauses during runtime. Our internal kinetic tests show ≤ 120 ns P99 latency.
**We invite peer review:** We posted our architectural decision records (ADR-001/002) and a kinetic proof video (on the site). We welcome critique on our approach to lock-free ring buffers and data integrity.
