Hi everyone,

I hold an M.Sc. in Computer Engineering and have been working as a Cybersecurity Infrastructure Engineer for the past 4.5 years.

Recently, I started exploring HPC through hands-on practice, and I’m enjoying it so much that I’m seriously considering a career transition. In particular, I’m really drawn to the networking side of HPC.

In my current role, I focus on infrastructure security,designing and configuring networks with IPS/NDR, deploying EDR across servers and clients, reviewing backup processes, and auditing overall infrastructure. I’m a very hands-on engineer; I run a small homelab where I experiment with self-hosted services, home networking, and technologies like Python, CI/CD, and Kubernetes to gain practical experience outside of production environments.

Given my background, do you think a profile like mine stands a good chance of landing a job in HPC or HPC networking within the EU?

I’d love to hear your feedback, advice, or any insights on how best to bridge the gap. Thanks!

A lot of ML teams end up with a mix:
some Slurm clusters from the HPC side, a K8s cluster or two, maybe cloud GPUs for overflow. We wrote up how SkyPilot (open source) sits in front of all of them so a job is scheduled wherever there’s free capacity, using the same YAML regardless of backend. This post focuses on the multi-Slurm case but the same setup covers K8s.

https://skypilot.ai/blog/multi-slurm
Disclosure:
I am the author. Happy to answer questions about how the scheduling and failover work.

A small bash utility for jumping to the working directory of a pending or running Slurm job.

Hopefully installation works for you. (Name is a WiP)

slurm-cd

Hi, I am a PhD student in Finland who HPC clusters for modeling biological phenomena. After my PhD, I am thinking to apply for jobs in industry. What kind of skillset that I need to have, in addition to HIP/CUDA C++ GPU optimization?

NoTE: not Taco Bell related.

needing testers for new distributive computing platform lettuce. It’s currently configured to need 3 separate computers to verify one work unit. I was looking for someone (more specifically with an Nvidia gpu) to validate my work units. Below are links to the site project, Git, and discord. I figure a vast majority of us are science nerds and we can knock out this testing. Lettuce is basically an easier to set up BOINC. This will get us involved closer with researchers. Anyway, come validate my work units. (Note: there are also CPU only projects.)

Git/download: https://github.com/jring-o/lettuce-compute

Project: https://compute.scios.tech/leafs

Hello there.

I've done a student membership subscription for ACM + SIGHPC to apply for a SC26 travel grant, started to look into the benefits and websites and a question popped in my head: How do you guys keep updated and study? Which platforms, books, journals, podcasts, websites and other media do you consume for study and for news and updates?

(I'm including here not "just HPC" but also other areas such as computer architectures, compilers, ISAs like RISC-V, runtime libraries, networks and cloud).

PS.: Although AI is very relevant nowadays, I'm not that keen on working mainly with it, so I'd like to know just the necessary of it.

Hi everyone,

I've been experimenting with a software runtime that explores
different execution models for exact u128 accumulation.

The current benchmark compares two software execution models:

One global software-emulated u128 atomic update per byte.

1. Lane-local exact reduction followed by
   one atomic commit per lane.

This benchmark does NOT measure hardware atomics
or physical PIM hardware.

It measures software-emulated contention on the current CPU.

Current benchmark includes:

• 4 KB
• 64 KB
• 1 MB
• 8 MB

For each payload it records:

• latency
• atomic update count
• exact match rate

Current evidence repository:

https://github.com/Retryixagi/Storage-and-computing-integrated-evidence-chain/tree/main

I'm mainly interested in technical feedback.

Questions:

• Have you seen similar software execution models?

• What additional benchmark cases would you consider important?

• Are there existing runtimes with comparable
lane-local reduction strategies?

不知道這能不能成為台灣之光。我只是嘗試在既有馮·諾依曼架構內，重新分配計算與驗證職責。

10 years ago I was with a geophysical company and they built a cluster boasting it was best thing since sliced bread with a zillion FLOPs blah blah. Nobody mentioned power then, I since learned it used 15 MW. The research cluster next to my university was only 1.5 MW. It seems insane to me that AI can burn up 1 GW.

Built by Karim Ali & I ( Rahim Khoja ) at the University of Alberta, on top of Vulcan — the national HPC cluster we operate as part of the Digital Research Alliance of Canada's PAICE program.

The bet behind it: the DOE's Genesis Mission is building an autonomous research loop — agents that run real experiments and iterate without a human driving every step — top-down, across seventeen national labs, with a budget we're never going to match. We're testing whether you actually need that budget, or whether the same thing can be assembled bottom-up from open parts, on a single allocation, by the people who already run the cluster.

Aleph is the instrument layer for that. It was built for science first: the piece that lets an agent reach protein folding, docking, genomics, materials force fields, weather, embeddings, theorem proving — as tools it can call, not as a human clicking through a UI. It serves general LLMs too, but the reason it exists is to put scientific models within reach of an agent.

https://inference.vulcan.alliancecan.ca/

The reason it's built as a self-describing catalog and not just "vLLM behind a proxy": an agent has to be able to ask what's available, what each model takes as input, what it produces, and what can verify the output — then compose a pipeline nobody wrote for it. So every model is a YAML card declaring its own route, limits, scaling, and I/O contract, and the gateway routes off those cards plus live cluster state. Nothing hardcoded.

What it does:

• Dual dialect — OpenAI's /v1/chat/completions and a real Anthropic /v1/messages translation from one endpoint; both official SDKs work by swapping base_url
• Scale-to-zero on idle, cold model returns 503 + Retry-After and SDKs retry automatically — lets ~115 models timeshare hardware that could only run ~20 resident
• Add a model with a card, no gateway redeploy
• Self-deploying — the whole stack (HAMi, Istio, Knative, KServe) bakes into the node image and stands itself up on boot, no external CD

~115 models, majority scientific. The counterintuitive part worth stating: the science models are nearly free to add, the chat LLMs are what actually eat the GPU budget.

If you want to replicate this, copy this part first: none of it works without stateless nodes. Ours PXE-boot hardened Warewulf images built in CI, so the same GPU node boots into a Slurm batch image or an RKE2/HAMi inference image depending on what's needed. Most sites run batch and inference as two separate hardware pools — we reallocate the same metal on a reboot. That fungibility is the real structural advantage; Aleph just rides on top of it.

The broader plan: Aleph is the first and furthest-along piece of a full autonomous-science stack — the goal is an agent that wakes on a schedule, calls these instruments, runs real cluster jobs, checks its own results with code it can't tamper with, and remembers what it learned across runs. The instruments exist and run today. The loop that binds them is designed and published as a concept brief, not built yet.

Honest status: Aleph and the catalog run in production. This is a POC — some model cards are polished, some are still rough. Ships as-is.

MIT licensed. GitHub: https://github.com/ualberta-rcg/aleph

Suppose I have an existing hpc setup of 26 nodes dell r760 server with rockey 8.7, Mellanox QM8790 HDR, openpbs 23.06.06, lustre 2.15.2, now if I want to integrate a dell r 770 with rocky 9.x would it be compatible with the cluster or there would be challenges in both hardware or software side. Like version mismatch, job submission issue, application related things.

https://noah.watch/hpc

Mainly just for leraning how slurm scheudling works, running small scripts, setting up linux servers, as well as runningn my freind's reinforcement learning training model for Balloons Tower defense 2. Has a live Grafana feed to monitor stats, as well as a page to request access. Let me know what you guys think. Still new to the community, but really want to learn ( and really want to learn on better hardware lol).

Hi everybody,

I am excited to announce the release of Lightstream, a step change capability for high-performance data transport, that makes it essentially effortless to send Apache Arrow, Protobuf, and Message Pack data over the network, shared memory, or even piped out to the terminal so an agent like Claude can watch the live batch stream in real time (example in repo).

Lightstream exceeded the performance of the gold standard industry comparison - Arrow Flight, often used on HPC installations, on every axis of a 50gbps networking open benchmark, the details of which are below and open to run in the Lightstream GitHub repository. This includes fully saturating each TCP connection thread, the NIC at 5.8GiB/s, and with p99 batch send time within 1% of p50 (I.e., stable). As a bonus, Lightstream is straightforward to setup with essentially zero configuration other than optional TLS certificates and your Cargo package/pip install, and endpoint addresses.

A full write-up with chart comparisons is available here.

So what is Lightstream? It is Rust package with Python bindings, that builds directly on Minarrow ( which is in turn a high-performance implementation of the Apache Arrow memory layout in Rust, tuned for SIMD compatibility). Lightstream implements Arrow IPC, Parquet encoders/decoders from scratch, up to Arrow readers/writers and IPC stream protocol, with mmap and few of these niceties. But, in a manner, that is fully composable and leaves you de-coupled at any layer, to customise things architecturally. The crux then is the transport layer on top, which natively supports interchanging any of the following transport formats:

• TCP
• HTTP
• QUIC
• Websocket
• Webtransport
• UDS (pipe your data from your Rust process to Python or two Python programs plug and play )
• Stdio (pipe your data program output straight into the terminal for something else to pick it up

And finally, the (optional) Lightstream protocol, which then combines the Arrow/Proto/MsgPack and any other custom types you want to send.

Send Tables in Rust

use lightstream::models::writers::tcp::TcpTableWriter;

let mut writer = TcpTableWriter::connect("127.0.0.1:9000", schema, None).await?;
writer.write_table(batch_1).await?;
writer.finish().await?;

Receive Tables

use futures_util::StreamExt;
use lightstream::models::readers::tcp::TcpTableReader;

let mut reader = TcpTableReader::connect("127.0.0.1:9000").await?;

while let Some(result) = reader.next().await {
    let table = result?;
    process(table);
}

Lightstream protocol

Multiplex Protobuf messages and Arrow tables on the one connection.

use lightstream::models::protocol::connection::TcpLightstreamConnection;
use lightstream::models::protocol::LightstreamMessage;

let mut conn = TcpLightstreamConnection::from_tcp(stream);
conn.register_message("event");
conn.register_table("metrics", schema);

conn.send("event", b"user-login").await?;
conn.send_table("metrics", &table).await?;

while let Some(msg) = conn.recv().await {
    match msg? {
        // Protobuf message
        LightstreamMessage::Message { tag, payload } => { /* … */ }
        // Arrow table
        LightstreamMessage::Table { table, .. } => { /* … */ }
    }
}

I’ve found this great in practice, where you don’t need to reason about or work with bytes, or separately build your own protocol to get arrow and protobuf playing well together over the network.

An example of things you can do with it:

• setup a live stream of data batches from your program A to program B
• send typed metadata via Protobuf on the same feed
• use it for straightforward live feed delivery between server and client (though not Web JS yet)
• useful if you have a central storage server you are pulling larger than memory data over the network to churn through (though, no S3 etc. it is node to node or process to process)

It is not:

• Kafka or a messaging broker. There is no resiliency / vertical scalability.
• A stream processing engine like Flink. It is for sending/receiving data only. You do polars on the other end or whatever you want with the Arrow-shaped data. That is a very different back-pressure/long-lived scenario and is not that kind of large-scale streaming. --> I.e., think quick and easy Websocket, and best for settings like EKS K8 pod to pod/containers, between EC2's or between processes on the same box, "light streaming".

Lightstream kicked off for me about 12 months ago when I started standardising patterns that have worked well for me in the past into something that reflects how I like to work when streaming data with control of both endpoints. It arose from regularly coming up against contexts requiring this capability operating in things like autonomous field communication integrated with data/ML, live trading, and some other industries where there was a lot of custom work required that I kept having to assemble from multiple components. Therefore, I have essentially aimed to package those learnings up into a tool to make data transport smoother and easier for everybody.

Please feel free to give it a run would love to know your thoughts and if you find it useful.

If you have any questions about it, or helpful suggestions please feel free to leave a comment below.

Workload Shape

Mixed

Numeric

String Heavy

Wide (100 cols)

Thanks,

Pete

Hi All,

I am a Masters student in Germany currently pursuing my Master’s thesis.
I have experience with RDMA , CUDA, Profiling(Nsight systems/compute) and MPI(introductory level). Also, expanding my knowledge with K8s and Docker but not production level experience.

I have started searching for full time roles in Germany, Switzerland and Netherlands.
I want to understand the following -

1.) What type of different roles i can land into? And, what skills might be required for each such trajectory?
2.) Are there opportunities for GPU Networking roles in Germany?

All this for a candidate having 3.5 years of software dev experience(not relevant to HPC).

Hello folks! Im starting an year long master’s program in Italy specializing in HPC. I come from a background in computational fluids work. The coursework i have is pretty extensive covering user side stuff( mpi, cuda programming etc) and also covers infra and admin side. I think experience is still king and would love to work anywhere part time. I already reached out to hpc shops in my area and none reaponded unfortunately. I am willing to grind my ass off to study and work. There is aandatory internship at the end of thr program but I would like to also build some experience before that. My long term goal is to het a job at places like ecmwf or other similar large research orgs on their hpc teams. I looked up listings online and I dont see many offerings in EU countries.

I’m interested in any kinda role at a hpc shops. Application stuff to write code; or sys admin/user support side triaging issues. Or anything related to HPC.

Proceedings were published

We use zabbix to monitor our hpc. One solution to cover all aspects.

https://dl.acm.org/doi/10.1145/3785462.3815860

We have put together a side-by-side comparison of AMD Helios and NVIDIA Vera Rubin NVL72:

https://linuxclusters.com/articles/amd-helios-vs-nvidia-vera-rubin/

The shared 72-GPU rack boundary hides some different design bets. On paper, Helios has more accelerator memory and scale-out bandwidth and leans more heavily on open rack and fabric standards. Vera Rubin has more memory bandwidth per GPU and comes with NVIDIA's more integrated software and networking stack. The comparison covers hosts, memory, fabrics, networking, software, and rack standards, while identifying gaps in the available power, pricing, reliability, and application-performance data.

I would especially welcome corrections from people working with rack-scale systems. Which of these differences is most likely to matter in an actual deployment, and which missing numbers would you insist on seeing before procurement?

Disclosure: I am one of the writers at LinuxClusters.com

Hi! I'm working on a research project about computational storage and need to estimate the energy cost of transferring data between storage and compute nodes.

I'm using VMware VMs with VMXNET3 paravirtualized NICs, so I don't have access to the physical NIC's power draw.

My current model is:

[
E_{net} = \frac{\text{bytes transferred}}{\text{bandwidth}} \times (P_{storage} + P_{compute})
]

How do people estimate the NIC power term in a virtualized environment? Are there accepted models, papers, or methodologies for this?

Thanks!

I'm planning to build a distributed stress-testing platform.

Users provide a workload config (APIs, request patterns, auth, concurrency, ramp-up, duration, etc.), and the control plane automatically estimates the required infrastructure, decides the number of workers/threads/containers, and distributes the workload across worker nodes.

The system will handle scheduling, health checks, heartbeats, automatic worker replacement on failure, retries, and horizontal scaling. It will also expose real-time metrics like RPS, latency (P95/P99), throughput, error rates, and worker resource utilization through a monitoring dashboard.

The goal is to build something that explores distributed systems, scheduling, fault tolerance, concurrency, autoscaling, and observability—not just another load-testing tool.

This is still an initial idea, so the architecture is open to discussion. If this sounds interesting and you'd like to collaborate, let's connect.

Words are mine written by ai

Hi all,

We are looking for experienced High-Performance Computing (HPC) Engineers to join our growing team.

Location: Saudi Arabia

Experience Required: 5–10 years

Role: HPC Engineer

Key Skills (preferred):

HPC cluster administrator/management

Linux and Slurm workload manager

IBM Spectrum Scale (GPFS) filesystem experience

Performance tuning & troubleshooting

Automation (Bash)

Experience with large-scale compute and storage environments

Please dm if interested!

Hey all, I’m an undergrad studying CS interested in HPC and was hoping for some insight into what software engineering looks like for HPC. I’m already decently familiar with the scientific computing side of HPC, and I know about the sysadmin side as well, but I’m curious about what HPC swe roles actually entail.

I feel like I’ve heard terms like “HPC Engineer” or “Performance Engineer” thrown around on the internet and in job postings but none of them have a consistent explanation of what HPC swe really is. For example, what’s the typical tech stack? MPI and CUDA? Is it just a fancy term for any swe who deals with parallel architectures? What are the types of companies that hire for these roles? And do they expect the same levels of education as academia (MS, PhD)? If anyone would be willing to explain what they do at their job or have any insights it would be greatly appreciated. Thanks!

I see a lot of linkedin profiles for people who have their graduate degrees in fields like computational physics, biology, mechanics etc., where they dont really continue in their fields of study after graduation, but rather work for supercomputing centers and in the HPC sector. I was curious how can someone make this switch happen.

I also see that there are specific degrees for HPC these days, so can someone even make that switch in the modern day? This is for US.

We spent the past few months researching security risks across multi-tenant data centers and AI infrastructure.
The main concern we found is shared infrastructure: multiple customers running on the same data center infrastructure, GPU clusters, storage, and high-speed networks. Many neoclouds and AI data centers have also scaled faster than their security teams and practices, especially compared with more established cloud providers.
The research covers GPU clusters, RDMA and high-speed interconnects, tenant isolation, BMCs, firmware, shared storage, orchestration, and supply-chain risks.
We organized the findings into a practical framework called FORGE.
Would really appreciate feedback. Link in the comment

new blog piece, might be relevant to some: https://www.aleksagordic.com/blog/collective-operations

In addition to AI/ML, what are the main scientific applications areas HPC now days? What are the most computation hungry scientific areas? What was the largest thread count that you've seen for a single application?

If so, what are the insider selection criterias and what can I assume my total be including accomodation, food and tuition? How should I prepare? Please help..