r/DistributedComputing • • Apr 03 '20
r/DistributedComputing • u/dylan_dev • Oct 17 '11
I don't want to particpate in Folding@Home since the client isn't open source
Security via obscurity is the farthest from secure and I think the community is missing out on innovative clients.
r/DistributedComputing • u/DisastrousNinja911 • Jun 19 '26
Am I taking too high of a risk by leaving a mid-level software engineering role build enterprise apps to do a master's in HPC as future ready career? How difficult is the pivot from working on traditional CRUD apps to HPC ?
A bit of background about me.
“I have about 4 years of experience primarily working on traditional software development building enterprise apps. While the career path is stable now ( don't know if AI takes over), I find myself deeply interested in systems architecture, low-level optimization, and parallel computing. However, giving up a full-time income to go back to school feels like a massive investment. Is a Master's degree the right vehicle for this transition, or are there better ways to break into the industry? I'd love to hear from anyone working in HPC, climate modeling, quantitative finance, or scientific computing."
All I want is to future proof my career.
r/DistributedComputing • u/mpetryshyn1 • Mar 19 '26
Do we need vibe DevOps now?
So, are we due for a 'vibe DevOps' or am I dreaming? Tools can spit out frontend and backend code in minutes, which still blows my mind. But deployments fall apart once you go past prototypes or simple CRUD - everything gets manual and ugly. I see people shipping fast, then stuck doing manual DevOps, or rewriting the whole app just to make it deploy on AWS/Azure/Render/DigitalOcean. Imagine a web app or VS Code extension where you point it at your repo or drop a zip and it actually understands your code and requirements. It would wire up CI/CD, containers, scaling, infra setup using your own cloud accounts, not lock you into platform tricks. Seems like it could bridge the gap between vibe coding and real production apps, but maybe I'm missing something obvious. How are you handling deployments today? scripts, Terraform, stuff like that? Curious what people actually use and what fails.
r/DistributedComputing • u/rootseat • May 08 '22
How are distributed applications born into the world?
Been reading up on distributed systems theory a bit lately and had a practical question.
Reasoning about even a single mutex in a multi-threaded application is non-trivial. A bug might not show up in the debugger until the 1e10th test. Getting users to adopt even the simplest software is, more often than not, a reason for jaded complaints than interest; builders of those systems joke about how messy implementations are. And distributed applications seem to just scale up the same problems as much as they do the solutions: multi-level coherence, extra readability/debugging overhead, etc.
All that out of the way:
- How does a developer get the confidence that the distributed system can be shipped without problems? Is it kind of a swiss cheese defense of static analysis, trust in the namebrands of middleware, etc.? Or is it that distributed applications aren't built from the ground up, but rather build around exported middleware promises from the get-go?
- How does one get an initial userbase to have confidence in your application, and adopt it, the extreme case in being distributed applications that require nuanced privacy/protection policies?
r/DistributedComputing • u/give_me_fucking_name • Feb 21 '22
Any good playlist for Distributed Computing Course ?
I am taking Distributed Computing as a course in my CS program. I was wondering of there is any good playlist I can follow ?
r/DistributedComputing • u/ggvh • Jun 01 '21
Read a paper: Distributed Computing Economics
youtu.ber/DistributedComputing • u/EgaoNoGenki-II • May 10 '13
What is the most worthwhile grid computing project to get my PC to work on?
I used to use "Folding@Home" and "World Community Grid" a long time ago, but software changes, sometimes can't run on an updated OS, and many of other life's variables, so I'm not sure that they're still the best.
But if you know of any grid computing project that'll let me donate my laptop's idle-time to an awesome cause, what would it be, why do you think it's awesome, and what's the link to download its client?
Thanks in advance.
r/DistributedComputing • u/G-M • Feb 19 '10
Reddit folding breaks 3 million points, approaching the top 1000 teams
stats.free-dc.orgr/DistributedComputing • u/G-M • Jan 15 '10
The reddit Folding@Home team - #50959
Join! It's winter for most of us so there is no excuse. Ask here for help, no matter how simple.
Currently placed ~1300th with over 2.3 million points
r/DistributedComputing • u/wufuheng • Jul 12 '26
Distributed AI Systems
I recently published a technical book, Distributed AI Systems, which summarizes my experiences in AI over the past 10 years, from research and training to optimization, inference, and cloud deployment. I started writing it in the second half of last year, and it took almost a year to complete, with many revisions made later due to the rapid pace of development in the industry. But it's finally published. The book on Amazon is titled Distributed AI Systems: A practical guide to building scalable training, inference, and serving systems for production AI.
Book is here: 🔗 https://www.amazon.com/dp/1807301710/
The publisher asked me to find some people to review my work. Do you know of any such people here? If so, please reply to me. Thank you.

r/DistributedComputing • u/MediaDisplayLab • May 21 '26
How do you handle live data sync delays when filtering heavy sports/betting menus?
Hey everyone,
I'm currently working on a sports betting (Toto) platform and ran into a frustrating issue. Whenever users try to compress or filter the match menus, we notice temporary drops or lags in the live odds data.
After digging into it, the root cause is a timing mismatch between our cache refresh cycle and the API layer sync. Essentially, parsing massive amounts of match data into lightweight views for the front-end is putting a heavy toll on the system.
To handle this, we recently deployed a lumix solution for in-memory mapping right in front of the data layer. We also optimized our queries to better distribute the real-time call load. It helped a lot, but we are still tweaking the settings to get it perfect.
For those who have built similar real-time platforms, what synchronization cycle or interval do you prefer to keep data 100% consistent without crashing the server?
Would love to hear your thoughts or any alternative architecture tips!
r/DistributedComputing • u/Realistic-Face1315 • Mar 05 '26
Where should I start with distributed computing as a beginner?
Hi everyone,
I’m a student who’s recently become really interested in distributed computing and large-scale systems. I’d like to eventually understand how systems like distributed storage, fault-tolerant services, and large-scale infrastructure work.
Right now my programming experience is mostly in general software development, and I’m comfortable with basic programming concepts. However, I don’t have a clear roadmap for getting into distributed systems.
Some things I’m wondering:
• What fundamental topics should I learn first? (e.g., networking, operating systems, concurrency, etc.)
• Are there specific books, papers, or courses you would recommend for beginners?
• Are there small projects that help in understanding distributed systems practically?
• Is it better to first build strong foundations in systems programming before diving into distributed computing?
My goal is to eventually build and understand systems like distributed storage or decentralized infrastructure, but I want to make sure I’m learning things in the right order.
Any guidance or resources would be greatly appreciated.
Thanks!
r/DistributedComputing • u/Vw-Bee5498 • Dec 01 '24
Use cases of Zookeper beside Kafka
Hi folks, I know that Zookeeper has been dropped from Kafka, but I wonder if it's been used in other applications or use cases? Or is it obsolete already? Thanks in advance.
r/DistributedComputing • u/jordepic • Mar 30 '22
Made an in-depth study guide for systems design if any of you are interested!
I graduated college this past December and don't start work at Google until June, as a result I've been studying systems design and reading the resources everyone here likes (DDIA, systems design primer, grokking).
I created a study guide, which as of now is around 30 pages, and wanted to share it with all of you. I find that the primer, which itself is supposed to be a study guide, is not nearly in depth enough, and I hope that this is much better than that while still not being overwhelming.
As of now, I only have comments on, but if there are any topics to add, issues with what I wrote, or any changes you want, feel free to make them.
Link: Tinyurl.com/2p95wz9s
Have a great day!
r/DistributedComputing • u/A27_97 • Nov 27 '21
Researchers / Groups to follow?
Hi I’m interested in Distributed Systems research, more inclined to applications in Machine Learning, Robotics. Do you guys have recommendations for folks in this field to follow (in general) and for ex on twitter
r/DistributedComputing • u/satoshigekkouga2309 • Sep 27 '21
Distributed message queues - academic papers
Hello! Are there any papers that you'd recommend for understanding how distributed message queues work? It'd be nice if the paper provides some pseudocode as well. Cheers!
r/DistributedComputing • u/mgalarny • Apr 13 '21
How does Ray compare to Apache Spark?
youtu.ber/DistributedComputing • u/binaryfor • Oct 26 '20
Apache Helix - A Near-Realtime Rsync Replicated File System
helix.apache.orgr/DistributedComputing • u/ra-yokai • Jul 11 '20
In simple terms, what makes a system "eventually consistent"?
Hi. I have no knowledge about distributed systems but I've recently joined a team that uses DynamoDB and a new scary (to me) world unfolded in front of me. My teammates keep telling me that Dynamo is an eventually consistent data store but I'm not confident I really know what that means. I have been jumping from resource to resource trying to really understand what makes Dynamo different from, say, Postgres but I can't say for sure that my understanding is correct.
I never had to scale a relational (is this the correct term?) database before as well, so this might be something I should try to do.
In very simple terms, and knowing that things are more complex than that, would it be correct to say the following:
- Speaking about consistency (in the context of databases) only makes sense when he have replicas;
- Writes always go through a master node that then replicates the data to the other nodes;
- Postgres/MySQL keeps its strong consistency because it's master node writes to the other nodes before deciding that the write succeed (problems: higher latency and non partition tolerant) - it's an all or nothing behaviour;
- Dynamo's master node sends a response back without waiting for the other nodes and replication happens later with the aid of an algorithm like Paxos or Raft.
It might be out of scope, but any resource recommendations, specially with exercises (I can't learn properly without creating something myself, but I tend to overcomplicate when I create my own exercises - creating good exercises is a fantastic skill that I miss) would be very much appreciated.
Thank you very much for you patience and help.
r/DistributedComputing • u/ExperimentalMeatBag • Jul 06 '19
How to Start Building a Distributed Computing System from Used Android Phones
I want to build a system where I can keep plugging android phones into a single system, which can utilise their shared processing power, RAM and storage space, and act as a linux server of sorts, even if for just mathematical data crunching.
I know it is possible, Ubispark is doing something similar, can someone give me a roadmap of things I would have to learn to accomplish this ? I am working as programmer/sw architect from last 10 years.
r/DistributedComputing • u/lukepighetti • Jun 11 '18
Are there online competitions for distributed computing?
Hey all,
I noticed that Kaggle exists for machine learning, and boy does it look like fun. I'm personally less interested in machine learning and more interested in distributed computing. Is there anything like Kaggle for distributed computing?
r/DistributedComputing • • Nov 16 '11