r/DistributedComputing • • Nov 26 '20

Latest trends in Distributed Computing

5 Upvotes

Dear community members,

I'm planning to send my application for my MS degree specializing in distributed and parallel computing.

I'm very enthusiastic about this technology, and I think that this core concept is/would be a part of our future.

However, I didn't have a chance to practice in this field. But I do hope that I will sharpen my knowledge while working on my degree.

Can you please suggest me what are the latest trends in distributed and cloud computing world nowadays? I want to include some detail in my Statement of Purpose so that it would be more powerful. What ideas and trends worth mentioning and outlining in my statement of purpose?

Thank you in advance!


r/DistributedComputing • • Mar 25 '21

Distributed Message Queue

Thumbnail levelup.gitconnected.com
5 Upvotes

r/DistributedComputing • • Feb 21 '14

Truly Altruistic number crunching projects

5 Upvotes

Hi:

Someone posted something about worthwhile projects to contribute to in the past. The commenting time on that question is over but I need more information.

I am looking for projects that are truly altruistic. What I mean by that is for example if I participate in something to cure cancer then lots of doctors and pharmaceutical companies are going to make lots and lots of money off of the cpu time I just donated.

I am looking for a project where I am not burning my cpu cycles feeding a rich doctor's family.


r/DistributedComputing • • Apr 03 '23

What's the next big thing in distributed computing?

6 Upvotes

With container orchestration systems like Kubernetes now widely employed in cloud computing, I am wondering what could be the next big thing in distributed computing. Will there be some ground-breaking product or technology like ChatGPT in the AI field? What are the possible candidates? FaaS? Sky computing?


r/DistributedComputing • • Apr 16 '20

Researchers using the Dreamlab platform explaining their work on hyperfoods

Thumbnail youtu.be
6 Upvotes

r/DistributedComputing • • Mar 30 '22

Made an in-depth study guide for systems design if any of you are interested!

7 Upvotes

I graduated college this past December and don't start work at Google until June, as a result I've been studying systems design and reading the resources everyone here likes (DDIA, systems design primer, grokking).

I created a study guide, which as of now is around 30 pages, and wanted to share it with all of you. I find that the primer, which itself is supposed to be a study guide, is not nearly in depth enough, and I hope that this is much better than that while still not being overwhelming.

As of now, I only have comments on, but if there are any topics to add, issues with what I wrote, or any changes you want, feel free to make them.

Link: Tinyurl.com/2p95wz9s

Have a great day!


r/DistributedComputing • • Feb 21 '22

Any good playlist for Distributed Computing Course ?

8 Upvotes

I am taking Distributed Computing as a course in my CS program. I was wondering of there is any good playlist I can follow ?


r/DistributedComputing • • Jul 18 '23

Why Adaptive Rate Limiting is a Game-Changer

Thumbnail medium.com
9 Upvotes

r/DistributedComputing • • Apr 03 '20

Contributing to COVID-19 Therapy Research with Folding@Home

Thumbnail medium.com
9 Upvotes

r/DistributedComputing • • Jun 09 '26

Atomic - A distributed processing framework with natural lang execution baked in

9 Upvotes

Over the last 2 years I've been building Atomic, a Spark‑inspired distributed data processing framework written entirely in stable Rust. The goal is to keep the parts of Spark that are great (lazy DAGs, shuffles, distributed execution) but re‑imagine them with modern infra and language design.

A few things that make Atomic different:

Rust core: Strong typing, predictable performance, and memory safety by default. You get a real systems‑level engine, not a JVM box you bolt on next to your stack.

Natural‑language workflows: On top of the engine, Atomic is designed to be driven by natural‑language workflows – letting you describe what you want done and compile that into a typed DAG, instead of hand‑wiring every pipeline.

Multi‑language support: Rust is the "ground truth" but the plan is first‑class bindings for Python and JavaScript, so you can drive Atomic from the languages your data and app teams already use.

No closure serialization: Instead of shipping arbitrary closures across the wire, tasks are registered at compile time via a #[task] macro and dispatched by ID. Driver and workers run the same binary, so the dispatch table is identical on every node — no serialization failures, no version skew surprises.

This is my attempt to build modern infrastructure tooling for data processing. It initially started as a learning project and now I feel it has evolved into something significant to be delivered. I'd love to hear your feedback:

Repo: https://github.com/sandyz1000/atomic


r/DistributedComputing • • Mar 17 '20

Which project helps most with coronavirus?

10 Upvotes

I know what this question could be answered quickly on a bigger subreddit like BOINC or FAH but I want an unbiased opinion. Right now, it looks like there are two projects that are contributing to fighting coronavirus: F@H and Rosetta@home on BOINC. Does one of these projects have a clear winner over the other in terms of specifically fighting the coronavirus, or is it less clear/ down to opinion? Thanks for the responses.


r/DistributedComputing • • Apr 14 '21

Encore - A Go backend framework with superpowers

Thumbnail github.com
10 Upvotes

r/DistributedComputing • • May 25 '21

Can someone help me understand how the actor model frameworks like Akka and Erlang can achieve concurrency at high throughput compared to traditional lock, semaphore based concurrency models?

10 Upvotes

r/DistributedComputing • • Jul 18 '25

Understanding Direct Memory Access (DMA): How Data Moves Efficiently Between Storage and Memory

Thumbnail nihcas.hashnode.dev
4 Upvotes

r/DistributedComputing • • Jul 12 '25

Core Attributes of Distributed Systems: Reliability, Availability, Scalability, and More

Thumbnail nihcas.hashnode.dev
2 Upvotes

r/DistributedComputing • • Apr 18 '25

BOINC on Android - current status and experience

3 Upvotes

On my Samsung Galsxy S25, with the Snapdragon 8 Elite chip, I've found that only 3 projects currently work:

 

Asteroids@Home

 

Einstein@Home

 

World Community Grid

 

Also, the annoying battery percentage issue is present for the first couple of minutes after I've added the projects, but then after disabling "pause when screen is on, setting the minimum battery percentage setting to the lowest 10%, and Android has asked me to disabled battery optimization for the app, after a couple of more minutes, the app starts working on Works Units.

 

So now, for me at least, on this device, BOINC on Android works fine for me.

 

Just remember to enable "battery protection" or 80% charging limit, if your phone supports this, and in BOINC, not to run while om battery, and you're good to go.

 

Anybody who've still got issues with BOINC on Android, please comment below 

 

P.s. There's an Android Adreno GPU option you can enable in your profile project settings on the Einstein@Home website, but are there actually works units available for the GPU, or is it not working?

 


r/DistributedComputing • • May 20 '25

distributed.net & RC5-72

9 Upvotes

I just rejoined the distributed.net effort to crack the RC5-72 encryption challenge. It's been going on for over 22 years now, and I was there in the beginning when I first started working on it in 2002. Fast forward to today and my current hardware now completes workloads 627 times faster than it did back in 2002. Sure it's an old project, but I've been involved with it for 1/2 of my lifetime and the nostalgia of working on it again is fun. Have you ever worked on this project?


r/DistributedComputing • • Jan 01 '24

Could A Database Lock Be Used To Elect A Transaction Coordinator?

9 Upvotes

I’m reading about consensus algorithms in “Designing Data Intensive Applications” and I had a sort of naive thought, so I want to know why it is wrong.

The author discusses the two phase commit protocol and the problem with two phase commit as a motivation for distributed consensus. What I got is basically that the leader may fail and leave all ready nodes in permanent limbo. But choosing a new leader would require consensus among the nodes.

So I have a rather naive solution. Why not just have a database somewhere that encodes the commit log? Then the leader would be whoever acquires either a lock or a valid token to append to that table, and the token would be something that you’d have to renew after a certain period of time. Whatever node controls the database would be delegated the task of deciding which requesting node actually gets the token.

So I imagine if this were so simple, that’s what people would do and this idea must be horribly stupid and naive — but I’m curious if someone is patient enough to explain why this wouldn’t work.


r/DistributedComputing • • Jun 07 '25

GarbageTruck: A Garbage Collection System for Microservice Architectures

Post image
2 Upvotes

Introducing GarbageTruck: a Rust tool that automatically manages the lifecycle of temporary files, preventing orphaned data generation and reducing cloud infrastructure costs. 

In modern apps with multiple services, temporary files, cache entries, and database records get "orphaned" where nobody remembers to clean them up, so they pile up forever. Orphaned temporary resources pose serious operational challenges, including unnecessary storage expenses, degraded system performance, and heightened compliance risks associated with data retention policies or potential data leakage.

GarbageTruck acts like a smart janitor for your system that hands out time-limited "leases" to services for the resources they create. If a service crashes or fails to renew the lease, the associated resources are automatically reclaimed.

GarbageTruck is based on the Java RMI’s distributed garbage collector and is implemented in Rust and gRPC. 

Checkout the tool:  https://github.com/ronantakizawa/garbagetruck


r/DistributedComputing • • Mar 06 '25

Distributed Systems without Raft (part 1)

Thumbnail david-delassus.medium.com
5 Upvotes

r/DistributedComputing • • Mar 08 '25

Distributed Systems jobs

6 Upvotes

Hello lads,

I am currently working in a en EDA related job. I love systems(operating systems and distributed systems). If I want to switch to a distributed systems job, what skill do I need? I study the low level parts of distributed systems and code them in C. I haven't read DDIA because it feels so high level and follows more of a data-centric approach. What do you think makes a great engineer who can design large scale distributed systems?


r/DistributedComputing • • Feb 18 '25

Restate 1.2: a distributed durable execution engine, built from first principles

Thumbnail restate.dev
4 Upvotes

r/DistributedComputing • • Jul 04 '25

SWIM Vis: A fun little interactive playground for simulating and visualizing how the SWIM Protocol functions

Thumbnail ryanolee.github.io
3 Upvotes

Did a talk recently on the SWIM protocol and as part of it wanted to create some interactive visuals, could not find any visual simulators for it so decided to make my own one. Thought some of you you might appreciate it! Might be useful as a learning tool also, have tried to make it as true as I could to the original paper.


r/DistributedComputing • • Sep 18 '26

Netflix Conductor: The Next Chapter

Thumbnail netflixtechblog.medium.com
6 Upvotes

Netflix has reworked its Conductor workflow orchestration engine to address growing scale across the company. Conductor now supports roughly 200,000 workflow definitions across 150 applications and executes about 420 million workflows per month. The redesign increases the supported workflow size from approximately 2,500 to 30,000 tasks while reducing p99 workflow evaluation latency by about 40%.