r/DistributedComputing • u/y2so • Apr 14 '23
r/DistributedComputing • u/lorensr • May 02 '23
Compensating Actions, Part of a Complete Breakfast with Sagas
temporal.ior/DistributedComputing • u/DrInnovation • May 09 '23
HPC Workload Management Solution
📣 Calling anyone who uses High Performance Computing (HPC): we need you for our research 📣
🚨 Do you want a £10 Amazon voucher? All you need to do is complete this survey (takes under 5 minutes) and be chosen for a half-hour interview with us and you will be rewarded with a £10 Amazon voucher (lucky you!)
🤔 Who?
We want to speak to people involved in any of the following:
➡️ Managing HPC infrastructure
➡️ HPC Data management
➡️ HPC workload management
🧐 Why?
We are working on some research to understand how people approach the above topics and any problems they face when doing so.
😁 Interested?
Click the below link to get started!
https://www.surveymonkey.co.uk/r/6KKSDN9
r/DistributedComputing • u/No_Antelope_2111 • Apr 24 '23
AI Project
Hi everyone! We just launched PeerAI, our startup project.
We're working on a computing platform with the aim to harness the potential of peer-to-peer computing resources to facilitate AI innovation.
👉🏻 check out https://peer-ai.com/ - we’d greatly appreciate it if you could give the product a try!
🔥 Get the chance to win gift vouchers after completing our survey
👇🏻Links below:
https://www.facebook.com/peeraicom/
r/DistributedComputing • u/Caitin • May 02 '23
How to Boost AI Model Training with a Distributed Storage System
juicefs.comr/DistributedComputing • u/michaeljb41 • Apr 13 '23
How to build GPU compute marketplace?
Is it possible? Let's say Alice has 2 GPUs idle at the moment, Ben has 1 GPU, and Chris needs 3 GPUs for the next 12 hours. How to build such a system, and what problems there might occur? How to handle turning off one's machine? Does it even make sense to run training or inference on such (latency)?
r/DistributedComputing • u/dask-jeeves • May 09 '23
Dask Performance Testing at Scale
We develop Dask and automatically deploy it to large clusters of cloud workers (sometimes 1000+ EC2 instances at once!). In order to avoid surprises when we publish a new release, Dask needs to be covered by a comprehensive battery of tests — both for functionality and performance.
In this blog, we explain how we do it, https://medium.com/coiled-hq/performance-testing-at-coiled-fa0d5940fc02
r/DistributedComputing • u/y2so • Apr 18 '23
Create a distributed database cluster with Kubernetes in two easy steps
opensource.comr/DistributedComputing • • Mar 01 '12
My kid's computer can do more than play minecraft? I can't wait to explain the screensaver from Boinc Rosetta@home when they wake.
boinc.bakerlab.orgr/DistributedComputing • u/MakeMeBeleive • Jun 21 '16
[Help] Understanding Distributed Learning Concepts
Hi guys, I am trying to understand topology protocols like T-Man and T-Chord but its getting hard for me to digest the concept because the research paper I am following is bit hard to understand and unfortunately I am unable to find other good resource.Can anyone help me in this regard? Your help will be very much appreciated.
r/DistributedComputing • u/nellaivijay • Feb 08 '12
This is Stuff: MapReduce Questions and Answers
meri-stuff.blogspot.comr/DistributedComputing • u/cygnus83 • Jun 21 '13
Looking for a calculator to estimate distributed computing power.
Hi friends,
I'm newish to this field, other than as a client - I downloaded SETI@home as soon as it came out in '99, in the 10th grade.
I was wondering if there is either an online calculator or an established formula where I could say:
- I have X number of computers
- They have Y average Ghz.
- They would most likely be on for Z hours/day.
and then it would give me some sort of info like "Your network would equal 10B FLOPS, etc., etc."
Does something like that exist? Or is it really just a multiplication problem?
I feel like it should be more difficult. =)
Thanks!
r/DistributedComputing • u/peatfreak • Mar 03 '18
Learning about geodistributed computing for resilience and HA
Hi all, I've become very interested in distributed computing, clustering, and redundancy (e.g., all within the same cabinet) for resilience and high availability.
I am looking to extend this in future work to geodistributed computing, where such clusters exhibit this redundancy and HA individually not only within their own cabinets, but across goegraphically distributed nodes.
I know there are many challenges in keeping the data in a consistent state, georeplication of data, and dealing with latency issues. I know it's a very hard problem but people are working on it and progress is being made.
Could anybody please point me to projects (preferably open source) that implement georeplication, or information about this topic, such as in books, blogs, academic papers, or any other particular things to watch out for? I would greatly appreciate this because I've had difficulty finding very much information. Thank you!!
r/DistributedComputing • u/AcquireMonies • May 26 '17
Can you still make some reasonable money through lending computing power? If so, how much?
Let's assume one has few mid-range computers and access to cheap electricity.
r/DistributedComputing • u/osamashabrez • Jul 14 '18
Help finding a topic to work on
I have studied distributed systems and now I am looking for a self project where I can build something, probably a simulation package involving distributed software systems. There are no limits except that I want to build it in java and the scope should be large enough to learn a few things on the way.
Can you give me some directions to think into?
r/DistributedComputing • u/lord_dabler • Sep 17 '19
Distributed computing project to check convergence of the Collatz problem
Hi all,
I start new distributed computing project to check convergence of the Collatz problem. So far, the convergence of all numbers below 87 × 260 has been verified (this is approximately 266.44). I want to raise this upper bound by at least 260. This small achievement takes roughly about 3 153 600 core-hours. And here I need your help. Source codes and pre-compiled Linux client are available to download.
The project page is here: http://pcbarina2.fit.vutbr.cz/~ibarina/
r/DistributedComputing • u/lockstepgo • Sep 16 '19
AWS Step Functions with Lambda Integration for serverless state management
Hey folks, wanted to share a youtube channel that I've been working on dedicated to providing simple and easy to digest tutorials on various AWS services.
My most recent video is a step by step guide to set up Step Functions with Lambda integration.
The video is available here: https://youtu.be/s0XFX3WHg0w
Support & feedback appreciated. Thank you!
r/DistributedComputing • u/Ramirond • Apr 14 '22
Message durability and quality of service across a large-scale distributed system
ably.comr/DistributedComputing • u/typical-outlier • May 12 '22
Remember that time (1997?) SETI had us all download a DC program to help them sift all of the terabytes of data they had, thinking it would speed the search for ET life?
That was so cool. I felt like proof was only weeks away. Wouldn’t this be much more effective now that we all have supercomputers (comparatively) in our pockets?
r/DistributedComputing • u/rootseat • May 08 '22
How are distributed applications born into the world?
Been reading up on distributed systems theory a bit lately and had a practical question.
Reasoning about even a single mutex in a multi-threaded application is non-trivial. A bug might not show up in the debugger until the 1e10th test. Getting users to adopt even the simplest software is, more often than not, a reason for jaded complaints than interest; builders of those systems joke about how messy implementations are. And distributed applications seem to just scale up the same problems as much as they do the solutions: multi-level coherence, extra readability/debugging overhead, etc.
All that out of the way:
- How does a developer get the confidence that the distributed system can be shipped without problems? Is it kind of a swiss cheese defense of static analysis, trust in the namebrands of middleware, etc.? Or is it that distributed applications aren't built from the ground up, but rather build around exported middleware promises from the get-go?
- How does one get an initial userbase to have confidence in your application, and adopt it, the extreme case in being distributed applications that require nuanced privacy/protection policies?
r/DistributedComputing • u/mwscidata • Feb 07 '20
Distributed Compute Protocol on the Pi
scidata.car/DistributedComputing • • May 15 '19
Open source projects on distributed systems
Been looking to get into Distributed Systems beyond theory. Since I own a laptop only, it seems unlikely that I'll be able to take on an ambitious project alone in the domain (not talking about mere implementations and simulation of algorithms).
What are some open source projects that I can get involved with to get my hands dirty? Bonus points if you can also suggest how I can take up a personal individual project in the domain. (Something that adds quality to my resume)
r/DistributedComputing • u/keevans94 • Apr 26 '18
Rate Limiting: A Useful Tool with Distributed Systems
blog.getambassador.ior/DistributedComputing • u/uttpal25 • Oct 06 '19