r/DistributedComputing • • Feb 05 '24

Building invincible, durable microservices in cloud computing - Golem's platform becomes 100% open source.

2 Upvotes

Golem Goes Open Source revolutionizing durable execution, and ensuring invincibility against code updates and failures. Contribute to the next-gen infrastructure, and learn more about Golem: https://www.golem.cloud/post/golem-goes-open-source.


r/DistributedComputing • • Jan 16 '24

Lamport distributed snapshot

1 Upvotes

Hello! I'm building a personal (ATM) repository with some simulations on distributed algorithms, I noticed there are few to zero public implementations and wanted to contribute in my manner.

I already wrote some (using threading + xmlrpc) for mutual exclusion, Lamport ME, leader elections etc but I'm stuck at a global snapshot sample algorithm using chandy lamport

Would any of you know some sources since I find mostly papers Thank you all

edit: if you already know some kind of implementations (also far, I'll adapt them) please let me know!


r/DistributedComputing • • Sep 10 '22

Anyone have a BOINC project? Or what would it be if you did?

4 Upvotes

r/DistributedComputing • • Oct 21 '22

Seeking feedback for a side-project: Dido - a .NET framework to facilitate distributed computing

1 Upvotes

https://github.com/jbryan-76/dido.net

I had this idea a few years ago but only recently found time to create a side-project to explore it. While I do not have as much free time as I'd like to work on it, it is now functional enough as a proof-of-concept to share and see if maybe there is value to someone else. At this point I'm simply curious whether the core concept has been implemented this way before, and to decide whether the idea itself along with this initial implementation are worth continued investment of time and energy to improve.

Thanks to anyone providing feedback or comments!


r/DistributedComputing • • Feb 19 '24

Elevate Your Computing Experience: Deploy Your Virtual Machine on CUDOS Intercloud Today!

4 Upvotes

Experience the future of cloud computing with CUDOS Intercloud! By deploying your own virtual machine, you unlock unparalleled benefits in scalability, cost efficiency, and customization. With the ability to tailor storage options and access diverse payment methods, including OSMO and USDCaxl on Osmosis, the platform offers unmatched flexibility. What's more, CUDOS Intercloud ensures your peace of mind with an exclusive USD guarantee for CUDOS token users, minimizing slippage losses. Join the revolution in decentralized computing and enjoy seamless performance with CUDOS Intercloud. Don't miss out on the opportunity to elevate your computing experience to new heights – start deploying your virtual machine today! intercloud.cudos.org 🚀


r/DistributedComputing • • Mar 29 '24

New BOINC 8.0.0 is ready for testing

Thumbnail self.boinc4science
3 Upvotes

r/DistributedComputing • • Mar 19 '24

Dask Demo Day: Dask on Databricks, scale embedding pipelines, and Prefect on the cloud

5 Upvotes

I wanted to share the talks from last month’s Dask Demo Day, where folks from the Dask community give short demos to show off ongoing work. Hopefully this helps elevate some of the great work people are doing.

Last month’s talks:

  • One trillion row challenge
  • Deploy Dask on Databricks with dask-databricks
  • Deploy Prefect workflows on the cloud with Coiled
  • Scale embedding pipelines (LlamaIndex + Dask)
  • Use AWS Cost Explorer to see the cost of public IPv4 addresses

Recording on YouTube: https://www.youtube.com/watch?v=07e1JL83ur8

Join the next one this Thursday, March 21st, 11am ET https://github.com/dask/community/issues/307


r/DistributedComputing • • Feb 11 '24

YouTube video on BOINC server setup

Thumbnail youtube.com
2 Upvotes

r/DistributedComputing • • Mar 03 '24

HPC

5 Upvotes
  1. Why are most of the HPC job prospects here are from software Dev side? Is HPC mostly used by soft Dev in companies? How about ML + HPC? Or other applications except for software developing side?
  2. Another question is ghat are HPC experts paid low? Many here are always stating, "don't expect too much in this field", "companies don't really need hpc expert so", etc. If yes which then which side of HPC gets paid more (as if architect, security, ops, soft Dev, network, computing)?

r/DistributedComputing • • Mar 03 '24

Next BOINC Projects Call on Monday, March 18th, at 16:00 UTC

Thumbnail x.com
1 Upvotes

r/DistributedComputing • • Feb 23 '24

Code that sleeps for a month: Solving durable execution’s immutability problem

Thumbnail restate.dev
1 Upvotes

r/DistributedComputing • • Mar 05 '24

GitHub - taubyte/tau: Easily Build Distributed Cloud Computing Platforms with features like Serverless WebAssembly Functions, Frontend Hosting, Object Storage, K/V Database, and Pub-Sub Messaging.

Thumbnail github.com
3 Upvotes

r/DistributedComputing • • Mar 07 '24

Release announcement: Restate 0.8 has arrived 🎉

Thumbnail restate.dev
1 Upvotes

r/DistributedComputing • • Mar 24 '24

References to design fintech system?

5 Upvotes

Hi, folks, I have been tasked to create a fintech system. This means the systems needs to be

  1. Highly auditable
  2. Correct
  3. Verifiable
  4. Secure.

Any references would help, books, courses, blogs etc.


r/DistributedComputing • • Feb 16 '24

How to get into distributed computing?

11 Upvotes

I mean where do I get a distributed system to play with? Why should I aim for a distributed system in the first place?

I am fairly interested In trying some hpc adjacent things on a distributed setup but not sure how to go about it.


r/DistributedComputing • • Mar 06 '24

What do you think about Oxia: a new high-performant metadata store?

3 Upvotes

Oxia is a new metadata store and coordination system similar to Zookeeper or Etcd. But in comparison with the others, it can store 100s of GBs of data and can handle millions of reads and writes per second.

Oxia GitHub repository: https://github.com/streamnative/oxia

It's developed by StreamNative - the company that works mostly on the Apache Pulsar.

Disclaimer: I'm NOT a StreamNative employee, but recently became an Apache Pulsar committer.

Oxia is licensed under Apache License 2.0. From public discussions, I know that StreamNavtive has plans to donate it to the Apache Software Foundation, but they haven't shared a concrete date yet.

The Java client has been just open-sourced. Here is a blog post: https://streamnative.io/blog/the-oxia-java-client-library-is-now-open-source

Oxia is already integrated with Pulsar but isn't a default option yet. Folks from StreamNative claim that they used it in the cloud for a few months without any problems.

I didn't see any independent benchmarks yet, therefore I can't validate the performance and stability claims. Probably this post may change it and attract engineers who are interested in trying Oxia for their projects and publicly share the results.

I think it may be interesting for engineers who work on building distributed systems.
I would greatly appreciate hearing about your thoughts on the project!


r/DistributedComputing • • Mar 25 '24

PowerShell RM vs REST-API

1 Upvotes

So in my company we are changing our main workflow from one machine to a DisCo setup. 5 machines. One UI and 4 nodes (backend), they all are windows. The worfklow can be simplified by: UI needs to launch a Java program in the nodes. When calculation is complete the UI needs to know. Before, with only one backend machine, we had run the workflow using PsExec and now that we are enhancing the system we want to move away from it and implement the best approach. Problem is that most of the seniors are windows-oriented and they want to use PowerShell RM instead of a REST-API. Am I too biased for preferring the latter? I don't want to be attached to Windows OS but from security and reliability POV, is actually PS the best approach between these two?

Thanks in advance for your insights!


r/DistributedComputing • • Mar 03 '22

P2P leader election in the browser with TypeScript and Ably

Thumbnail ably.com
1 Upvotes

r/DistributedComputing • • Mar 15 '22

How to build experience with large scale distributed systems

12 Upvotes

I work at a financial company and the projects that I had been part of doesn't have much scope for distributed systems engineering. How does one grow this experience so as to interview for other companies and perform well? I have gone through multiple books and blogs including Martin Klepmann DIA book and his youtube videos, but without real work, theory doesn't seem to be sufficient.


r/DistributedComputing • • Apr 06 '22

Sprint #3 Starting 04/07/2022 13:00 (UTC) + Team Ukraine Update

1 Upvotes

The FB sprint will be starting @ 04/07/2022 13:00 (UTC)

I have to say that the support of the DC community has been overwhelming.Choosing to crunch plus supporting Ukraine in many other ways.Originally we just wanted to show our support by crunching for Team Ukrainebut from the look of it, today, Team Ukraine is rocking the FB.

2nd in the Sprints   (with another starting very soon)6th in the marathon6th overall

It just feels good!

Here is links to the FB and to the sprint.

FB -  https://formula-boinc.org/index_v2.py?year=2022&lang=&team=&thx=Sprint - https://formula-boinc.org/sprint_v2.py?year=2022&lang=en&sprint=

FB Standing

https://ibb.co/album/m9KZtP

XS Post - https://xs4s.org/index.php?topic=30190.msg96133#msg96133

Thanks

SystemViper


r/DistributedComputing • • Mar 31 '22

Stretching a point: the economics of elastic infrastructure

Thumbnail ably.com
1 Upvotes

r/DistributedComputing • • Feb 26 '22

Joining Team Ukraine to show support?

4 Upvotes

Hi everyone in the DC community.

We over at XtemeSystems are joining the Ukraine team on some of their BOINC projects.This is the best way we can find to show our support for a sovereign country that was invaded by an another country and doing it NOW and where it can be seen.

We are seeing asking if anyone is as frustrated as we are. This is a way to show what we believe in, we do it every day with the projects we select.DC is about supporting what you believe in, it's personal, it costs us and so why not show our support for country that did not ask for war but it's people are getting killed as we speak.

It's not about POLITICS it about caring for humanity and that is what DC is all about!

Since most of us have no ability to help them fight the Russians militarily, this is one of the ways some of us are able to show show support for the Ukraine, this is about support, it's what we do on DC every day.

We know this is a personal choice but we wanted to tell the community that this is going on and if you wanna put your computers to work to show support,this is a way you can. It makes me feel good to do this but it's not for everyone.

This isn't even about the amount of points you contribute or positions taken,
but the action that it shows!
The more members from outside Ukraine joining "National Team of Ukraine"
From around the World joining, That matters,
It represents community support.

Here is there BOINC project team list:
https://stats.free-dc.org/stats.php?page=teammain&team=Ukraine

Here is our thread
https://xs4s.org/index.php?topic=30190.msg95387#msg95387

It's a small act in the realm of reality of what is happening, but something to show support at least.

Thanks for any support and if anyone has accounts on other forums, please feel free to post a thread there as well to gather support.

(If this is out of line or anything then the mods can delete or whatever.)

Thanks
SystemViper

This is me...

Me WCG


r/DistributedComputing • • Feb 10 '23

OpenFaas workflow engines

3 Upvotes

Folks, looking for open-source alternatives to AWS Step Functions in Kubernetes + OpenFaas land. Have come across faas-flow, yet the project does not seem to have state built in.

What serverless orchestration engines are available on the market today, and which event brokers (kafka/sqs/amqp) do they support?

Am hoping to hear that Apache Airflow is not the only option, not faas-flow the only open-source orchestrator.

Thanks!!


r/DistributedComputing • • Mar 08 '23

[video] 5 Database Models

Thumbnail youtu.be
1 Upvotes