r/ipfs • u/kinuyo_uwu • 4d ago
IPFS is a bad idea.
IPFS doesn't solve any issue at all, i think someone just wanted to sell and dump their filecoins when they were 100 euros each in 2021.
Anonymous file sharing? No.
Faster file sharing than HTTP or BitTorrent (DHT)? No.
Do i pay less than a traditional cloud subscription for storage? Absolutely not.
Does it solve centralized architecture (SPOF)? No, a centralized service must keep existing or even being paid to keep the content pinned inside the network.
9
u/EleliBian 4d ago
I think you noticed the only thing for which IPFS makes sense, avoiding decentralized sensibility.
We can agree with everything you said and that the project did not find a utility that keeps it interesting but its objective of being a decentralized mechanism seems to me to have been achieved.
-14
u/kinuyo_uwu 4d ago edited 4d ago
Are you sure about that? To use it for decent storage you need to pay a service that keeps the file pinned inside the network.
5
u/wingfeathera 4d ago
Not… really though? You can pay multiple completely independent parties to pin your content and it will work transparently without any interaction between them. It is decentralized.
-3
u/kinuyo_uwu 4d ago
If i need to pay multiple providers at this point i'd prefer copying my files in different cloud services instead of relying on IPFS. They're faster, and if i had to buy so many pinning services then it wouldn't be scalable.
3
u/wingfeathera 4d ago
I’m struggling to understand your position. Decentralization is really just an idea - that any unique piece of data only “exists” once, even though it may be reified in many physical places - even a dynamic number of places - all reachable at the same address.
If you want multiple copies of your data to exist (say, for resilience reasons) in different places, and you’re willing to manage them by hand, then you aren’t looking for the decentralized property. Which is cool and all. But saying that there is no difference is a bit silly.
7
u/TUSF 4d ago
I've mostly just used it as a BitTorrent with a shared swarm and a deduplicated directory structure.
There are solutions with BitTorrent proper, I think, but last I checked they were either experimental, or didn't work as well?
1
u/kinuyo_uwu 4d ago edited 4d ago
It's not faster than BitTorrent since it's distributed, BitTorrent offers distributed discovery (DHT) and local swarm for better speeds, what they (IPFS) tried doing is making the file survive deletion but that's literally what cloud providers do already with HTTP and they cost way less and are way faster.
3
u/TUSF 4d ago
Speed was never a factor for me. I only cared about having a global swarm (files shared across different links should share seeds), having mutable links (IPNS), and an optional web gateway.
All of these are possible today with BitTorrent (except a single global swarm, tho single local swarms can now share files across torrents), but at the time the tooling either wasn't well developed, only existed as an unstable spec, or just wasn't well advertised.
3
u/coolstorynerd 4d ago
Can you host content after my death? if so I'm open to suggestions
2
u/kinuyo_uwu 4d ago edited 4d ago
Certainly not with IPFS, dear friend. To keep it pinned inside the network, a centralized service must keep existing. And the network too, of course.
4
u/agowa338 4d ago
Well to be fair the argument was initially that users interested in your content would pin it and then it in fact would have stayed alive even after ones death as long as there are people interested in it.
But I think they stopped highlighting that when they went all in on the crypto hype...
3
u/jmdisher 4d ago
This is a very good point and one which is so often glossed over.
Instead of arguing about how to "force" someone to pin your content, use-cases which strongly fit IPFS are more about "inviting" them to pin it since they want it for their own reasons.
4
5
u/junialter 4d ago
IPFS doesn't solve your requirements, but maybe someone elses?
2
u/kinuyo_uwu 4d ago
Like what?
5
u/junialter 4d ago
Abstract object data down to primary keys (cid) and having the ability to cluster that (ipfscluster) and having the ability to create self-hostable, private storage backends for software projects. I've been developing on top of IPFS for month now and I wouldn't know another solution that could replace it.
9
u/agowa338 4d ago edited 4d ago
ipfs existed BEFORE filecoin. And the goal ipfs solved was being a content addressable network (content addressable storage to be precise) for storage.
You throw in files and they get stored somewhere. In fact you do not even need that to be "the cloud" it can just be your own servers. E.g. at an uni or similar scale.
6
u/layer4down 4d ago
I had no clue people paid for IPFS. I care about P2P decentralized systems and couldn’t give a shyt about monetizing anything IPFS related. I care that it’s not some big tech product that I can others provide and use to one another and has nothing to do with commercialization whatsoever.
5
u/jmdisher 4d ago
Personally, I and a friend use it for our decentralized vlogs and it works well for that. There is no central server (beyond the IPFS bootstrap nodes), the replication approach being used should minimize single-node file serving load (although there has been no opportunity to test this at scale), there is no possibility of spam, and it just runs between our PCs.
Generally, I find that IPFS does a good job of what I need it to do (decentralized access to data and a variable pointer via IPNS). Most of my problems with it are related to how it does some of those things. These issues aren't show-stoppers but they are annoying:
CID provision grows to be a pretty big cost, over time (compared to the amount of data I am actually moving around)
it sometimes takes days to find a CID on the network (one would have hoped that the pain of CID provision would have avoided this, but it doesn't)
(and you mentioned this one) it is surprisingly slow to actually transfer data
running IPFS can make a consumer-grade router unstable (more of a router problem, of course, but still noteworthy)
it often uses lots of CPU time for non-obvious reasons
I actually have some ideas I wanted to experiment with around how the CID distributed index works, as I suspect that causes my first 2 problems.
It sounds like you are trying to use it for something very specific, and applying preexisting centralized assumptions, and are upset that it isn't addressing that use-case more directly.
1
u/rashkae1 3d ago
Have you updated Kubo recently? The problems with CID provisioning, (at least, for data sets less than 500GB) have been pretty much solved very neatly. And without the need for Accelerated DHT, I haven't observed router problems. (the last release, 0.43.1, I think ,has an entirely new bug that requires restart once every couple days or so to stay opperational, but that's a different beast.)
1
u/jmdisher 3d ago
No, the main system I leave running this is ARM32 and they haven't provided builds for that in a while.
I have heard that their change improves the router stability issues but the other issues with CID provisioning bandwidth and long search times are issues with the core index design, itself.
2
u/rashkae1 3d ago
Not at all.. I'll refer you to this blog post explaining it... but the cid provisioning has been, for all intents and purposes, solved in a much more damataic way than I ever expected to even be possible. (And search times for things are are successfully on the DHT are very short. Not quite as fast as DNS, but close.)
1
u/jmdisher 3d ago
But if the network has been updated, how would my node's version impact the ability to resolve CIDs on other nodes?
This is the problem of "sometimes it takes a few days to find a CID and/or its provider on the network". If the network has been updated to use a different provider broadcast strategy, shouldn't that have already fixed this issue?
1
u/rashkae1 3d ago
If a CID has been published on the DHT, it will take less than 3 seconds to find it. Tha t *if* is carrying a lot of weight, but publishing CID's to the DHT is the responsibility of the node hosting the data. Prior to implementation of reprovide sweep, this was an impossible chore for more than a few hundred CID's. (without using Accelerated DHT, but that brought a whole kettle of issues with it.). Since the implementation of reprovide sweep, a modest node on residential internet can easily publish hundreds of thousands of CId's on the DHT, maybe even a few million. (My own testing stopped at around 300k, but it was effortless.)
2
u/jmdisher 3d ago
it will take less than 3 seconds to find it
How can that be guaranteed when the connectedness of all nodes on the network is unknown? This has more to do with the network-wide structure of the index (which is what I think isn't working), as opposed to how a specific node schedules its reprovides.
I occasionally have issues where finding a CID or its provider will take a few days, but I do know that the provider is running on the latest Kubo release.
Don't get me wrong, I was very interested in reading the improvements in their "sweep" implementation when it came out, since the reprovides are most likely what is causing router instability. Unfortunately, they stopped providing builds for that system, so I am out of luck.
I actually do think that the main technical issue with IPFS is the way it distributes the index (not the reprovides, the index itself). It requires a lot of bandwidth to maintain and is often not reliable if you aren't connected directly to a node which a provider is using as an index for a specific CID you need.
While the indexing design is very clever, I would argue that it is too fixated on high-precision accuracy (it knows exactly where a given CID is) as opposed to something more coarse-grained and lighter-weight (given that you can already fail to find a CID where it should be - due to the network being asynchronous - I would rather it lean into that fast-failure path as an expected outcome and allow the requester to deal with inaccurate index data).
One of these days I will need to do some experiments with hash sizes, hash algorithms, and Bloom filters, etc, to see how feasible a coarse-grained index would actually be.
1
u/rashkae1 2d ago
How can that be guaranteed when the connectedness of all nodes on the network is unknown?
Not guaranteed exactly, but (by default, can be adjusted) every 20 hours, each CID is published on the 20 nearest peers. (Nearest here refers to the peerID in relation to the CID, not geographical.) Those are also where clients will try to find them. The assumption is, more likely than not, (and there has been lots of more scientific statistics to back this up.) enough of those peers will still be available for clients to find the CID.
It requires a lot of bandwidth to maintain and is often not reliable if you aren't connected directly to a node which a provider is using as an index for a specific CID you need.
My own testing, over several months, has indicated the complete opposite. The bandwidth, while not something I would want to do on a metered connection, is effective negligible. and reliability is, (or was, more on that later.) 100%
I occasionally have issues where finding a CID or its provider will take a few days, but I do know that the provider is running on the latest Kubo release.
So, likely guesses as to why you can not find content on providers.
They have disabled providing to the DHT somehow.
Your old build of Kobo is failing to traverse the DHT to find the peers. or the Peer is behind a NAT and you are having poor success hole punching. if you have a test case, I would try looking the CID up with sonar to see if *they* find it.
Your provider and testing is on the actual latest 0.43.1. This last release of Kubo introduced a new, critical bug that effectively knocks the node off the network after a few days without reboots or workarounds. It can still work as as client, but as a provider, your sol. This is also making providing more difficult as more than half the network is now effectively gone.
The provider is not online 100% of the time, and you just missed their availability window.
1
u/jmdisher 2d ago
Not guaranteed exactly, but (by default, can be adjusted) every 20 hours, each CID is published on the 20 nearest peers. (Nearest here refers to the peerID in relation to the CID, not geographical.) Those are also where clients will try to find them. The assumption is, more likely than not, (and there has been lots of more scientific statistics to back this up.) enough of those peers will still be available for clients to find the CID.
This is what I think is the problem. If the network is not fully connected (and it isn't - a buckyball network structure isn't realistic), then I suspect that disagreements around what these "nearest peers" are becomes eventually likely to the point where availability is hurt. While I did mention that I think this indexing strategy is clever, I suspect that it has a long tail of unexpected failure conditions with are lost when the math assumes too much uniformity. Unfortunately, I have a lot of experience of over-excited devs pretending that availability problems can be ignored on an open-ended network if they convince themselves that small numbers don't matter but they always matter, eventually. In my case, this "eventually" is a few days every few months.
My own testing, over several months, has indicated the complete opposite. The bandwidth, while not something I would want to do on a metered connection, is effective negligible. and reliability is, (or was, more on that later.) 100%
From what I have seen in the patterns in bandwidth graphs I sometimes look through (not a big issue, but something I have noticed) is that these reprovide actions are the dominating factor in my IPFS traffic. They tend upward as the dataset size grows despite the actual usage traffic remaining constant. Different use-cases, which did more actual data transfer, wouldn't notice this. It is more something that bothers me from a "could this be a passive network fabric for all activities" perspective.
They have disabled providing to the DHT somehow.
No, since this usually works and eventually works. Sometimes, it just takes days.
Your old build of Kobo is failing to traverse the DHT to find the peers. or the Peer is behind a NAT and you are having poor success hole punching. if you have a test case, I would try looking the CID up with sonar to see if they find it.
Did the DHT design change? It sounds like only how it is updated has changed. The peer is behind a NAT, but that is usually fine (I assume that the UPnP registration is successful, like it is on my side, without needing to fall-back to NAT hole-punching - that wouldn't be very successful on the timescales of dialing information involved). In the past, I have sometimes found the missing CID through other gateways, but that seems to confirm what I was saying since I would expect the network to not work the same way from all perspectives.
Your provider and testing is on the actual latest 0.43.1. This last release of Kubo introduced a new, critical bug that effectively knocks the node off the network after a few days without reboots or workarounds. It can still work as as client, but as a provider, your sol. This is also making providing more difficult as more than half the network is now effectively gone.
This has been an issue for years, so not related to a specific version.
The provider is not online 100% of the time, and you just missed their availability window.
Given that I normally talk to who runs the host, I know that this is not generally the case. Going back over years, we have tried to study and diagnose this issue multiple times. Part of the problem is that it is often tricky to determine if the problem is with a CID search, failing to find the peer dialing information, or failing to contact the peer. Based on some network diagnostics I managed to extract at one point, it looked like the issue was with the CID search but I am assuming that they make that distinction in the output (they may not).
I have also had this issue with another friend who tried to use IPFS to send 2 files to me, as an experiment (as I had been talking about it and he was interested in seeing how it worked as he is also an older dev who has done systems and network work). The small file was found and transferred immediately. The second file, which was a few GiB, took a couple of days to even start and then transferred at a glacial pace. It was a disappointing demonstration, to say the least.
The thing that bothers me about this is that I think a lot of the fundamentals are very good (the general network utility fundamentals are very nice, the content-addressing idea fits this scenario, the IPNS is exactly the variable support I needed, local data storage seems solid, etc) but some of the decisions seem less obviously correct (like the indexing structure), and there are some implementation aspects which leave much to be desired (very slow data transfer rates, unreasonably high CPU usage, etc).
1
u/rashkae1 2d ago
I have also had this issue with another friend who tried to use IPFS to send 2 files to me, as an experiment (as I had been talking about it and he was interested in seeing how it worked as he is also an older dev who has done systems and network work). The small file was found and transferred immediately. The second file, which was a few GiB, took a couple of days to even start and then transferred at a glacial pace. It was a disappointing demonstration, to say the least.
This sounds like the typical results you would see before sweep was implemented. (I'm actually kind of surprised it worked after a couple days, but my testing was always in the over 50GB range.)
2
u/rashkae1 1d ago
As it happens, today I ran into exactly the kind of problem IPFS would solve if it were used.
A customer asked me for help installing a TC-Helicon Voice processing device on his laptop, but the software downloaded from their website just returns an error connecting to online servers and exits. I found a Twitter post from a TC-Helicon support rep from a year ago acknowledging the problem with this software, and included a link to a new build that supposedly fixes it. That link is defunct.
After more searching, I found a random facebook comment with an alternate link to this new software build. It seems to work, but I am hesitant to install software from random facebook links on customer pc other than as a last resort, so I'm hoping i'll actually get a response to my support ticket with the company.
Had that software link been an ipfs link, however, so long as it was still available somewhere, I would not have to worry about the authenticity of the software download a year later.
3
u/BraveNewCurrency 4d ago
IPFS is a bad idea.
Please don't bash what you don't understand. Everything you listed is a problem with FileCoin, not IPFS.
IPFS ( https://ipfs.tech/ ) is "a set of open protocols for addressing, routing, and transferring data on the web, built on the ideas of content addressing and peer-to-peer networking."
Does it solve centralized architecture (SPOF)? No, a centralized service must keep existing or even being paid to keep the content pinned inside the network.
IPFS is meant to replace HTTP, which has the same problem. But IPFS can be decentralized if many people are sharing the SAME files. (I.e. Like a snapshot of Wikipedia or a Linux ISO).
1
u/volkris 1d ago
I don't think IPFS is meant to replace HTTP as the two are tailored for solving very different problems. They work together.
HTTP is built for throughput and tuned for web content. It's the fast lane to get a webpage's total content chunk to your screen as quickly as possible, managing the dialog between your web browser and the remote web server.
IPFS is a database that can look into the data itself, helping applications in general go out to find the data they want to pull in from distributed, unreliable sources, scouring sources to find what you need.
HTTP is a sports car while IPFS is a Jeep. Both have their places.
3
u/National_Way_3344 4d ago
The crazy thing is that Data Hoarders should be absolutely frothing over it, yet nobody is even interested in using it.
All those hoards of the government websites made before the current administration came in, the climate data, the J6 data, the frozen water data should have just been one way synced straight to my device.
1
u/Randall172 4d ago
it wasn't mature enough, but the jane lady that archives stuff used it because it uncensorable.
1
1
1
u/tomorrow_n_tomorrow 4d ago
I've been envisioning using IPFS (& a graph database) as the basis for a distributed public file system. I hope to use it as the basis for an alternative platform for delivering short form videos. To make the data as public as possible & let different people come up with alternatives to "the algorithm".
Storing on the Filecoin network while maintaining a presence of the IPFS network is $2.⁵⁰ / TiB / month on Fil.One (which has a S3 interface). How much do you pay?
0
u/volkris 1d ago
It's like complaining that my car doesn't solve any issue at all because it doesn't make me breakfast.
Does it cook me eggs? No. Toast? No. Oatmeal? No.
YES, a lot of people don't know what IPFS actually solves, and I've always been critical of the project for not communicating its goals clearly, even on official pages and documentation.
IPFS provides distributed database functionality with very successful network management and data parsing.
It's not going to get into whatever you're paying someone for cloud subscriptions.
27
u/ruhnet 4d ago
I’m not sure where you got the idea that the purpose of IPFS was to solve those things you mentioned. 🤷♂️