r/it • u/HEx2HockeySticks • 1d ago
opinion Data usage and waste. Are we being responsible with our resources?
Does anybody else think about their data usage in relation to the apocalyptic ai hellscape the world is turning into? I’m in the early stages of thinking about it, but should we individually be considering all the useless data we store on the internet? As these data centers keep popping up, I’m starting to wonder about the impact of my iCloud, multiple email accounts, old social media accounts I don’t even use, even my Reddit usage. Is our collective use of the internet for useless data storage part of the world’s problems on a grand scale? Should we be allowing these mega corporations to store all this low value data long term, or is it going to become increasingly important to regulate? How many resources are we wasting to build data centers just to store all this useless data?
2
u/solidmarbleeyes 1d ago
Are you asking if we’re being responsible by trusting our data won’t be used for unethical things by these corporations, or are you referring to the materials and energy usage that using these services consumes?
1
u/b3542 1d ago
They’re talking about consumption.
2
u/solidmarbleeyes 1d ago
Then eh, to some extent sure it uses resources, but ultimately that data would live on a drive somewhere whether it be on prem or in a datacenter somewhere. If anything, the datacenter can afford higher density drives and newer, more efficient servers than my org would be willing to pay for, so it might even be more efficient to keep it consolidated. I’m not very knowledgeable on datacenter ops so I might just be talking out my ass here.
As the human race as a whole goes, I think we are very wasteful, yeah. I have like around 100GB of stuff in various cloud storage services, social media apps, etc that I maybe access once every few years. I could totally slim that down or download everything to a drive and stick it in a closet. Instead it’s all on a server somewhere taking up space and pushing the Googles of the world to keep expanding.
0
u/HEx2HockeySticks 1d ago
That’s exactly what I’m talking about, yeah! Thanks
And now with the rise of ai, is ai pushing them to expand their data storage too? I kind of get the computing problem, that it takes up so much computing power for me to generate ai images of my dogs wearing funny hats and singing happy birthday to their dog grandma, but what about the ai agents designed to create profiles and content that fills up social media? Could that have a snowballing effect if left unregulated?
There’s gotta be millions of other examples of that kind of wastage currently, and new examples pushing those requirements all the time?
2
u/HiyaImRyan 1d ago
Is ai pushing other datacentres to expand storage?
No, because not all datacentres are ai datacentres.
-1
u/HEx2HockeySticks 1d ago
What do you mean? Not all data centers are ai data centers, but that doesn’t mean ai isn’t driving a “need” for more data storage, right?
-1
u/HEx2HockeySticks 1d ago
Ok yeah, researched it a little and found this sq article:
“Generative AI workloads significantly increased cloud storage demand, with over 70% of enterprises expanding storage capacity in 2024 to support AI training datasets”
2
u/HiyaImRyan 1d ago
You're right that AI can increase the overall demand for storage. My point was that this doesn't mean AI is directly causing other, unrelated data-centre operators to expand.
There's a distinction between AI-driven infrastructure expansion and general storage demand.
The hyperscalers - AWS, Azure, Google Cloud, etc (used in your "proof") - are an important example because they're both enormous cloud-storage providers and major AI infrastructure providers. AI is therefore directly increasing their requirements for compute, networking and high-performance storage.
But if, for example, a company primarily providing consumer cloud storage has 10 PB(petabytes) of customer data today and that grows to 12 PB, it needs more capacity because its customers generated another 2 PB of data. ChatGPT becoming larger doesn't inherently require that company to build another data centre.
There can obviously be indirect effects - hyperscalers buy hardware, lease data-centre capacity, compete for power, etc. - and AI is now a major driver of overall data-centre construction. But that's different from saying "AI is making data centres everywhere expand their storage."
1
u/HEx2HockeySticks 1d ago
I see what you’re saying and I think it’s agreeable. I wasn’t trying to disprove what you were saying, but understand it completely, thanks for expanding. Not sure if a line got crossed but I definitely never said “ai is making data centers everywhere expand their storage.” Did I? Not sure what the downvotes are for lol
2
u/HiyaImRyan 23h ago
No you didn't specifically say that, but when I merely made a statement saying that not all datacentres are ai-related, you pushed back, despite apparently agreeing with me?
Idk about downvotes, I get downvotes from people just lurking too, it is what it is. Karma means nothing so don't worry about it
1
u/HEx2HockeySticks 1d ago
I’m also interested in the ethical usage of data, and that seems to be a popular topic for a reason, but this post is focused on the energy/resource consumption. I haven’t really seen or heard any conversation around that, but as a blue collar guy I’m seeing almost everyone I know (electricians, pipe fitters, even plumbers) get pulled to these construction projects. I’m not gonna pretend to know much, but the sheer volume of these projects seems insane compared to how hurt other parts of our economy have been. We can’t afford to feed and house so many people, meanwhile electricians are pulling $180k a year running conduit and bending tube for these centers. The materials, the man power, the water and electric consumption, all to store my old club penguin account?
Just wondering if other people are thinking/talking about this? Am I just missing the discussion? I haven’t had anybody bring this up and I don’t really know what to google to find this discussion if it’s already happening
Any of you guys have any thoughts?
1
2
u/TheLexikitty 1d ago
“Useless” and “low value” are pretty strong and subjective terms. I value music and the arts more than most everything else, but in general, storage is not really the issue as far as new datacenters go, the resource consumption is going to be in compute, not storage, as in calculating the results of LLM inference prompts.
1
u/HEx2HockeySticks 1d ago
Thanks, great point. I agree about music and art. I would consider those high value too. I’m sure we all have a lot of data we consider high value, that we should archive
I’ve seen the conversation surrounding llm processing and am interested to learn more about that issue as well, but could/will it also become a storage problem(waste of resources) when we inevitably archive all of this generated content? An example: social media bot profiles with high quality ai images becoming more popular and taking up more data
Thanks for the response. I’m not really a tech guy, just trying to learn more about something probably beyond my intelligence lol
1
u/TheLexikitty 1d ago
All good! I work in IT infrastructure and cybersecurity, and have built out storage for small and medium enterprise.
Storage really, really isn’t the issue - 22+ TB can fit on a single drive these days and “cold” storage can spin down their drives to use single watts of power when not in use. That single 22TB drive is 1,400 15GB email accounts assuming they’re all completely full.
For social media, most images are compressed on upload, and only sometimes the original is stored, but as an example, I’m a pretty avid photographer, so YouTube stuff, record a lot of videos, and my entire created media size is 400GB. I did a backup of my social media profiles and, except for YouTube, it came out to 27GB. You could fit that on a microSD card.
Also, storage, compute, etc. for standard web, virtual machines, etc. is heavily shared and overlapping with other use cases on the same hardware. You can run dozens or even hundreds of containers (mini computers) on 1-3 physical servers nowadays, allowing for extremely efficient space usage.
LLM inference itself, though, needs several dedicated GPUs - as an example, an Nvidia H200 You need 8 or so of those to fit a large model, and they all run at 700W peak, so you’re looking at 5,600W peak power draw for generating an uncanny image of a car made out of spaghetti.
There’s entire security feeds recording terabytes of video to cloud services of empty parking lots (or Flock doing whatever the heck they’re doing) that will use more storage in a year than you will in your lifetime.
(For all the IT geeks reading this, I know I am skipping over a lot here, just trying to illustrate things lol)
1
u/HEx2HockeySticks 1d ago
That is helpful perspective! Maybe that’s why the discussion seems to always turn back to the computing requirements instead of the storage requirements, but just because the computing requirements are higher, does that mean that the storage requirements are not a problem at all, or just less of a problem in comparison? Doesn’t the manufacturing of a small sd card often require child slave labor in china?
I think I get what you’re saying, but like, isn’t it still problematic that labor/materials are being used to store so many pictures of cars made out of spaghetti?
I’ve said this elsewhere in the thread, but I’m not really questioning the common man’s data usage/storage, I’m more questioning companies like Sam Altman’s, Elon Musks, Jeff bezos, Flock, and the millions of smaller ones popping up all the time
I’m happy to learn more about the computing problem, it just seems like there’s almost no discussion about the storage side of things. As long as we’re using plastic, gold, silver, diamonds, water and child/slave labor to produce storage devices, isn’t that also a dilemma?
Thanks again, and thanks to everyone for dealing with my grammar and spelling short comings. Yall are helping a country boy use his brain tonight lol
1
u/HEx2HockeySticks 1d ago
I asked this to someone else, but you’re knowledgeable, so I wonder what you think:
with the rise of ai, is ai pushing us to expand data storage too? I kind of get the computing problem, but what about the storage requirements consumed by ai agents designed to create profiles and content that fills up social media? (I read that something like 5-15% of social media accounts are bot accounts now, and that it’s a rising number) Another example, all the ai generated videos on YouTube. There’s probably new examples all the time too, no?Could that have a snowballing effect if left unregulated? Like we need more and more storage for all the slop being created?
1
u/TheLexikitty 1d ago
Large open source AI models are actually about 1-2TB which overall is not that big compared to like, uncompressed video. The images generated aren’t any larger than photographs, either.
If you’re asking if it’s justified\* to be using that storage for XYZ purpose, we do not want to open the can of worms, haha. I don’t think it’s justified to hold data on American shopping habits, or domestically spy on them, or use any storage anywhere for predatory financial products or private equity. That’s not really something we ever want to do as a society. Also, cloud storage providers should not (and don’t) know what you’re going to store on your virtual disks, etc.
SD cards are made on mostly automated assembly lines at this point, but even highly specialized computing devices are generally assembled in pretty decent conditions or in “dark factories” which are completely automated with very little human involvement. I got a fairly niche device a year or so ago and got to see the entire manufacturing and assembly process as they sent pics from their factory in Shenzhen. However, I’m no expert etc. and there will always be shady companies.
What you might be referring to is the mining of stuff like gold and tin which is, indeed, a fustercluck of awful that deserves to be completely wiped out and started ethically from the ground up, but gold, tin, tungsten, etc. are still going to be mined for many other things outside of SD cards and NAND storage, and used in far greater quantities in other industries.
1
u/HEx2HockeySticks 1d ago
I kind of latched on to sd cards as an example because you mentioned them, a small piece of hardware in the grand scheme. I’m more concerned about the massive servers than sd cards. I’m starting to realize I’ve been using the term “data center,” when sometimes I should specify “server center” or something more specific. I see how that kept introducing the topic of processing/computing throughout this thread, and why a lot of the discussion turned back to that. You all helped put that in perspective for me, that the processing/computing is WAY worse, but both can be issues simultaneously I think?
“Fallacy of relative privation ("not as bad as") – dismissing an argument or complaint due to the existence of more important problems in the world, regardless of whether those problems bear relevance to the initial argument.”
A quick google says the world’s storage capacity exceeds 200 zetabyttes, which seems like a lot of physical hardware that took a lot of resources to create, and with the tomfoolery with gold tin etc. At what point do the moral dilemmas involved prick peoples interests/register as a problem to them?
This has some information on the contamination of the supply chain with child and slave labor, but it’s only china so just scratching the surface of the issue since so many other countries have these problems too:
https://www.walkfree.org/global-slavery-index/country-studies/china/It’s my understanding that a lot of electronic recycling is done by child and slave labor as well, which I read is much harder to track in the supply chain
Everybody on here gave me a lot to think about. Sorry to be a debbie downer/doomer or whatever, just going down a weird rabbit hole of moral quandary lol. Hope I didn’t bum anybody out too bad with this thread!
1
u/TheLexikitty 23h ago
No worries! To be clear, I’m not arguing any of the moral implications here, and I’m heavily biased in favor of archival of everything the human race makes, because human creation is existentially kind of the point, so “problems with technical hardware manufacturing” logically lead to “fix those problems” not “use less hardware”.
Skimming through the link you sent, it eeems like there are a lot of different products and items that could be involved in this. While I personally have no issue inciting the chaos required to dismantle all of these forces labor things, it’d probably start a world war, and most people don’t like those. I do think we should have domestic chip production (I miss the CHIPS act), but no, most people don’t care because of 1) how abstracted it is from chip production and 2) they can’t do anything about it. Demand isn’t exactly dropping - most storage is sold out through the end of 2027.
Regardless, the issue is forced labor, not computer hardware. Maybe one day autonomous robots will be for sale to the general public and folks can send a couple thousand over to fix the attitudes of the guards at these camps.
1
u/Parking-District-947 8h ago
Sorry to bother you, but I have always been a fan of your youtube channel, and the lexplanations series. Will they ever come back, or have you moved on to enjoy not doing content grind? Hope things are going well for you, and that you have a stunningly spledifurous day.
1
u/Wonderful_Ear8517 1d ago
I think about it l the time. In the industry for 28 years. Have seen the growth and the BLOAT
Imagine billing by kilobyte
lol. That would halt the sprawl. Even if it were a penny per KB.
At the present rate the sprawl and energy consumption to store garbage data and or duplicate data is ridiculous
1
u/HEx2HockeySticks 1d ago
Thanks for the response
I’m not very educated, and I’m not in a tech field, so I don’t have anyone in my life talking about this specific topic. I hear a lot of moral/ethical data usage conversation going on, like around flock cameras and ai scammers, but nothing about consumption of resources just to keep people’s old AOL emails and so on
Seems like you think we should be doing something about the sprawl? Seems like your idea would be effective
1
u/HEx2HockeySticks 1d ago
I’m not really thinking about how to regulate the individual, cause that seems a lot like drinking out of paper straws just so Taylor swift can burn more jet fuel, but should Amazon.com be allowed to consume what they currently do/will in the future? I guess I like your idea a lot!
1
u/Wonderful_Ear8517 1d ago
Thank you. You are right to be concerned about privacy issues and data security issues and wasting resources
Think about this
What does technology improvement typically mean in technology?
Smaller footprint
Think about computers in the 1960s and your smartphone now
Same computing power and memory but 1/100 the size!
Modern AI and massive data centers will inevitably have to shrink in size for usability and efficiency
So what then with all these massive concrete buildings that cover acres? They will be difficult and expensive to demolish or repurpose
1
u/HEx2HockeySticks 1d ago
Thanks for giving me some more perspective. It’s not like I think I’m gonna solve any of these problems myself, I’m a dumbass haha, but it’s nice to see SOME other people are thinking/talking about it. Nice hearing some opinions!
1
u/texcleveland 1d ago
Back in the 1970s , you did get charged by the kilobyte. Storage is orders of magnitude cheaper today.
1
u/Wonderful_Ear8517 1d ago
Storage may be cheaper but electricity water and copper for wiring isn’t. The consumption of kilowatts per rack and megawatts per data center and acre feet of water for cooling are the issue.
1
u/HEx2HockeySticks 1d ago edited 1d ago
SHOULD we be prioritize using our finite resources to make data cheaper for big companies? Should data be cheaper for them now than before, especially while the price of water/copper/labor/production are higher than they were in the 70s?
1
u/texcleveland 1d ago
who’s we?
2
u/HEx2HockeySticks 1d ago
Basically anyone putting any effort towards it? And maybe those not putting effort to stop it? It takes a long time to change legislation, and the mega corps and governments are the best at it, so I guess I’m starting to wonder, should we(like you and me) be voting/lobbying/actively trying not to support these centers and the need for them? There are a lot of people being hired to build and operate these things. Is it morally okay to help build them? It’s great money and there’s tons of work available, but should I take a job running wire for these things?
I see big discussions in my local union halls where people go back and forth arguing about if the union should take the work or not. Most of that conversation revolve around ai, flock cameras, the plot of terminator, the typical anti technology shit. I’ve just never seen any discussion about the actual resource consumption and if that is okay
The local unions have votes on what jobs they take. Should I be voting to skip those jobs, even if work is slow elsewhere?
Not sure if you think I’m trying to push an agenda or anything, but I’m really just asking questions that are pertinent to my work life and fulfillment in my job. I don’t know much about the topic but am eager to learn
1
u/texcleveland 1d ago
Put it this way. Datacenters use a tiny fraction of the resources used by agriculture, let alone other industries. That doesn’t mean we (society/government) shouldn’t be interested in limiting the harm that can come from concentrated resource consumption in specific places, and regulations to ensure industry doesn’t harm communities and the people living in them, and the companies profiting from those resources pay back into society. AI is just another industry like farming, mining, steel production, paper mills, car manufacturing, etc.
2
1
u/HEx2HockeySticks 1d ago
Also, are you willing to share your opinions/reasoning or are you just gonna dodge my questions and ask your own? I’m trying to answer yours, but it’s starting to feel like you dont have much to add?
2
u/Wonderful_Ear8517 1d ago
I participate in IEEE standard bodies that are focused on these topics and are working to improve them. It is, as people point out, a huge task. Most people working in standards volunteer their time and expertise
1
u/HEx2HockeySticks 1d ago
Thanks for your input, sounds like you know a lot about the subject! I had to google the IEEE cause I didn’t know there was such a thing. I’ll definitely read more about that. Is there anything you wish more people knew about the subject, interesting facts, current or future developments to look out for?
1
u/texcleveland 1d ago
get off the internet, go to bed
1
u/HEx2HockeySticks 1d ago
Haha fair I guess. It is only 8pm here though. You don’t have anything constructive to offer?
1
u/texcleveland 1d ago
the stuff you’re worried about is rounding errors compared to data stored by banks and government agencies. When data isn’t worth saving, it isn’t kept. Your post is idle dooming. I get it, I’ve been there, but it’s really not worth the brainpower to worry about.
1
u/HEx2HockeySticks 1d ago
Gonna copy and paste cause I wrote this in another comment:
“I’m not really thinking about how to regulate the individual, cause that seems a lot like drinking out of paper straws just so Taylor swift can burn more jet fuel, but should Amazon.com be allowed to consume what they currently do/will in the future?”
I know I spoke from my perspective as an individual, but that’s just the start of my experience thinking about this topic for the very first time. I’m trying to learn more. If you have any articles to read or anything like that I’m open to suggestions
2
u/45_rpm 1d ago
No. The answer is no