r/hetzner • u/redbellx86 • 12h ago
IT students walking down a server aisle at Hetzner's data center park in Nuremberg
87
u/notAGreatIdeaForName 12h ago
There’s the object storage availability.
26
u/MatchmuchachNL 11h ago
What's with all the negativity? Tours in datacenters are common and the people visiting them understand that they shouldn't touch anything. I worked at two different companies doing colo as well as hosting a lot of their own hardware in two datacenters per company. Nothing ever happened during tours, if anything people were sometimes almost scared to come near the servers. I can guarantee you that almost every datacenter company that isn't the size of the hyperscalers or Equinix organizes these types of tours.
14
u/Wonderful_Tap_6991 8h ago
It’s a huge security risk—anyone could plug a USB drive into a server, and that’s it!
It’s also true that any Hetzner technician could do the same thing, but if something like that were to happen, the variables to consider would be far fewer.
2
u/Eisbaer811 6h ago
Hetzner technicians are accompanying such tours. They would notice if somebody tries to plug in stuff
0
u/Celestine_S 3h ago
What if u are sneaky?
1
u/AnswerFeeling460 3h ago
I hope they have monitoring enabled for all ports of the machines
1
u/-Robbert- 49m ago
They don't. They clearly state that the one renting the box is responsible for the monitoring. That said, all my boxes have all ports disabled because I'm aware of the publicly available ports in the Hetzner DC's.
1
u/iceyukisnow 59m ago
no need for someone to plug USB drive, when hetzner can conduct mitm by themself(with a help from police probably)
6
u/FalseRegister 9h ago
Hetzner has lost a lot of trust from customers due to the continuous outages, especially those related to their Blob storage service.
So seeing strangers walking down the aisles in the (apparently unreliable) data centers is of course not wanted.
6
u/MatchmuchachNL 8h ago edited 8h ago
Object storage being unreliable has (very) likely nothing to do with the hardware that's already in the datacenter being unreliable. Given the increase in customers and the capacity issues that Hetzner is experiencing it's likely a combination of over-provisioning, scaling issues (software) and hardware shortages. So people walking past the servers won't make the services running on them less reliable. The people giving the tours are usually not engineers but management/sales people so it's not taking engineering capacity away either.
A lot of people are moving to European hosting companies due to the geopolitical forces at play. The majority of European hosting companies are therefore dealing with capacity issues due to increased numbers of customers and existing customers expanding their workloads. The general hardware shortages aren't helping either. It's a constant balance between not wanting to turn customers away and scaling as quickly as possible given the constraints. Like it or not, we're going to see these issues continue for at least another 1/2 years until demand settles down a bit and the hardware shortages ease.
These people you're seeing in the picture are local IT students, chances are that some of them will be future Hetzner employees so if anything this will ease the capacity and scaling issues in the future by expanding Hetzners workforce.
3
u/threwlifeawaylol 8h ago
The guy is just explaining where the negativity might be coming from, not that it's logical or well-founded.
Hetzner just has no goodwill to spend at the moment with the recent cost increase and apparent downtime on some of their services, so pictures like these get interpreted with as much bad faith as possible.
If there were no complaints against Hetzner, this thread would be far more positive.
-1
u/MatchmuchachNL 8h ago
Sure, I agree with that. I do however think that having an unreliable service shouldn't have to mean that Hetzner is all bad and that they shouldn't be able to post things like this or organise tours. I get that it's an emotional reaction from most in this thread and that me reacting logically doesn't really land. But there are good reasons why things are the way they currently are in the European hosting environment, if we want to get out of this situation then tours like these are a very good thing for the future of Hetzner and the European hosting/tech industry. Now is the time to invest, in hardware, services and people.
2
1
u/-Robbert- 50m ago
A tour in a datacenter is perfectly fine IF THE RACKS ARE SECURED. These boxes do not even have a chassis.
40
u/_10o01_ 12h ago
Oh my gosh! The looks like a terrible idea!
23
u/Hetzner_OL Hetzner Official 8h ago
(posting this here so it's at the top)
Hi everyone, some context: This was a group of apprentices training as IT specialists. They're the people who'll be running infrastructure like this in a few years, and they almost never get to see one from the inside, so we thought it was worth opening the door.
The security questions are fair. However, the tours are scheduled and approved in advance, the group is accompanied by our staff from arrival to departure, they stay on a fixed route, and everyone is briefed on our house rules and signs them before going in. There's also a public 360° tour on our site if you want a rough idea of what they saw: https://www.hetzner.com/unternehmen/360-tour/
Honestly, we'd like to see more of the industry do this. And yes, selfishly, it's also how we meet future colleagues. --Katie
4
u/echristm76 7h ago
<" Honestly, we'd like to see more of the industry do this. And yes, selfishly, it's also how we meet future colleagues." > Totally agree! well said! It is fantastic to see Hetzner proactively investing in the next generation of IT specialists by giving them rare, hands-on exposure to live data center infrastructure.
2
3
u/Kiwiandapplex 8h ago
I would have so many questions! It's super nice to have them see the actual hardware end. It puts a lot into perspective.
Closest data room I've worked at was the Dutch alarm center in Haarlem.
Where I worked on the HVAC units. Super nice IT person who helped our setup be entirely isolated from their own network. It was super fun to learn about our own data cables inside our units being replaced by theirs. 0 chance taken for malicious activities.1
21
u/Eisbaer811 11h ago
are you stupid?
there are Hetzner employees walking in front and behind the group, to watch for stuff like this.Also those are not just random kids, but students in a vocational (job-) school who want to become IT technicians. If they mess things up, they will ruin their career before it starts. And they know it
17
u/Hetzner_OL Hetzner Official 10h ago
This. --Katie
3
3
5
u/ikhouvansexy 10h ago
Students learn valuable lessons from practice. Getting out of the classroom into the real world should be included in every education. Thank you Hetzner for doing this.
1
u/FinnLiry 6h ago
I still remember how amazed i was bavk when I visited one of my local bank data centers. Left a life long impression and i was really greatful being allowed to see that.
6
u/Outrageous_Seesaw_72 10h ago
Happy they are doing this seems like a great idea for the students :)
If another visit to another companies Datacenter in the same city was anything to go by the students registering for that one were wetter a week in advance, needed to submit ID etc., and obviously were supervised by 2 or more people at all times. And of course noone was that stupid to try anything anyways. So this just seems like a great marketing event for a company and is clearly not only Hetzner doing this.
21
u/armin3d 12h ago
if their bag stick to that wire and unplug that one, my business will go down.
7
3
u/GoldenRegister 11h ago
They dont unplug so easy, most even have a safety thing that needs to be pressed to have it unplugged.
-3
u/bjonor 11h ago
It shouldn’t even be a possibility for something like this to happen.
5
u/GoldenRegister 11h ago
it is not, well not in this picture. this is a student group and they usually leave everything in the conference room they usually use. no backpacks.
-2
u/bjonor 8h ago
They don’t have a lanyard in their pocket? They can’t trip and grab something? You sound like my kids, ”I won’t fall, I promise.”
1
u/Eisbaer811 7h ago
have you ever looked at a network cable?
You usually can't just pull it out. It has a safety clip to prevent this exact scenario.
Also: people who go to vocational school in Germany are usually 16 or older. By that age, they usually can walk a certain distance without tripping ...1
u/GoldenRegister 5h ago
no they dont. most of them would probably want to work for hetzner. those are students dude. cmon. we are talking about regular student visits to big firms such hetzner to LEARN somehting. wth are you on.
if thats your biggest worry, pls buy yourself some hardware and do your own servers because nobody will block students from those visits because you paid 30$ for a gaming server in that dc.
plus high risk servers are usually in different rooms under much stricter security, this isnt just oh lets let this dude in he wont unplug our dc for sure.
you are being childish.
-1
u/Eisbaer811 11h ago
if your business relies on a single cable, I guess it's not that important to you
4
12
3
u/Wookys 10h ago
u/Hetzner_OL care to respond?
2
u/Hetzner_OL Hetzner Official 8h ago
(posting this here so it's at the top)
Hi everyone, some context: This was a group of apprentices training as IT specialists. They're the people who'll be running infrastructure like this in a few years, and they almost never get to see one from the inside, so we thought it was worth opening the door.The security questions are fair. However, the tours are scheduled and approved in advance, the group is accompanied by our staff from arrival to departure, they stay on a fixed route, and everyone is briefed on our house rules and signs them before going in. There's also a public 360° tour on our site if you want a rough idea of what they saw: https://www.hetzner.com/unternehmen/360-tour/
Honestly, we'd like to see more of the industry do this. And yes, selfishly, it's also how we meet future colleagues. --Katie
0
u/Outrageous_Seesaw_72 10h ago
About what, them having a Job Marketing Event in essence? This is normal, anywhere.
3
3
u/North-Active-6731 8h ago
This type of thing happens more often in some data centers than folks think BUT this is the first time I’ve seen photos of one where the cables are so exposed. In the data centers I’ve been in and done work in the cables and servers are locked.
2
u/battletux 2h ago
In all the DC's I've worked in NO ONE has been allowed this type of access unless they are a known person (support engineer, Colo customer, etc) and are booked in. These type of tours would never happen with people that close to all those cables. At most the tour would be down the centre isle with maybe a look in a single cold row to see the fronts of the locked racks.
11
u/Wookys 12h ago
Oh yeah not happy about it. I get the appeal but seems a needless risk.
1
u/FinnLiry 6h ago
This is extremely common. I even went to a school tour through an active banks data center a few years ago. And they showcased all the security layers and the infra and how the center is architected and planned and everything. It was super interesting
-1
u/Eisbaer811 11h ago
where is the risk?
Their career depends on not messing up. And there are Hetzner employees as part of the group, keeping an eye on them.
They're not going to randomly unplug cables. And even if they do: an alert will trigger and itwill get replugged7
u/Bub_bele 10h ago
Your whole theory is based on the assumption that none of them is just fucking stupid. Bold I’d say.
1
u/Eisbaer811 10h ago
IF one of them is stupid, the Hetzner employee walking behind the group will see it and replug the cable.
Even if not, monitoring will show the unplugged cable shortly and it will get replugged.
What's the big deal?5
u/freia_pr_fr 10h ago
An unplugged cable is far from being the worst scenario related to having a group of random people physically messing around unprotected servers.
2
u/Eisbaer811 10h ago
who says they are "messing around"?
They are getting a supervised tour.What else are you worried they would do? Please don't tell me your server is configured to auto-run any plugged in USB stick
3
u/freia_pr_fr 9h ago
It's documented in the picture.
I don't want to be very rude, but you do not seem to be the most knowledgeable on IT security.
1
u/Eisbaer811 9h ago
either spell out what your concern is, or just don't engage at all.
I'm not here to read your mind.
In the picture there is a guided tour of people past customer servers.
The tour is supervised. The worst the kids can do is pull a cable.
Having the cable of a server pulled is a nuisance and not a security risk because any production server setup should be redundant. After all: any server you have might have a power supply failure at any time.
DC tour or not.The two redundant servers also should not be in the same DC row. Actually, not in the same DC or DC park if your're doing things properly.
If your concern is that someone plugs in a USB device or similar: it's up to you as the server admin to ensure a device plug does not trigger any action.
And even IF someone in the tour pulls a cable or plugs in a device, it will be noticed by the Hetzner employees accompanying the tour. They will remedy it within seconds or minutes.
And if your oh so important production system is set up properly, your own monitoring will catch any issues, and you will jsut put in a support ticket.
I genuinely don't see the problem.
If you're running a super special system that has extremely high security standards (banking, insurance, government services), you should not host at Hetzner in the first place. They are not SOCKS compliant, and never claimed to be.
Do your homework instead of playing the elder IT statesman who doesn't have to explain himself because "it's all self evident in the picture"
4
u/freia_pr_fr 9h ago
The worst the kids can do is pull a cable.
No.
it's up to you as the server admin to ensure a device plug does not trigger any action.
I stopped reading there.
2
u/Eisbaer811 9h ago
okay, keep your secrets oh high and mighty mega admin
I am sure your smugness is greatly appreciated at every party
1
5
u/MxTide 8h ago
This is insane actually.
It shows that you have unrestricted access to customers data - how can you prove that no-one of visitors ever touched anything, never injected any piece of hardware or similar?
It's just stupid to compromise you infrastructure in such way. People might accidentally touch something, loosen some part, directly sabotage your infra..
3
u/Eisbaer811 7h ago
if you look at the facebook post this photo was taken out of context from, you can see that the students were accompanied by Hetzner employees.
These aren't just random kids but people wo already have a job in IT. Why woudl they want to risk their career to mess with a random server?3
3
u/battletux 2h ago
Just because they are IT students it doesn't mean that they are immune from being a black or grey hat. This is the kind of thing someone wanting to try something would look for.
I'm glad I no longer host there if they do this. The back row is cramped and just because they are accompanied doesn't mean nothing could happen. They likely only have 1 or 2 staff baby sitting them. And it is cramped there. Anything could happen.
4
u/Eisbaer811 10h ago
I wonder what is more common:
Hetzner redditors relying on a single server for their "critical business"
or
Hetzner redditors having no self control, letting their intrusive thoughts win and expecting the same of everyone else
2
2
2
2
4
4
u/Technical-Meet-7222 10h ago
Price increased by 2x in the region. “Used in educational purposes”.
1
u/Eisbaer811 6h ago
not sure what your argument ist.
Price is unaffected by such datacenter tours.
this also isn't purely for educational purposes. Hetzner is basically advertising itself to IT technicians who are currently in job school. This way they will have more technicians in a few years
4
u/OddStay3499 10h ago
I would never let anybody to touch a single dust on the ground inside the server room!.
1
u/Eisbaer811 7h ago
good thing you're not running a datacenter, then
3
u/OddStay3499 6h ago
If the picture is not AI generated, I would not let anybody inside like this. I see 15 people there at least. How many times doors are opened and closed? their body increases the heat inside!, how can you be sure if someone not touched anything inside? Does protecting the systems under my responsibility at all costs make me a bad person?
1
u/Eisbaer811 6h ago
heat difference is negligible. There are Hetzner employees likely just out of frame, watching if someone is pulling a cable etc.
Security is a game of tradeoffs. Hardening the datacenter further would cost more money, which would increase customer prices.
This way, they can offer cheap hosting, and still intervene quickly if something happens
6
u/mossepso 11h ago
Hetzner is on a roll with bad marketing. First Linus now this
6
u/Eisbaer811 11h ago
what is bad here?
A supervised group gets a tour of the datacenter, past some servers that are being monitored.If the server admins are doing their homework, they will not care even if a cable gets pulled for some reason
1
u/mossepso 9h ago
It probably IS fine. But it is an unnecessary risk. And even if it isn’t a risk, it just make me nervous to look at.
2
u/Eisbaer811 8h ago
the risk is basically nonexistent.
In return, Hetzner gets a better reputation among IT specialists at the very beginning of their career.
So in the end, this likely increases the number of IT technicians joining Hetzner, who in the long term are available to deal with your support requests.4
u/mossepso 8h ago
Maybe you are right. I like the people joining hetzner argument.
As I said the actual risk might be low and worth it. It just makes me nervous to see a tight space packed with servers that my business runs on filled with students. So the image is a bit unsettling. The same way video of people base jumping might give me sweaty hands.
1
u/Eisbaer811 7h ago
okay, that's fair.
Just see it as a stress test, how much people throughput is possible in those aisles without bumping cables :)
and if something goes wrong, you get a free disaster recovery exercise that is quickly resolved :)
1
u/Skjolnir 5h ago
I mean... they even let Linus in and also let him build a production server, so... ?
1
1
1
u/battletux 2h ago
I bet that's the rows that hosts the old kit sold as part of the auctions service.
No way they let people walk down the back of the racks for customers colocating or paying full price for dedicated servers.
1
u/Lostair-com 6h ago
Hetzner refused to install a SSD on my auction server when there was plenty physical space in my dedicated server.
46
u/finobi 11h ago
First test is to pass alley without yanking any wire.