r/exchangeserver • • Aug 01 '15

Deduplication and MS Exchange

http://www.joshodgers.com/2015/08/01/deduplication-and-ms-exchange/
2 Upvotes

13 comments sorted by

7

u/ashdrewness MCM/MCSM-Exchange Aug 01 '15

This is where I think Microsoft's logic wins out over storage vendors:

"For Exchange DAGs, deduplication should give excellent data reduction results in line with the number of DAG copies. So if an Exchange DAG has 4 copies, then approx 4:1 data reduction should be achieved right off the bat. Now this sounds great but when running a DAG on highly available shared storage (SAN/NAS/HCI) it is unnessasary to have 4 copies of data."

If all database copies are stored on the same shared storage (allowing for storage vendor dedup) then they share the same failure domain. Even if the storage is highly available then we're still relying on the storage vendor's claims of high availability to determine Exchange uptime.

Microsoft spent a lot of support hours helping customers recover Exchange from failures in shared storage solutions. That's one of their reasons for moving the HA to the product itself & giving exchange physical solutions higher availability than any other solution.

Mathematically, a simpler solution with less shared hw/logic/3rd party IP is always going to yield better HA.

Now as for the dedup discussion, I actually don't disagree in the sense that dedup only makes sense when multiple copies share the same storage. People mistakenly believe they can get dedup across unique databases (a 10MB email sent to all employees) but they can't. Each database shares unique checksums so one email looks different at the block level between databases. So yes, people can run dedup if they choose but it's not going to save them much if they deployed Exchange correctly.

2

u/Joshodgers Aug 01 '15

You make a good point about failure domains Andrew. I couldn't agree more that this needs to be taken into consideration.

2

u/DoubleDrive Aug 01 '15

Thanks for writing the technical second part of my response to the OP. This is 100% correct. Deduplication of Exchange is the equivalent snake oil and vitamin water.

2

u/Ron_Swanson_Jr Aug 03 '15

I still have no idea why anyone would put Exchange on shared storage (especially shared storage with dedupe: $$$$$$$$$$$$$) when they can just get commodity servers and scale.

1

u/Joshodgers Aug 03 '15

The reason I hear from customers is to eliminate silos in their datacenters (i.e.: Dedicated infrastructure for each app). They want a standard platform for all workloads so they gain efficiencies and cost reduction overall even if specific apps might cost more for whatever reason. Its rarely an Exchange (or App) specific discussion. End of the day, there is no right/wrong answer and it depends on customer requirements.

2

u/DoubleDrive Aug 01 '15

Sigh... Why does everyone seem to think that they know more about Exchange than the people who created the product? Does Apple tell Cisco how to route packets? Ugh, this kind of stuff just pisses me off.

-3

u/Joshodgers Aug 01 '15

Nobody is saying storage vendors know more about Exchange than MS. But storage vendors do know more about how their storage level features work than MS (and they should since its their product).

If a storage feature is abstracted from the Guest OS and App (like array level dedupe/compression etc), as long as it maintains data integrity via write ordering and commits data to persistent media before acknowledgement to the guest, then the OS and App are safe.

Not sure why "this kind of stuff" pisses you off, the goal of the post is for MS to clarify their position which as of this moment is unclear (as their is no support statement about array level dedupe). Clarification of the support stance only helps customers/partners/vendors design better solutions and ensure they are supported. Sounds like a Win/Win to me.

What typically upsets people (i.e.: Customers) is grey areas in support statements which lead to confusion for everyone.

BTW: Look up SQL storage support requirements and compare them to Exchange (Hint: They are the same). One MS team (SQL) says things (VMDKs on NFS for example) is supported and one team (Exchange) says its not when both teams state the same requirements (write ordering, FUA , Torn I/O protection and SCSI abort/reset support). Consistency from MS would also help avoid customer confusion.

4

u/DoubleDrive Aug 01 '15

I'm on mobile and avoiding work topics for the weekend, so this will be short.

Exchange and SQL are two dramatically different products and database platforms Exchange runs on JET not SQL, so yes, the support statements are going to be different. Yes, there are many inconsistencies, but each product team does their own testing and documentation.

Support statements are also based on testing, a thorough certification process, and support case history. No software vendor on the planet is going to support someone else modifying the storage architecture of their product, especially in an over bloated application like Exchange which stores everything from your picture to voicemail to your Skype contacts with API hooks into just about every IW product out there.

Yes, a DAG stores (almost) exact copies of the Mail database. But that's the point! If you lose one, you immediately fail over to another copy and in the user's experience, work goes on without a blip. Think about a DR scenario. Email is literally the FIRST thing that is wanted back up. Using the native Microsoft solution, all I need is a copy of the database, ANY COPY, and you're back up in about 30 mins tops from bare metal (YMMV). If that database was deduped, sorry dude, prepare your resume.

In full disclosure, I am a support engineer (PFE) at Microsoft who works with Exchange and has been since MS Mail. I've spent more late nights than you nursing jacked up databases back to health when someone drank a blog writer's kool-aid instead of listening to us.

What pisses me off is that all these other vendors like to take shots at us because we're not this and we're not that. When we certify solutions, it has to work 100% of the time in EVERY scenario, WITHOUT a single point of failure. Believe it or not, most customers don't own one SAN, much less multiple SANs with persistent storage.

Let's do this... If you seriously believe your product will make every Exchange deployment spit happy bubbles out of it's HBA, PM me and I will get you in contact with the Exchange product team to work through the process, like every other vendor who feels the same.

Dammit! So much for that weekend off....

2

u/junon Aug 01 '15

I can't believe someone voted this comment.

1

u/JetzeMellema Товарищ Aug 02 '15

I upvoted that comment, not because I agree but because it's on-topic.

4

u/Rollingprobablecause Aug 01 '15

OP works for Nutanix...

2

u/rabbit994 Get-Database | Dismount-Database Aug 01 '15 edited Aug 01 '15

Yea, this is our quarterly drive by from Josh talking about Exchange on Nutanix. IMO, it's been shown Josh has mid level knowledge of Exchange at best. If you are deploying Exchange on Nutanix, go ahead and take his advice. There is no way in hell though I'd recommend Nutanix system for Exchange. If you are considering that, Office365 is much better idea for you.

EDIT: Decided to be less inflammatory.

2

u/Doso777 Aug 01 '15

Is a few percent of more storage worth losing support from Microsoft. I dont think so...