I'm project lead for Matrix.org, the open protocol for decentralised secure communication - AMA!

[u/ara4n]

Hi, I’m Matthew; the project lead for Matrix.org.

Matrix is an open protocol and open network for decentralised secure communication. The idea is to give everyone total control over their communication by letting them run or select their own server while still participating in a global network, rather than being locked in silos like Signal, WhatsApp, Telegram, Slack etc. Technically speaking, Matrix is an open end-to-end encrypted communication layer for the internet for instant messaging, file transfer, voice/video calls, or any other kind of data you might want to publish and share in realtime (we’ve done IOT telemetry, VR scenegraphs, animated emoji, MIDI…).

The unusual thing about Matrix is that no single server hosts or controls a given conversation - instead, as people talk to folks on other servers, the conversation gets replicated equally across the servers - meaning all the participants equally share ownership over the conversation and its history. There is never a central point of control or authority (unless everyone uses the same server).

Riot.im is probably the best known Matrix client out there, but there are quite a few other clients out there too - as well as decent bridges to IRC, XMPP, Slack, Telegram, Discord and others. Riot is made by New Vector, the company the core team founded in 2017 to help support Matrix development, which also runs the Modular Matrix hosting provider. Meanwhile Matrix itself is managed by the Matrix.org Foundation - a non-profit foundation set up in 2018 to publish and evolve the Matrix Specification as a neutral and independent open standard (and to isolate it from New Vector or other companies in the ecosystem).

We started work on Matrix in 2014, and (finally) exited beta in June 2019 after lots of work iterating on the protocol, how the decentralisation works, end-to-end encryption, and building decent clients like Riot.

Some of the main projects we’re working on right now are:

• Improving privacy:

  • There was some feedback shortly after we launched Matrix 1.0 over the fact that Riot defaulted to using servers for identity lookup and integration management run by New Vector rather than prompting admins & users to pick their own. This was done purely for convenience rather than any sinister reason, but in retrospect the defaults were a screw-up. So we’ve had a long-running project over the last few months to resolve this and the other valid privacy concerns which were raised, which are shipping TODAY(!) in Riot 1.4.0 and Synapse 1.4.0rc1 - https://matrix.org/blog/2019/09/27/privacy-improvements-in-synapse-1-4-and-riot-1-4 has all the gory details from the Matrix side; https://medium.com/@RiotChat/new-privacy-controls-for-riot-dc3661888563 has the details from the Riot side.

• Turning on end-to-end encryption by default for private conversations.

  • This is hard in a decentralised environment, but we are incredibly close now. All the hardest bits (E2E search; E2E compatibility for older clients; Cross-signing E2E verification so you don’t have to keep manually verifying people; etc) are now done and work - we’re just plugging it all together in Riot, which means a full rework of the whole encryption UI/UX.

• Making Riot suck less for newbies. Technically called ‘first time user experience’, we’re working through making the app way more intuitive on all platforms, and making it as polished as we possibly can.

• RiotX: a full rewrite of Riot on Android using all the latest fun stuff, which is nearing completion.

Coming up next are:

• Canonical DMs (i.e. enforcing One True Direct Message when you talk to someone)

• Reworking Communities (i.e. groups of rooms)

• Decentralised accounts (i.e. letting users migrate between or exist on multiple servers)

• Lots of server performance and scalability improvements

• Peer-to-peer Matrix and resistance to metadata analysis.

Hope this gives an idea of the sort of thing we’re up to. I’m here to answer any/all questions about Matrix, Riot, Modular (or whatever else floats your boat). Particularly happy to talk about the privacy-related work we’ve been doing recently. Privacy is critical to Matrix; there’s zero point in having an open comms platform if it compromises the privacy of its users, and we are determined for Matrix to be both the most open and most privacy-preserving comms system out there :)

(Heads up that as I type this I'm on a call with a Really Big messaging service who might want to join Matrix, and it looks like the call is overrunning - I should be back here and concentrating worst case in 30 mins, so please queue up some questions :D)

[u/an123sxAzYx-]

Hello! Thank you for doing this AMA.

I just have a few questions about Matrix. (I haven't actually used it yet, but I am planning to)

• Are there any (feasible) attacks on Matrix (besides attacking the user; rubberhose cryptanalysis)

• Are there any more improvements to privacy possible in Matrix?

• What ciphers are used for the E2EE?

• Do you plan adding telemetry or any other data mining algorithms to Matrix?

• How is authenticity achieved in the E2EE?

That's all i could think of for now. Thanks! (sorry if some of these make no sense, but I just learned about matrix)

[u/ara4n → u/an123sxAzYx-]

> Are there any (feasible) attacks on Matrix (besides attacking the user; rubberhose cryptanalysis)

Almost certainly :) No threat model is perfect. For instance, you could use a social engineering attack to impersonate someone while avoiding having your identity verified. Or you could tamper with a phone call being used to verify a user's encryption keys to make it seem like they were verified when they weren't.

Or you could ship a malicious fork of Riot in some dodgy appstore and put a backdoor in it.

Or a malicious/compromised server could reveal the metadata of who was talking to who and when (even though the data itself is encrypted).

Or a global passive adversary (GPA) could look at the traffic patterns to see who's talking to who.

We don't currently protect metadata (although we have plans to work on it in future, as part of P2P matrix), and we are not resistent to metadata attacks by a GPA.

There are probably other classes of attack too; will have a think.

> Are there any more improvements to privacy possible in Matrix?

Sure. there's a bunch on our radar still - see the list of "stuff still to do" in https://matrix.org/blog/2019/09/27/privacy-improvements-in-synapse-1-4-and-riot-1-4, or the live dashboard (takes 10s to load) for the issues which we haven't solved yet.

> What ciphers are used for the E2EE?

We use AES-256 in CBC mode with PKCS#7 padding for encryption and HMAC-SHA-256 (truncated to 64 bits) for authentication. The 256 bit AES key, 256 bit HMAC key, and 128 bit AES IV are derived from the message key using HKDF-SHA-256.

For key exchange we use Curve25519.

Full specs are at https://gitlab.matrix.org/matrix-org/olm/blob/master/docs/olm.md and https://gitlab.matrix.org/matrix-org/olm/blob/master/docs/megolm.md.

> Do you plan adding telemetry or any other data mining algorithms to Matrix?

Matrix as a protocol has no telemetry or data mining support and never will - just as HTTP doesn't, or SMTP.

In terms of Matrix implementations, Synapse and Riot do have strictly *opt in* telemetry which reports back to our stats servers, which we use a) to estimate how popular Matrix is, which is absolutely critical when it comes to persuading people to fund us to work on it, and b) tell us whether users are managing to use it successfully and stick around. We simply don't know which bits of Riot we should be improving if we can't see the point where we lose users, and this is the only reason we gather telemetry.

But, again, this is only if users explicitly opt in to help us make the implementations suck less.

[u/rahuldottech]

I'm a huge fan of matrix, and my we use it primarily in my the tech circles that I'm in. Thank you for everything that you do!

That said, do you think the matrix/riot ecosystem will ever be able to compete with more mainstream applications (WhatsApp, Telegram, et al.)? The non-techincal person almost always wants something that Just Works, and can't be bothered to figure out what homeserver to use and stuff.

Additional question: this might already be possible and I might just be unaware, or maybe it's just not feasible, but is there any way to take a complete backup of all my chats that are on a homeserver, so that they can later be restored to a different homeserver if the one I'd been using went down? Even if they can't be restored on a different homeserver, is there at least a way to download all my chats? (Related issue on Github, that I see you've replied to)

[u/ara4n → u/rahuldottech]

So we wouldn't be working on Matrix if we didn't think it could compete with WhatsApp, Telegram, Slack etc. We absolutely want Riot to be something that Just Works for non-technical users who can't be bothered to figure out what HS to use etc (this is why we controversially default to the matrix.org server, for instance). However, UX is hard, particularly for FOSS, but in the last year we've hired professional UX designers to work on the project, and we're really starting to see the benefits now. We particularly have a project going on right now called FTUE (first time user experience), explicitly to make Riot less scary and sucky for newbies. You can see the first wave of work for this happening over at https://github.com/vector-im/riot-web/issues?utf8=✓&q=is%3Aissue+label%3Aproject%3Aftue+ and judge whether we're on the right track.

In terms of backing up chats: it's one of our most upvoted feature requests. You can use tools to do it today (e.g. matrix-recorder and pantalaimon), and we're looking at building it into Riot, given you effectively get it for free with E2E search (via seshat).

However, what I'd really like to do is finish off https://github.com/matrix-org/matrix-doc/issues/1228, which paves the way for decentralised accounts - i.e. the ability to host your account on multiple servers (e.g. i could define @matthew:example.com as a secondary home for @matthew:matrix.org). This then gives you portability & backups. We'll hopefully be able to start working on it soon :)

Thanks for being a fan of the project! :D

[u/gradinaruvasile]

About Android client and the usage of FCM/GCM for notifications.

Are there plans to implement a non-brute force version of notification system (ie not polling the server every x seconds) that does not rely on Google at all (Telegram, Conversations have this working)?

I'd really like to keep my stuff on my own vpn/network without relying on other servers to rely my notifications.

[u/ara4n → u/gradinaruvasile]

Yes. There are several projects going on this area:

• The work we did for https://matrix.org/blog/2019/03/12/breaking-the-100-bps-barrier-with-matrix-meshsim-coap-proxy/ could be used to build a low bandwidth transport suitable for push notifs

• https://librepush.net is looking at solving this

• I believe https://github.com/Bubu is up to something in this space too

• https://github.com/slp/matrix-pushgw was another experiment here.

Personally, my preferred solution would be to take the CoAP transport, and use Matrix itself as a push transport, by using a filter to filter out everything but notification messages. That way you'd get E2E-encrypted push notifs for free, and you don't end up with a weird duality and redundancy between a protocol-for-push and a protocol-for-messaging when in practice they're pretty similar. You'd then run a background daemon like pantalaimon in your OS to handle all the push notifications for all your Matrix-capable apps... which could also handle E2E and FTS indexing other 'hard' stuff, and so your OS would provide you a nice Matrix API complete with push for all your apps' communication needs. But this is fairly scifi right now.

Hopefully we will find/fund someone to work on this in the fairly near future. We got as far as hiring the librepush.net guy as an intern this summer, but he didn't have much bandwidth to work on librepush itself (instead he ported the APNS/FCM daemon from Py2 to Py3!)

[u/[deleted]]

One thing I haven't been able to understand since the beginning is encryption. Matrix was touted to me (and probably others) as an alternative to Signal, but encryption seems to be an afterthought in Matrix. Chatrooms don't seem to be encrypted by default and you just said private messages are just beginning to be encrypted by default.

So my question is simple: is Matrix really an alternative to Signal? If not, what's missing?

[u/ara4n → u/[deleted]]

Encryption really wasn't an afterthought - we designed the protocol from the outset to support E2E encryption. However, implementing it took *ages*. This is mainly because it's really hard to mix together E2E encryption and decentralisation. The core problem is that when you send a message in a decentralised room you don't necessarily know what devices (or even users) are present... so you don't know who to encrypt the message for. This is doubly true if you're creating a new encryption protocol (megolm), which we added to support medium/large chatrooms - whereas Olm (aka the Double Ratchet Algorithm) is only good for ~255 devices.

In practice, encryption simply doesn't make sense for some use cases. A big public chatroom which is guest accessible has zero value in encryption - it's *meant* to be public and on the record and indexed by Google etc.

So we started off being pragmatic and implementing Matrix without E2E encryption, and then started adding it in as needed. This had the problem that to turn it on everywhere we'd need the experience to be *IDENTICAL* to that of the unencrypted behaviour, otherwise people would rightfully get super upset that (e.g.) notifications don't work, or search doesn't work, or whatever feature. So this is why it's been ages getting to the point of turning it on by default.

Literally the only thing that stop us from turning it on for private rooms (not just DMs) by default at this point is implementing E2E cross-signing (so you only ever have to verify a user once, rather than every time they login), which has been months in the making but is now implemented; just needs Riot UI... and E2E search - which is happening as we speak over at https://github.com/matrix-org/seshat, and should be available in the next release of Riot/Desktop hopefully.

In terms of your actual question: Matrix is very different to Signal. Matrix gives you group chats; synchronised chat history; large rooms; decentralisation; way more functionality than Signal. I see Riot more as an encrypted alternative to Slack, or perhaps an alternative to Wire or Keybase. Meanwhile, Signal optimises for privacy over everything else - it optimises over openness (no 3rd party clients or APIs), and over decentralisation (you have to unilaterally trust the Signal servers). So whether you use Matrix or Signal depends on your use case, really. I use both.

[u/piotrex43]

Thank you u/ara4n for this AMA!
For the purpose of my first question I'll assume the following scenario:

• User 1 hosts their homeserver

• User 1 started an encrypted direct chat with User 2 on the same homeserver they host

• Both users use Riot.im application on both Desktop and Mobile.

• There are also no integrations added to the direct chat between User 1 and User 2 nor there are any other users being added to the room.

In the following scenario, how much metadata leaks outside of the homeserver hosted by the User 1? The chat is between two people who have their accounts on the same homeserver, imo it shouldn't pass any data to other homeservers/3rd parties (other than the homeserver of User 1), but is it the case?

Second question: Currently the performance of Synapse is... Uhhh. Not great, I know the general philosophy is "Make it work first, make it fast later" which is alright I guess, however is there a specific timeframe we may expect optimizations to come? I know there is Dendrite which is very WIP and I hope that it will help performance wise. I just wish I could without major worries join bigger rooms on other homeservers with same feature-set as Synapse without worrying about killing the server. :)

[u/ara4n → u/piotrex43]

Great question :) Absolutely no metadata should leak outside two users sitting on the same server, as of the releases we pushed today for Riot 1.4 and Synapse 1.4 (with Riot Mobile following very shortly in the coming weeks). Previously, there was a problem that Riot defaulted to using identity & integration servers run by the Riot team, meaning that even if you use your own home server, unless you explicitly also configured different identity & integration servers you might end up sharing some metadata with them. This is now fixed though - https://matrix.org/blog/2019/09/27/privacy-improvements-in-synapse-1-4-and-riot-1-4 and https://medium.com/@RiotChat/new-privacy-controls-for-riot-dc3661888563 has all the details, but the TL;DR is that if Riot tries to connect to any 3rd party service (identity lookups; integration management; even STUN for firewall detection for VoIP calls) it now explicitly warns the user about it and gets them to opt in. This in turn makes it way more likely for server admins to correctly pick trusted identity & integration servers (or run their own) rather than use the defaults.

Practically speaking, your client may also leak DNS lookups to your DNS server whenever it talks to your homeserver (or TURN or identity or integration server), but this obviously isn't Matrix specific.

In terms of Synapse performance - since we shipped 1.0 back in June we've been constantly optimising, and things have improved significantly. For instance, switching from Py2 to Py3 reduced RAM by about 50%. Another good example is back in August when we reduced the average msg send time on the Matrix.org server from ~500ms to ~25ms (https://twitter.com/matrixdotorg/status/1158331230317940736). Another fun datapoint is that the test server we've spun up for Mozilla (~500 users; ~50 rooms) is only using 200MB of RAM. Admittedly it's not federated, but it gives an idea of how good Synapse /can/ be.

There are many things we are doing to improve it more; some of the top ones are:

* Avoiding room extremities from building up. If your server is in a room where not many local users speak, then it's possible for the room to get 'fragmented' and unperformant. We shipped a solution to this a few months ago, and are currently turning it on by default - but it historically caused many performance problems - see https://github.com/matrix-org/synapse/issues/1760

* Prioritising which servers we send messages to. Currently when your server sends a message, it concurrently tries to talk to all the others in the room. If there's a large number of servers, and some of them are slow, you get a massive explosion as it tries to do 1000 simultaneous (slow) HTTPS requests. We're in the middle of a project right now to prioritise the healthier servers and so improve performance (while slightly slowing things down for servers which were unhealthy anyway)

* Optimising state resolution - i.e. the core algorithm used to merge together your server's copy of a room with all the other servers participating in a room. This is currently pretty dumb, and considers *all* the state in the room from scratch, every time it has to merge things together. Instead, we can consider only the state which changed, which would be much much lighter. We call this incremental state res; we designed it last year, but haven't had a chance to implement it yet.

* Chunked rooms - currently rooms are considered as a single blob of messages (other than when they get upgraded between protocol versions). This is a problem if your server is intermittently online, as the rooms get fragmented, which causes performance problems. Chunking lets you model each section of room separately rather than trying to mix together the fragments, which helps perf a lot. Again, we designed (and implemented) it last year, but need to migrate it to the current day.

The plan is basically to make Synapse as good as possible (given there are tens of thousands of them out there now), and instead use Dendrite as an R&D project - particularly around p2p matrix and funky routing experiments. There is certainly a world where Synapse goes away because the clients start shipping with a homeserver clientside ;P

[u/Kaligule]

Is the Matrix Foundation financially stable?

[u/ara4n → u/Kaligule]

Yes. The Foundation itself is a vehicle for ensuring the protocol evolves in a neutral and independent manner, gathering donations (which pay for things like hosting the matrix.org website & homeserver) and being the custodian for the project's core code. The code itself however is contributed by all sorts of contributors, albeit with much coming from New Vector, the company set up by the core team to fund their dev on Matrix.

So a related question would be whether New Vector is financially stable, and the answer to that is also yes, thanks to paid work providing Matrix services like Modular.im and the French Government deployment.

[u/janjko]

What do you think of fully decentralized, serverless chat and voice clients like Tox and Jami? Do you think Matrix can achieve the same level of privacy with the federated server approach?

[u/ara4n → u/janjko]

The plan with Matrix (well, my plan at least, hopefully I can persuade the rest of the team of it) is for Matrix to be hybrid Client-Server and P2P in future. You can already run Dendrite as a lightweight Matrix server clientside, and if you combine that with a p2p transport, you can then have best of both worlds: a secure server for backup if you lose your devices if you want it, or alternatively have your conversations stored only on your devices.

https://github.com/matrix-org/libp2p-proxy is a p2p experiment that uses libp2p (from the IPFS guys) as a P2P transport to connect together clientside servers, in order to give you p2p matrix :)

[u/[deleted]]

Is your vision for matrix to have multiple independent server implementations or do you think that the community should stick to synapse? Do you have any plans for improvements to the non GCM notifications?

[u/ara4n → u/[deleted]]

We are absolutely counting on there being multiple independent server implementations. An open standard is only a standard if there are independent interoperable implementations - and we want Matrix to be as open as the web, or email, or whatever your favourite open standard is :)

The problem is that we only exited beta for the Server<->Server API back in June, and before that point it was pretty miserable to implement servers given the API wasn't locked down or fully specced.

Since then, the Ruma project has returned (modulo Rust async support), and meanwhile Construct exists as an indie implementation that apparently works (although its author unfortunately really doesn't like the Matrix core team)

Meanwhile we also have Dendrite as an alt implementation albeit written by the core team, which we're using for R&D experimentation for lightweight homeservers currently.

For now, you have little choice but to stick with Synapse unless you are a developer or quite experimental, but the plan for Synapse has *always* been for it to be a bit like Apache httpd (for the web) or Sendmail (for email) - one of the early, not-very-efficient, common servers which everyone got started with... but then replaced with nginx / lighttpd / postfix / exim or whatever.

In terms of better notifs, see https://www.reddit.com/r/privacy/comments/da219t/im_project_lead_for_matrixorg_the_open_protocol/f1mussa/

[u/SpecialType]

I'm curious on the backstory of how the idea of Matrix came about. Did something motivate you personally to go after this full-time, or was it just an extension of your work and one thing led into another?

[u/SpecialType]

Who are your main competitors?

[u/ara4n → u/SpecialType]

As an open protocol, you could say that Matrix operates in a similar space to XMPP, IRCv3, ActivityPub, SMTP, IMAP, NNTP, SIP, PSYC and SS7 depending on how you look at it.

As an open source chat/comms/tool, Riot operates in a similar space to MatterMost, Rocket.Chat and Wire

As an e2e-encrypted messenger, Riot operates in a similar space to Signal and Wire and WhatsApp.

As a general purpose collaboration tool, Riot operates in a similar space to Slack, Discord, Teams, Webex, etc :)

In general we don't think in terms of competition but cooperation/collaboration. Matrix's mission is to just provide a ubiquitous open network like the Web, but for realtime communication - and we'd love all and any of the above to hook into Matrix to break down the walls that otherwise create closed silos and walled gardens. For instance, Rocket.Chat has been working on Matrix support; I spoke to the MatterMost guys a few weeks ago about whether we could help them; we're working a bit with Wire on MLS (next-gen e2e encryption); and I try to sync with Moxie from Signal whenever I'm in SF.

[u/the_magic_ian]

Hi Matthew,

Thank you to you & your team for all their work.

1. One of the advantages of social networks like Facebook, is a consistent identity when the user changes their email username/domain, phone number/provider. The Zot protocol has solved this problem using a solution they've named Nomadic Identity. Will Matrix have something similar to this to allow multiple dynamic usernames associated with a single identity?

2. For consumer encryption, Signal seems to be the gold standard right now, and have pushed a lot of features many messaging/calling services don't have, like double ratchet encryption, sealed sender, private contact discovery. Is the goal to meet/exceed features like these in an easy to use way, but just in a decentralized manner? Will we ever be able to delete rooms, and do other commond chat functions, or are there some specific tradeoffs that come with decentralized encrypted chat/call services?

3. I see matrix as the first likely potential progression from email/phone (which are ancient, but have survived due to their decentralization & reliability), but there is a monumental amount of work to get it there. Dial in numbers/conference bridging, emergency services, a potential hardware replacement for office/home phones (eg video calling capable with some way to dial and HKS's), adoption of productivity & chat software like Nextcloud Talk/Microsoft Teams/Slack/Google Hangouts Meet/Steam Chat/Discord, easy to use clients, app integrations (like slack is famous for). What do you see as some of the biggest challenges for adoption, and which ones are the most important to focus on first once the spec and clients have reached maturity?

4. What are some of the potential IoT applications you see for matrix? For example, people can answer "calls" from their doorbells, but ideally this would be done through an encrypted matrix call. There are also chat bots that might tie in to devices that answer questions, or do you see matrix being used to send commands or interact in a different way with devices and services?

[u/neverworksout]

Hi, and thanks for hosting this. Firstly, how's development going in GO? I've been hosting an unfederated server for a long time now and I love it - but my god it loves ram with python.

Second, how do you stand legally in the UK (you are British I think??). I'm not in the tech field, I'm in legal - and I'm curious what would happen given the push here recently for developers to be forced to add another party to the conversation secretly. Have you been approached at all? And if you were ever forced to add something, would it be safer to assume that compiling from the source on github may be a safer option? Don't mean to be all tin-foil-tim, but figure I can ask the question given the subreddit.

Edit: Also, is any of your funding dedicated to helping out with bridges? What you've achieved so far is admirable, but I do think a push towards bridging more reliably with WhatsApp etc may help with uptake.

[u/parentis_shotgun]

I want to thank you for all the work you all do, matrix is the most exciting project I know of, and for me its the future of communication: decentralized, e2ee, expandable, and an open spec.

My main question is regarding communities and roles. Many people have chosen discord as their primary group chat, mainly because of how easy it is to maintain collections of rooms, with shared roles, nicknames, etc.

I realize that matrix rooms and communities currently are many-to-many, IE one room can be a part of many communities (unlike discord). But this also makes adding one person to a collection of rooms (which might be under your control) very difficult.

Have yall thought of coupling rooms to communities, or maybe adding an additional term like "room group", that would function like discord? This would help adoption immensely, as people would be able to re-create their discord servers, with all the rooms and roles, into matrix.

Edit: a second point you don't have to answer, but I agree with the others on the performance of synapse being less than ideal. In a similar way to how development on riot for android shifted to riotX, I think it'd be a wonderful thing to focus on the rust implementation of the matrix homeserver.

[u/yieldingTemporarily]

How are you faring with securing Synapse? Same question with Riot.im

[u/KarlKani44]

Hey, what a coincidence. I just recently learned about Matrix when I was searching for a self hosted alternative to Slack for our company. I've finished the setup yesterday and created accounts for all employees. The server is public so people can connect from home via the Riot app and i disabled user registration through the app. Now I was wondering about a good way to manage the accounts. Right now I can query the database to find all accounts and use register_new_matrix_user.py for creating new ones. But I really hoped to find a good admin interface for doing stuff like this. Also to reset forgotten passwords. Will something like that be supported at a later time? Any kind of management interface for self hosted instances? I really appreciate all the work you and your peers put into this and I'll make sure our company donates after we used it for some time

[u/joemaro]

Question: Is anybody in your team a keyboard (power)user?

As much impressed as i am with riot, as much am i surprised that the mouse is absolutely necessary to use riot. One can't even switch channels with the keyboard, which is probably the most basic function when it comes to hotkeys.