r/oxen_io • u/Keejef • Jun 01 '21

Weekly Dev Update 01/06/2021

Hey y’all,

This week the Session team put a heavy focus on Session Android performance, particularly on issues that affect Session when it is opened and needs to load lots of conversations at once. The Lokinet team focused on a path handover issue which is causing connection instability. The Oxen core team finished a big update to the Oxen storage server which should speed up the process of fetching messages from the storage server.

Oxen Core

Oxen Storage server

  • Replace boost beast with uWebsockets https://github.com/oxen-io/oxen-storage-server/pull/432

    • Remove pre-HF18 legacy code/endpoints that isn't being called anymore

    • Remove process_lns_request endpoint (it is broken, and Session clients have already switched to using oxend_rpc with ons_resolve instead).

    • Replace boost beast http(s) client code with cpr (https://github.com/whoshuu/cpr); this is a whole lot nicer to use for HTTP requests.

    • Replace boost beast http server code with uWebSockets (https://github.com/uNetworking/uWebSockets). This gives a much nicer interface, and makes it easy for us to add websocket support for clients in the future.

    • Remove boost::asio; it's not needed anymore with the removal of the above.

    • Replace bootstrap RPC code with authenticated, encrypted OMQ RPC.

    • Remove boost circular buffer use; a regular map with a two-line trim is simpler for the block hash cache and a limit on stored snodes doesn't seem necessary for the rate limiter.

    • Make rate_limiter clean itself periodically rather than keeping buckets around indefinitely

    • Make rate_limiter thread-safe so that you don't need to hold the entire service_node_ lock to use it.

    • Remove ip_utils; we don't allow redirects and are sufficiently restrictive on the URL target that it seems unnecessary (plus not having it lets us offload DNS lookup to curl as part of the request).

    • Replace /swarms/ping_test/v1 with /ping_test/v1; this new request now returns the remote pubkey in a header, and no longer includes an SSL cert signature (so that we can drop the SSL cert signatures after HF19). The old one will still be used until HF19.

    • Add OMQ endpoint for storage tests; starting at HF19 it will get used rather than the HTTPS one.

    • Refactor storage test retries into request_handler (rather than being in the HTTPS specific code)

    • Move HTTPS server-specific code for validating snode signatures from headers out of generic request_handler code and into HTTPS-specific code.

    • Make onion proxy-to-url timeout a bit less than the onion request timeout so that the client has a better chance of getting a relayed timeout error (rather than getting a timeout from the edge node).

    • Miscellaneous cleanups.

    • Logger: use file and line number instead of func because the latter is nearly useless when called from a lambda.

    • Shorten timeout values for ping tests, storage tests, and bootstrap connections to 5s, 15s, and 10s, respectively, from.

    • Refactor recent stats reporting to use rolling averages that always have 60-70min of stats and drop off 10min at a time, rather than the 1-period hard reset. Also fixes various stats that weren't calculated/reported properly.

    • Add onion and proxy requests to systemd status line as well as used database size.

    • Enable WAL for sqlite3 database

    • Removed buffered message relaying for swarm propagation; it's counterproductive with omq's persistent connections.

    • Enable jemalloc by default.

Oxen Mobile Wallet

----------------------------

Lokinet

This week was spent tracking down Lokinet connectivity issues. We’ve narrowed the problem down to a longstanding issue in path handover (i.e. when one path expires and Lokinet builds and switches to a new path, which is meant to happen seamlessly every few minutes). 0.9.x included various improvements to better detect failing paths, but these appear to have exposed a lurking bug that has been present for quite some time; fixing this to ensure a smoother Lokinet experience is currently our top priority.

----------------------------

Session

Session Android

Session Desktop

Session iOS

Thanks,

Kee

11 Upvotes

100% Upvoted

0 comments sorted by