17 Mar 2017

Weekly Review: March 17, 2017

A week that starts with brownies and board games with friends can't be bad - though I may have to lay off the baking for a while, it's taking a physical toll. In other hobbies, my P2P event stream project has hit an exploration phase, so results this week are hard to quantify. That's natural, there's a lot new here - I'll just try to summarise what I've found.


I decided on a simple format for the dust client/server protocol. It's basically JSON-RPC, but using bencode instead of JSON - this avoids an extra dependency, and makes binary data easy to encode. This protocol is now implemented in a bare bones client program called dust, which interacts with a dustd daemon; similar to how the docker program interacts with dockerd. This client/server design should make a user-interface easy to build in other languages.

I've also been planning for the peer discovery stage. One of the immediate problems I'll hit is working around NAT in home routers. There are several techniques for punching through, but it's a lot easier with UDP than TCP. Instead of implementing my own crappy streams on top of UDP, I've been investigating an alternative: the Stream Control Transmission Protocol (SCTP) - It's used for the data channel in WebRTC.

SCTP is a message-oriented protocol like UDP, but each message is assigned a stream and all data within each stream is delivered in order, like TCP - potentially a best of both worlds. It can also be encapsulated in UDP packets, which could be very useful for NAT hole-punching! I found friendly information on SCTP a little thin on the web, so I've picked up a copy of the book UNIX Network Programming: The Sockets Networking API, which includes several chapters on it.


After alluding to it last week, I've started my blog post on timestamps in distributed systems. I think it might spawn a second post on version numbers too - hopefully they'll be a useful reference to point people to.


I've been testing Dan Leslie's chicken-scheme package. It provides auto-complete and documentation support without a running Scheme REPL - I've found it useful when working on C integration modules, which need to be compiled.


A few links to keep you entertained:

Enjoy your weekend!