GopherCon 2015 CFP
So my talk was not selected for GopherCon. I’ll be turning it into a series of posts on things that were unique, discussing building an immutable log in Go, going over my distributed pong game (examining the game network programming I did and how I broke the physics calculations among servers etc.) and a bit on the Company Retrospective. Here is the full CFP in all its glory.
Title
An introduction to distributed systems in Go
Are we there yet? Building highly scalable distributed systems in Go
Abstract (600 or less)
An introduction to building large scale distributed systems and how immutability and distributed logs are key concepts that can drive architecture decisions to increase confidence in a distributed system. Discussed are the evolution of a single system written in Python that was transitioned into three new systems written mostly in Go and redesigned around the key concept of immutability and distributed logs. The conclusion centers on applying the same principles to a smaller domain of a distributed pong game.
Details
Theme
The theme of the talk is to bring to the audience both an introduction to key concepts around building distributed systems and the hands on nitty gritty lessons learned from practical experience when those concepts were ignored and applied correctly.
The flow of the talk
“Here are some great principles” “Here is how we learned them the hard way” “Here is how we built something better” “Here is how you apply them to a different problem”. The key takeaways would be an understanding of how immutability and consensus work together to allow highly scalable distributed systems.
Outline
Introduction to Distributed Systems What makes a system distributed 8 Fallacies of distributed systems Introduction to the CAP theorem Techniques to handle consensus Paxos Raft 3 Phase commits Your database is lying to you ACID guarantee comparison Popular KV consistency comparison Understanding Consensus & Replication Introduction to immutability & distributed logs Literature Pros Cons How to scale with immutability What does it buy you? What does it require of you? What makes it different? What if you don’t do it? How does it affect system design. We enable accountants to share data in a way they understand Talk about our first system, distributed graph calculation on Google App Engine Hundreds of millions formula graph calculations / day Building a data update system on top of it, ~700 million updates / day Building a linking system on it to define the data flow Customer base has grown, reached the maximum throughput Created three new systems based on our experience Linking (Go) Data Sharing (Clojure) Calculation (Go) How we use immutability How we use a distributed log We’ve now entered billions possible per day Not just scale, resilient to failure Go retrospective - What did we really gain as a company? What price have we paid? Different developer backgrounds Different levels of experience (0 professional to decades) For our purpose What works well What doesn’t work well in practice? How does Go play well with the whole cycle for us Prototype Dev & QA Performance Dev Ops Production Applying these principles to a different domain - Game server in Go Distributed Pong Architecture overview Shards Clients Coordinator Consensus Sharded physics calculation for collision detection and motion 3 phase commits for cluster agreement Frame to calculate Sharing phase between shards as objects move between shards Immutability distributed log for horizontal streaming of updates to clients alternate time warp that writes new correction state compaction allows clients to join mid game and replay state Scaling the size of the pong arena and the number of players over the shards Using r-tree’s to break up zones for physics calculations to happen in shards.
Outcomes
The intended outcome is for me to be able to share practical experience in building highly scalable solutions in Go in a manner that imparts some wisdom and experience to attendees so they feel confident in exploring and promoting Go and CSP style programming in their own professional careers.
Intended Audience
Developers interested in building highly scalable solutions on Go. Individuals who are curious how transitioning to Go went in a semi large engineering effort. People who have heard of distributed systems but don’t necessarily understand the key challenges in building them.
Pitch
Why is this talk pertinent
Building distributed systems in large engineering environments and for high scale is a core value for the Go language. This talk gives an introduction into how those systems are built and how Go itself works with those principles to build resilient and scalable distributed computing solutions.
What is your involvement in the topic
I moved to the team that initially wrote the first distributed calc system in Python and wrote several enhancements on top of it. I then became the lead engineer that helped architect the new systems and write the new distributed calculation system. I also wrote the distributed pong game from scratch.
Explain why this talk should be considered and what makes you qualified to speak on the topic
I think this talk is very appropriate for GopherCon because it touches on good principles, practical experience with the language in a professional environment, and shares a fun focused application of those principles. I feel qualified to speak on this topic as I’ve been a key part of writing the old systems and the new ones and have been right there with championing these principles that are starting to return large dividends for us. Personally, I’ve spoken on Go at smaller than Gophercon venues (on Go itself & Machine Learning in Go) and have other public speaking experience so I feel qualified to speak in such a setting.
After spending $450, I was very disappointed with the lack of technical depth of several of the presentations at GopherCon 2015. Several people questioned why there were no presentations on distributed computing. It is rather sad that this presentation was passed over in favor of others.
I decided against going specifically because the talk line up would be of little use for people wanting to learn how to do deeply technical things in Go. I worked pretty hard to build a proof of concept for the talk to demonstrate distributed computing principles. Maybe I’ll do a video of it since I’ve been lacking in time to write out detailed blog posts for each section. Feel free to comment on which part you’d like to see discussed in more detail. Thanks for the comment Random Gopher.