Notes from ProgNET London 2019

In the past three days I attended the ProgNET London 2019 conference.

Here my key notes from the various sessions and workshops I've attended:

  • The Pac-Man Rule at Conferences

  • Scaling Systems: Architectures that grow

    • How long the user wait until he consider your app down? That's what matter. Be pragmatic in your efforts to improve availability. If your availability requirements are met just stop!

    • Do the minimal amount of work on the critical path. Use speculative execution and/or queue work that can be complated later.

    • Optimize for read by implementing simple caching algorithms (use 80/20 rule). Start caching on the client before introducing caching server-side.

    • Distribute requests across multiple systems (scale out).

    • Always monitor your production system to verify your changes are impacting availability as you expect

  • Channels in .NET

    • Original paper: Communicating Sequential Process

    • Provides a way to implement efficient publish/subscribe scenarios between tasks

    • Can dispatch 20+ million messages in under a second without blocking threads.

  • Quantum Computing 101

  • Surviving Microservices

    • Microservices implementation has to be good, it won't be fast and will not be cheap

    • Business must support a microservices transition as it can cost million of dollars. All key players must analyze use cases, domain, sub-domains and resources to identify the microservices boundaries.

    • OpenAPI Generator

    • Build plenty of dashboards around your logs.

    • "If you don't write your logs, you won't have them"

    • Microservice consumers must provide test cases

    • Implement a disaster recovery protocol and test it

    • Start with 1 or 2 services and add more over time if needed

  • The Pipeline Driven organization

    • The enable real continuous integration and delivery you need to have the pipeline to make decisions (no more human approval gates)

    • The biggest barrier in adopting full automation are people needing to change their behaviors

    • Every time something happen that reduce the confidence in the pipeline, you change/update it

  • Distributed Cloud Native System

    • Istio is one of the most popular open source service mesh that works on top of Kubernetes

    • Helm is the package manager for Kubernetes

    • Kubernetes concepts: namespace, node, pod, labels, selectors, external volume, replica set, deployment

  • Elastic Leadership

    • Teams are in three stages: survival mode, learning mode, self-organizing mode

    • Leaders know what to do when they don't know what to do

    • Teams are in learning mode if everyone is a little bit annoyed, a bit out of their comfort zone

    • Two-Days Elastic Leadership workshop

  • Performance

    • 3 Key Elements: execution time, throughput, memory allocations

    • Performance expectations should be discussed before implementing features

    • Always measure and optimise

    • Find the right balance between performance and redability

    • Benchmark.NET is the framework of choise for performance benchmarking in .NET

    • Span<T> offer an unified view over a continuous block of memory (even unmanged)

    • Span<T> is a ref struct so can only be used on the stack.

    • Use Memory<T> if you need a span allocated on the heap

    • String.Create can improve performance when initializing large strings

    • ArrayPool can improve performance by reausing arrays in your application

    • Pipelines make it easier to do high performance IO in .NET and use ArrayPool under the covers

    • stackalloc in C# allows you to allocate memory on the stack

    • The new System.Text.Json namespace provide new high performance ways to work with Json. At the high level you can use JsonSerializer/Deserializer. Mid level you can use JsonDocument. Low level you can use UtfJsonReader/Writer.

    • How to sell performance optimization to your business: find a quick win, prototype an optimization, use the scientific approach, convert results into memory and determine the cost to benefit ratio.

    • Book: Pro .NET Benchmarking

    • Book: Pro .NET Memory

  • Event-Driven Systems

    • Read Martin Fowler definition of microservices

    • Smart endpoints and dumb pipes

    • Book: Building Microservices

    • A Microservice is effectively a way to encapsulate a system behind an API. It's a logical boundary. It's not an app running in a container. It might be made of multiple services inside.

    • If you can't deploy your microservice indipendently you are not getting the benefit of microservices

    • The "Join" Problem: Each API call from a microservice to an another decrease the overall availability of a microservice (if a microservice make 4 API calls to 99.95% available microservices, it's availability becomes 99.8%). The service mesh and event driven architectures help to solve this problem. It can enable the overal system to continue working in a degradated state even if one microservice goes down.

    • Event Carried State Transfer solve the Join problem by using local caches.

    • Cache Outside Data not Inside Data - Never export microservice internal data

    • SAGA Pattern to maintain data consistency across services

    • Consistency vs Availability is the trade-off

    • The Outbox Pattern

    • Transaction Log Tailing

  • malloc() in Linux always succeed. Linux kills a process randmonly when there is no memory.

  • If you use the cloud, make sure you have a strong support contract. You will need it!

  • Read How Complex Systems Fail

If you want to attend the conference next year, you find more info on ProgNET London 2020.

Sharing is Caring
>