September 14

Notes from ProgNET London 2019


In the past three days I attended the ProgNET London 2019 conference.

Here my key notes from the various sessions and workshops I've attended:

  • The Pac-Man Rule at Conferences

  • Scaling Systems: Architectures that grow

    • How long the user wait until he consider your app down? That's what matter. Be pragmatic in your efforts to improve availability. If your availability requirements are met just stop!

    • Do the minimal amount of work on the critical path. Use speculative execution and/or queue work that can be complated later.

    • Optimize for read by implementing simple caching algorithms (use 80/20 rule). Start caching on the client before introducing caching server-side.

    • Distribute requests across multiple systems (scale out).

    • Always monitor your production system to verify your changes are impacting availability as you expect

  • Channels in .NET

    • Original paper: Communicating Sequential Process

    • Provides a way to implement efficient publish/subscribe scenarios between tasks

    • Can dispatch 20+ million messages in under a second without blocking threads.

  • Quantum Computing 101

  • Surviving Microservices

    • Microservices implementation has to be good, it won't be fast and will not be cheap

    • Business must support a microservices transition as it can cost million of dollars. All key players must analyze use cases, domain, sub-domains and resources to identify the microservices boundaries.

    • OpenAPI Generator

    • Build plenty of dashboards around your logs.

    • "If you don't write your logs, you won't have them"

    • Microservice consumers must provide test cases

    • Implement a disaster recovery protocol and test it

    • Start with 1 or 2 services and add more over time if needed

  • The Pipeline Driven organization

    • The enable real continuous integration and delivery you need to have the pipeline to make decisions (no more human approval gates)

    • The biggest barrier in adopting full automation are people needing to change their behaviors

    • Every time something happen that reduce the confidence in the pipeline, you change/update it

  • Distributed Cloud Native System

    • Istio is one of the most popular open source service mesh that works on top of Kubernetes

    • Helm is the package manager for Kubernetes

    • Kubernetes concepts: namespace, node, pod, labels, selectors, external volume, replica set, deployment

  • Elastic Leadership

    • Teams are in three stages: survival mode, learning mode, self-organizing mode

    • Leaders know what to do when they don't know what to do

    • Teams are in learning mode if everyone is a little bit annoyed, a bit out of their comfort zone

    • Two-Days Elastic Leadership workshop

  • Performance

    • 3 Key Elements: execution time, throughput, memory allocations

    • Performance expectations should be discussed before implementing features

    • Always measure and optimise

    • Find the right balance between performance and redability

    • Benchmark.NET is the framework of choise for performance benchmarking in .NET

    • Span<T> offer an unified view over a continuous block of memory (even unmanged)

    • Span<T> is a ref struct so can only be used on the stack.

    • Use Memory<T> if you need a span allocated on the heap

    • String.Create can improve performance when initializing large strings

    • ArrayPool can improve performance by reausing arrays in your application

    • Pipelines make it easier to do high performance IO in .NET and use ArrayPool under the covers

    • stackalloc in C# allows you to allocate memory on the stack

    • The new System.Text.Json namespace provide new high performance ways to work with Json. At the high level you can use JsonSerializer/Deserializer. Mid level you can use JsonDocument. Low level you can use UtfJsonReader/Writer.

    • How to sell performance optimization to your business: find a quick win, prototype an optimization, use the scientific approach, convert results into memory and determine the cost to benefit ratio.

    • Book: Pro .NET Benchmarking

    • Book: Pro .NET Memory

  • Event-Driven Systems

    • Read Martin Fowler definition of microservices

    • Smart endpoints and dumb pipes

    • Book: Building Microservices

    • A Microservice is effectively a way to encapsulate a system behind an API. It's a logical boundary. It's not an app running in a container. It might be made of multiple services inside.

    • If you can't deploy your microservice indipendently you are not getting the benefit of microservices

    • The "Join" Problem: Each API call from a microservice to an another decrease the overall availability of a microservice (if a microservice make 4 API calls to 99.95% available microservices, it's availability becomes 99.8%). The service mesh and event driven architectures help to solve this problem. It can enable the overal system to continue working in a degradated state even if one microservice goes down.

    • Event Carried State Transfer solve the Join problem by using local caches.

    • Cache Outside Data not Inside Data - Never export microservice internal data

    • SAGA Pattern to maintain data consistency across services

    • Consistency vs Availability is the trade-off

    • The Outbox Pattern

    • Transaction Log Tailing

  • malloc() in Linux always succeed. Linux kills a process randmonly when there is no memory.

  • If you use the cloud, make sure you have a strong support contract. You will need it!

  • Read How Complex Systems Fail

If you want to attend the conference next year, you find more info on ProgNET London 2020.



You may also like

The best way to stay up to date with C# 10 features

My Technical Journey from my first program to Lead Software Engineer

Top 10 C# Developer News from Microsoft Build 2020

Agile Estimation

{"email":"Email address invalid","url":"Website address invalid","required":"Required field missing"}

Subscribe to our newsletter now!

Get instant access to the Master C# 9 webinar to learn all the new exciting C# 9 features quickly.
Get regular videos and news on C# and .NET and offers on products and services to master C#.

We will collect, use and protect your data in accordance with our Privacy Policy.