Herding Code 212: Steve Sanderson on Web Development in 2020

While at NDC Oslo, Jon talked Steve Sanderson about Web Development in 2020 – future JavaScript features, adventurous optimizations, and constraint-based styling.

This week we’ve also got a full transcript (thanks to Damian Guard!).

Download / Listen: Herding Code 212: Steve Sanderson on Web Development in 2020

Show Notes:

    • Hello
    • ES 2015 & 2016
      • (0:52) Jon asks if ES 2015 and 2016 is the new name for ES 6 and 7
      • Steve explains how the standard body are going with yearly updates
      • (1:25) Jon asks what some of the new features are
      • Steve talks about how it looks so different and how the import and class features help
    • Polyfils & transpilers
      • (2:14) Jon asks if he can use those features today
      • Steve explains the polyfills and transpiler options
      • (2:46) Steve talks about the Babel transpiler open source project and why people should write ES 2015 code today
    • What about Typescript?
      • (3:19) Jon asks if Typescript is still an option
      • Steve explains how Typescript is a transpiler and provides some similar features as well as strong typing
    • Web Components
      • (4:05) Steve talks about web components, how they’re not implemented in browsers today and the polymer.js project that can bridge that gap
      • Steve talks about scoping, the Shadow DOM and how components are encapsulated from the rest of the page
    • Async & await
      • (06:04) Steve talks about his interest in the upcoming async/await support and how it goes further than promises in order to clarify the flow of success and errors
      • Steve mentions that Babel can polyfill async/await support today
    • Generators
      • (7:30) Jon asks for an explanation of generators
      • Steve compares generators with the yield function of C# and explains how the function is marked with an asterisk to signal it is a generator
      • (8:08) Jon asks if this is the end of promises
      • Steve explains that generators are built on top of promises and how they can chain waiting for multiple ones sequentially
    • Emscripten
      • (8:53) Jon asks about Emscripten
      • Steve talks about the Emscripten bytecode format and how it can make Javascript faster as well as providing the foundation for new web languages
      • Steve talks about the LLVM bytecode converter and how it only really works with languages like C and C++ that don’t have a runtime like Swift, Rust, Go etc.
      • (12:06) Jon asks about the demo compiling SQLite
      • Seve talks about why SQLite works when others don’t.
    • Shared Array Buffer
      • (12:48) Jon asks about the Shared Array Buffer
      • Steve talks about how Javascript took an isolated approach to parallelism and how Shared Array Buffer allows the breaking of this boundary to enable some high-performance parallelism
      • (14:05) Jon mentions how the ASP.NET 5 team are doing similar things for performance
    • Grid Style Sheets
      • (15:00) Jon asks about Grid Style Sheets
      • Steve points out the "Grid" in Grid Style Sheets refers to the company behind it and not the approach which is a rule, constraint and solver layout system like Auto Layout in iOS
      • (17:14) Jon points out it’s Javascript and so people can use it today
      • Steve talks about where it might go and explains how variables are much different to typical variables
      • (18:34) Jon asks about the animations in the demo and Steve reveals it’s a CSS transition
      • Jon and Steve discuss how GSS and CSS might not mix well together
      • Steve warns GSS is really cutting-edge
      • Jon asks about debugging constraint rules and Steve agrees this is a challenge right now
    • Goodbye
      • (20:30) Jon says goodbye as Steve heads off for his plane.




Jon: Hello and welcome to Herding Code. I am sitting here at NDC speaking with Steven Sanderson or Steve Sanderson, sor ry.

Steve: Either’s fine with me.

Jon: Either’s fine, great. I caught the tail end of your talk yesterday. It was fascinating, you were talking about future Javascript features or browsers features.

Steve: That is right – I was trying to predict the future of client-side web application development. I don’t know how accurate my predictions will be but some of them are fairly safe bets like we’re going to start using ES 2015/2016 stuff pretty soon and then there’s a whole raft of other features coming towards us.


Jon: Ok. So that was actually the first time I’ve seen ES 2015 and 2016. I think I’ve heard in the past ES 6 and 7.

Steve: Yeah, it’s just a bit of new marketing really. The folks behind the standards there have basically committed to bringing out a new update every year. So we’ve got this annual cycle rather than up until know it’s been every five years or every ten years in one case. So given that it’s going to be annual it it sort of makes sense to give the name a year.

Jon: Right, yeah, definitely. Kind of like Visual Studio has done.

Steve: Exactly.


Jon: So what are some great features there in 2015/2016?

Steve: Yeah OK so ES 2015 was previously known as ES 6 has got to be the biggest set of changes that have come in Javascript since it was first introduced. If you look at something that’s written with ES 2015 syntax – if you’ve never seen that syntax before you would probably not even guess it was Javascript. It looks really different. So you’ve got import statements – which give you a clean alternative to having loads of script tags. You’ve got actual classes which are the same at runtime as regular Javascript prototypical inheritance but gives you a nice syntax for declaring that you can remember.

Jon: We’re finally taking advantage – class has been a reserved word in Javascript forever, right?

Steve: Oh, yeah, I think that might have been the case.


Jon: Can I start using those features? I know there’s some poly-fills and transpilers or whatever.

Steve: Yeah, I’d say if you want to start using ES 2015 today it’s really not as difficult as you might think. Assuming you want to target all browsers since, let’s say IE 9, then you can basically use almost all of the ES 2015 features. You just need to use a transpiler.

The one that everybody seems to be most into at the moment is called Babel (BAY-bel) or some people call it Babel (BAB-bel). I don’t know what the right pronunciation is.

Anyway it’s an open source project that will take your ES 2015 code and produce an ES 5 equivalent of it. It’s really fast, easy-to-setup and to be honest with you I can’t think of any good reason not to be doing that. If you’re not doing that you’re sort of just keeping yourself in the past really because this thing is a fully ratified standard now. We know this thing is definitely landing – all the browser makers are committed to bring this in. There isn’t really a good reason to not be using this today.


Jon: I’ve looked at using Typescript for that kind of class support in the past. Is there any kind of downside to continuing with that or are they similar?

Steve: No there’s certainly no downside to that. If you’re getting benefit out of Typescript then absolutely use it. Typescript in a way is a transpiler because it is taking some syntax that is a lot like ES 2015 – it just adds some additional features on top of that to do with strong typing. So if you’re getting some benefit out of that strong typing then yeah definitely use Typescript. On the other hand if you don’t want to use the strong typing then you’ll find that something like Babel will do the compilation, like an order of magnitude, faster.


Jon: So that’s kind of a no-brainer as far as a future to dig into. What else did you talk about that’s more forward looking?

Steve: I was also looking at web components which is sort of all tied up with ES 2015. It’s kind of a part of this package of things arriving at the moment but unlike that making that work across all browsers is not quite as guaranteed today. So the best known polyfill for web components is polymer.js but in terms of IE that’s only IE 11 onwards. But it’s a really good feature. Web components is a way of combining together a bunch of scripts and markup and styles to make a reusable package of UI features that you can put in your application and you can assign them to a custom element name so you can have something like sales-chart as an element and when that goes into your application your custom sales chart appears and it has all the behaviors your want. And it can be, to some extend, encapsulated from the rest of your page as well. So this is a feature called Shadow DOM and it’s a way of telling the browser that a certain element should be abstracted away as far the page that contains it is concerned. Code running outside it won’t see the elements inside. If, for example, you were using jQuery and you were using a selector and it happened to match something inside that component it won’t actually match it because it’s considered to be like a separate document. So you don’t get accidental leakages of behavior into your component or out of it – it just makes reuse easier.

Jon: And the same for stylesheets as well?

Steve: Yeah, that’s right. If you bring a stylesheet into your web component and it’s certainly going to apply inside that Shadow DOM root it won’t apply outside it.


Jon: And I believe you mentioned polymer.js is the polyfill for that?

Steve: Yeah, it’s the best known polyfill but Polymer is only the way of doing something like web components. If you’re going to use something like React or Knockout or Angular any of those kinds of things they’ve all got their own kind of concept of reusable components with custom elements so if you’re using any of those things you probably can pretty much do this anyway in a fairly cross-browser way.


Jon: Right. Ok. So then what other features did you cover?

Steve: I was interested in the upcoming await/async support. 

Jon: That’s always been kind of a challenge in Javascript.

Steve: It so is isn’t it. Yeah. So Javascript for the long time was plagued by this everyone’s got a slightly different way of doing callbacks thing which is quite troublesome and then it all got formalized around promises. Promises is a fantastically elegant concept that gives you a really clear way of working with callbacks and chaining them and combining them and so on. So promises are brilliant and they’re now a native browser feature but there’s still a lot of opportunity to get confused with promises because there’s so many way you can chain them together and not necessarily even understand how success and error values are flowing through that chain that you can still get things wrong by mistake. I find this happens all the time in my work. Not me, of course, but other programmers get very confused.

Jon: Right.


Steve: I might do sometimes. Whereas as any C# programmer knows with await and async keywords you can write your code as if it was synchronous.

Jon: It’s so obvious what’s going to happen.

Steve: You can’t get that wrong really. That’s part of the ES 2016 proposal and if you want to do that today you can either use Babel which will transpile async/await just directly or you can use the generators feature which is in ES 2015. Slightly different syntax but it gives you the ability to do something almost exactly the same.


Jon: Can you explain generators a bit?

Steve: A generator for a C# programmer is exactly like the yield keyword that came in with C# 2. So you can have a function that returns an infinite number of values. Like every time that you ask it for the next value it will give you the next thing and it will never stop doing that. That’s yield in C# and generators in ES 2015 do the same thing. So you can have a function and you put a little asterisk after the keyword function to say that it’s a generator and then inside there you can have yield, return, whatever. You can do that inside a while(true) loop or something like that and then the thing that calls it will receive each of these values. 


Jon: Do I just throw away promises now?

Steve: No, you still need to use promises. The way you can use this to make promises feel synchronous is by having some other that invokes your generator that gets back the sequence of values. Each time it gets back a value it will say "well what kind of thing is this? If it’s a promise then I’m going wait for that promise to complete before I ask you for the next value." And so, if you are yielding a series of promises in effect it’s like your promises are being called sequentially one after the other. Which is why your code ends up looking synchronous. It’s not very easy to understand verbally but when you see it written down it’s quite straight-forward and nice.


Jon: So then there was Emscripten?

Steve: Yeah, that’s right. So just yesterday or the day before depending on timezones there was an announcement from Brendan Eich that there’s going to be a new bytecode format coming to the web. So instead of Javascript just being expressed in plain text as it always has been it will be possible to describe your code in a binary format. The idea with that really is just to take all of the good work that’s been done with Emscripten which compiles native code to Javascript and Asm.js which is a special subset of Javascript that can be optimized very well and say "Okay, we don’t need to write out Asm.js as legal Javascript any more we can just write it out as binary code and that saves the browser a lot of work in terms of parsing." Asm.js is very fast at runtime but it still imposes a high parsing cost on the browser and that’s particularly noticeable on mobiles where people complain that say a large game might take multiple seconds to start during which the browser CPU is just 100% parsing so it would be nice to eliminate that cost altogether. Also, you get smaller use of network traffic because it’s very compact and also even though the semantics of web assembly are the same as asm.js initially they are open to adding additional web assembly only features in the future. So it might be possible to have new types of parallelism for example that’s only possible in web assembly. If they decide that it would be inappropriate to have it in Javascript it might still be appropriate for there to be a low-level feature that other languages can compile down to.

Jon: Ok.


Steve: So yeah, that’s all about performance really. And you can kind of do it today you know. We’ve got Emscripten today, you can compile pretty much arbitrary C code. You can get your asm.js – it will be very very fast in Firefox – the only browser that supports it today but other browser makers have committed to implementing that to. I know IE certainly has, I don’t know about all of them. I’m pretty sure they don’t really have a lot of choice these days.


Jon: Yeah.

Steve: That’s useful for a lot of things but one thing that you can’t really do very well with that today, but it’s kind of promised in the future, is compile some other language, that’s not C or C++, into Javascript. So theoretically you could do it because Emscripten takes LLVM bytecode and turns that into Javascript. So in theory anything that compiles to LLVM bytecode can be translated into Javascript but in practice it’s much more complicated than that. So if you take a Swift application for instance, and that uses the whole LLVM toolchain, it will produce LLVM bytecode and you might be able to get Emscripten to produce Javascript but it’s not going to run because you don’t have the Swift runtime there in Javascript. And it’s the same with Go and Rust and so-on and in certain cases people have managed to get little bits of experimental code working but really it’s not a realistic option today but in the future it’s quite likely with web assembly that they’ll be a strong reason for the makers of these other compilers to want to support web assembly.


Jon: Ok. I believe you had tons of great demos during your talk – this was one where you said it basically very difficult to get a demo working for it. Was it that one?

Steve: I gave a demo of compiling SQLLite – the embedded database – down into Javascript which is absolutely trivial. It works flawlessly and you need to do nothing clever at all. But SQLite is just written with C so it’s kind of the best case – a very easy thing to get working. The thing I struggled with was – yeah I did try and compile some Swift and some Go and some Rust and none of them would work. I checked all the mailing lists and nobody else can do it either.


Jon: Okay. Wow. So next there was shared array buffer.

Steve: Yeah. Shared array buffer. So one of the limitations that Emscripten faces today is that it can only use the features that are available to Javascript. Obviously. Because it’s running on a Javascript virtual machine. And that means that there are certain types of parallelism that you can’t do because Javascript has always worked hard to be single threaded. Now we’ve got web workers with Javascript but they are deliberately limited. With the web worker each of your workers is only allowed to access it’s own private memory space and when you wan’t to communicate between your application and those workers you have to post these serialized messages to say what you want them to do and they post a message back saying what they did. And that’s cool but they can’t really approximate the performance of native code that will spin up multiple threads that all just mutate the same memory simultaneously without any complicated message passing. So shared array buffer provides a way to pass a reference to a block of memory where you’re effectively saying "I opt in to having no control over timings here. I accept the fact that anybody can mutate my memory at any time with no guarantees." And when you opt into that for your shared array buffer then you can get much higher performance in certain cases.


Jon: It’s interesting we were talking to Damian Edwards yesterday about some of the server work that they’re doing optimizing serving ASP.NET 5 and that was something they ran into on the server side was this kind of shared buffer and passing and minimizing the transitions and all that kind of stuff.

Steve: It’s funny isn’t it. On one side of the industry we’ve got the move towards functional programming and no shared state and at the other end you’ve got this need for exploiting all the core of many core processors with as little amount of overhead as possible where you do need to mutate shared state. So we’re kind of straddling these two competing goals there and for certain applications like, for example, high performance games or something like that the shared state parallelism is clearly winning and other things like certain types of business applications functional programming seems to deliver much higher benefits than that.


Jon: Okay. I remember one of the other ones that I was actually really excited about because I’d looked at before and meant to play with and hadn’t yet was Grid Style Sheets.

Steve: Oh yeah, yeah. So that’s funny. The first thing to know about Grid Style Sheets is that it’s got nothing whatsoever to do with grids. It’s a company called TheGrid which, you know, is a cool name for a company but it’s a little confusing when you make a product called Grid Style Sheets. Anyway the idea with Grid Style Sheets is to kind of break free from all the layout limitations of regular CSS. Any web developer in the world has struggled with making things get vertically centered or shrink-wrapping to their content. You know, that kind of thing. It’s embarrassingly difficult for us, even when we consult on some of the hardest problems in the world, we can’t make something centered inside something else.


Jon: And it’s funny because you feel like I can describe the problem. I want this to be here and this to always be here but never bigger than the parent. Or whatever, right?

Steve: And that’s exactly what Grid Style Sheets does. It’s an example of a constraints-based layout engine and with a constraints layout engine you don’t necessarily care so much about hierarchy and padding and things like that you just make a rule that says something like "My popup menu needs to be left aligned with some other element and it needs to be vertically centered in a certain area and it needs to be big enough for it’s content" – that kind of thing. So it’s just a few constraints you give and then the constraints solver will choose the appropriate position for it. And it’s the same technique that’s used in some modern iOS applications. They’ve got a technology called Auto Layout and that allows iOS to have things like – you’ll see they’ve announced the Windows 8 style sidebar thing in iOS 9 I think. Well don’t Apple a good original idea that one. And the way applications can fit into arbitrary sized screens is by using technologies like Auto Layout where the developer just specifies a set of constraints and the operating system positions things. And Grid Style Sheets allows you to do precisely the same thing but on the web. It’s a cool, very cool technology. It’s a bit experimental at this stage, I don’t know of too many people really using it.

Jon: And it’s just Javascript right? So I mean I can start using it today?

Steve: Absolutely. I think that the people building that would aspire to it one day maybe even being a native feature. So you have your style type equals text/gss instead of css and then you have this alternate syntax for defining layout. Which is a heck of a lot more expressive and straight forward than regular CSS in certain cases. It’s also got some really cool powerful features that have got no equivalent in CSS. For example, well a simple example is that it has variables. Now of course if you’re using SASS or LESS you know about variables but these are cool variables. They don’t just have a specific value they have a value where you can define some rules. Like you can say this variable is positive, this variable is less than 250 but you don’t say what actual value it is. The constraints optimizer will choose the value in order to satisfy all the other constraints. Also, it supports if/else conditions so you can say something like "if this element is bigger than a certain size then I want this layout otherwise some other layout which is like media queries but, you know media queries are limited to querying just the window width or some other basic thing like that. Whereas with GSS you can query absolutely arbitrary things and it’s evaluated at run-time so if you mutate the DOM over time then it will update its layout accordingly.


Jon: One really nice feature I liked about it too is that it animated between. And so, I watched as you resized the window and not only would it apply the constraints but it would like really nicely smoothly move to it.

Steve: Yeah, so that’s not actually a natively GSS feature. I just put a CSS transition rule in. So like transition * 1 second or something.

Jon: Smart, I like that!

Steve: So in an application when everything to just instantly snap to the correct position you wouldn’t have that rule but it’s quite sweet as you say to see things so of fluidly slide into their correct position. Which is dead easy because GSS internally is using position: absolute or something for everything so all your animations just work.

Jon: Now, because of things like that it might be difficult to use some GSS and some traditional CSS together.

Steve: Yeah, I’m sure that’s probably true. Yeah. I think there’s a lot of reasons why if you are adopting this you should expect to be in for a slightly bumpy ride. You will definitely be one of the first adopters in the world if you pick this up today. You’ll be the one that’s actually figuring out what the best practices are, what stuff does and doesn’t work that well so yeah I wasn’t standing up there trying to advocate that everyone should take this on today but if you’ve got an interesting little side project and you want to try out some really cutting-edge possible future thing then yeah it’s a really fun thing to play with.


Jon: And one thing too I was all excited about it and kind of sold on it and you explained too that it can be difficult to debug. If something’s not working it’s like actually solving a constraint and you don’t get that clear understanding of what I did wrong.

Steve: I could imagine some future tooling that shows you visually how things have been anchored and laid out. But at the moment it’s a bit like if you give it not quite enough information it will just make some decision that might completely bizare like it chooses to position an element off the screen or something like that and it may not be obvious until you really think very carefully why it’s done that.


Jon: Great, well this is fascinating. I’m looking forward to watching the video again and digging into some of this stuff. I understand you’ve got to run off and catch a plane.

Steve: Yes, that is true.

Jon: So thank-you very much for your time and was great talking to you.

Steve: Cheers.

Herding Code 211: James Mickens on The State of Computer Security and Bitcoin and Thomas Jefferson and Internet of Terrible Things and Prawns and Oslo’s Terrible Secret

While at NDC, Jon talks to James Mickens about his terrifying computer security keynote presentation.

TLDR you are doomed.

Download / Listen: Herding Code 211: James Mickens on The State of Computer Security and Bitcoin and Thomas Jefferson and Internet of Terrible Things and Prawns Oslo’s Terrible Secret

Show Notes:

  • Hello
    • (00:24) All is lost. Listen to Danzig.
  • Stop putting your money in Bitcoin
    • (01:00) James says it’s not a good place to put all your money unless you want to be poor.
    • (01:42) Bitcoin is a commodity, not a currency. Investing in commodities is hard. It’s not a stable place to put money. When it’s going up, the smart people are taking money out.
  • The legal system – and specifically Thomas Jefferson – won’t help you
    • (03:56) Judges and lawyers are very far removed from modern technology. Analogies to older forms of communications doesn’t work.
    • (04:44) Emojis are not telegrams. They’re incredibly terrible, though.
    • (06:30) Thomas Jefferson wouldn’t help with issues relating to iPhones. If your phone broke, would you take it to a really old person for help? Because Thomas Jefferson is a really, really old person.
  • The Internet of Terror
    • (08:10) The internet of things is the new cool buzzword, but nobody wants to reboot their house.
    • (09:17) The security story for the internet of things world is cartoonish. When your refrigerator starts sending spam, James will laugh at you.
    • (11:52) The first several generations of IoT houses will be terrible. Things will get better on a relative scale, but will always be terrible on an absolute scale.
  • Node.js
    • (13:32) James knows he’s in danger for dissing Node.js, but he can’t help himself.
    • (15:08) We’ll see the rise of the JavaScript Full Stack Developer, which could be powerful, but it’s also what Sauron wants.
  • Oslo
    • (16:10) Oslo is a wonderful city. It never gets dark, which makes you feel safe but if James was a criminal he would be all over the extended working hours.
    • (16:54) Shrimps are prawns, and unlike American shrimp they have legs and heads.
    • (18:06) French fries cost approximately one mortgage payment.
    • (19:42) James speculates on whether Oslo harbors some terrible secret. The condos look like lairs for James Bond villains.
    • (21:11) Jon saw a building with a big sign that said Edderkoppen, which he had recently learned means The Spider. Perhaps that is Oslo’s terrible secret?

Show Links:

Herding Code 210: Ian Cooper on Microservices and the Brighter library

While at NDC Oslo, K Scott and Jon talked to Ian Cooper about Microservices and using the Brighter library for Command Dispatcher / Command Processor patterns.

Download / Listen: Herding Code 210: Ian Cooper on Microservices and the Brighter library

Show Notes:

  • Why Microservices?
    • (0:48) Ian explains that "micro" doesn’t imply number of lines of code but "A bounded concept/business capability within your Domain" (put forth by Martin Fowler and James Lewis)
    • (1:20) Ian talks about breaking the domain problem down further and further for simpler testing, better fault tolerance and incremental releases.
    • (2:20) "If you can’t QA everything you need to be able to monitor and respond to issues rapidly."
    • (2:42) Scott Allen asks if Devops is a driver for Microservices rather than physical deployment or team size.
    • (3:00) Ian talks about the scale limits of developers and teams and how component based architectures.
  • Have we been here before?
    • (4:00) Ian talks about how Microservices is the next generation of component-based architectures after DCOM, CORBA, SOA and the importance of understanding what worked, what didn’t and why.
    • (4:49) Asking a component for a cup of tea vs telling something how to make tea highlighting the difference between Microservices vs RPC. RPC was very coupled to behavior which lead to fragile APIs.
  • Finding the "micro" sweet spot
    • (5:50) Jon asks how you manage the complexity of orchestrating many smaller pieces.
    • (6:15) Ian advises against going too small – Nanoservices – where the overhead of a service overshadows the utility value of it.
    • (6:30) "It’s really hard to get a feel in a new domain of where those points are that you can slice effectively" – one solution is to start exploring the domain in a traditional monolithic way and to break the parts apart at the seems to get the tradeoff right.
  • Tooling and support
    • (7:06) Jon asks what a good way to manage these services including profiling and monitoring.
    • (7:20) Ian recommends some tools to help:
      • New Relic for introspective monitoring and diagnostics.
      • Logstash or Splunk for log analysis and the usefulness of adding a GUID to a request that flows through messages and logs to correlate the activity.
      • Zookeeper or Consul for service registration and discovery.
    • (8:28) Scott Allen and Ian talk about how Microservices take forward SOA principles such as autonomous services, not sharing types and stable interfaces.
    • (9:00) Scott Allen asks what options for communications between the services are and Ian compares HTTP, Sockets and message queues like RabbitMQ.
  • Ian’s Brighter .NET lightweight Microservices project
    • (10:20) Basic two parts of Brighter are:
      • Command Dispatcher/Processor – Maps a command to a processor with a pipeline where you can insert orthogonal operations like logging and monitoring
      • Task Queue – Allows some commands will be handled asynchronously elsewhere
    • Very simple for the developer – just write a command and a handler.
    • Easy to embed in your existing Windows service if using something like Topshelf.
    • Provides timeouts, retries and a circuit breaker inspired by Netflix’s Hystrix in a declarative manner.
    • (12:45) Scott Allen clarifies how easy it is to get two services talking to each other using this via RabbitMQ.
    • (13:00) Ian talks about future support for Azure Service Bus and the possibility of producing one for Redis – RabbitMQ and Amazon SQS are already supported.
    • (13:30) Scott Allen asks if this is used in production and Ian explains how Huddle started with using RX on the server and had difficulties managing subscriptions.
    • (14:20) Brighter is on ThoughtWorks Technology Radar and evangelized by ex-Huddlers at their new roles.
    • (14:40) Ian talks about the importance of good documentation and welcomes feedback on theirs.
    • (15:10) Ian mentions they facilitate hexagonal architecture.
    • Scott Allen asks if you can use in-process and Ian explains the subtleties
  • Wrap-up
    • (16:00) Ian’s time is sucked up by being a a new dad, congratulations!
    • (16:30) The one job of a parent is keeping children alive.
    • (16:40) Thank-you and goodbye.

Show Links:

These show notes were contributed by Damien Guard. Thanks!

Herding Code 209: Robert Friberg on In-memory Databases and OrigoDB

While at NDC Oslo, Jon talked to Robert Friberg about in-memory databases in general. They discussed OrigoDB, Robert’s open source in-memory database, as well as Redis and SQL Server Hekaton.

Download / Listen: Herding Code 209: Robert Friberg on In-memory Databases and OrigoDB

Show Notes:

  • Hello
    • (00:18) Jon asks Robert what his presentation was about. Robert describes his talk covering in-memory databases, comparing OrigoDB, Redis and Heketon.
  • OrigoDB and working with in-memory databases
    • (00:40) OrigoDB is Robert’s open source project. Jon asks why Robert decided to write his own in-memory database when there others available. Robert says that OrigoDB is unique, and that they’ve been working on it for a long time and there was nothing available when they started on it.
    • (01:18) Jon asks how it compares to relational databases. Robert says that when you move to in-memory, you’re no longer constrained by the need to structure your data in a way that can be easily stored on disk. You can take advantage of the random access nature of memory, and thing more graph oriented than stream oriented.
    • (02:07) Jon asks if the data is persisted and follows ACID principles. Robert says that the data is in memory in any form you like, and you log transactions when you make changes – writing the events to disk or a database.
    • (02:48) Jon asks how data is loaded when an application starts up. Robert says it loads the most recent snapshot and then replays all events that occurred after the snapshot. You can choose the serialization format – JSON, binary and Protocol Buffers (protobuf) are supported. Protocol Buffers fast, compact and interoperable.
    • (03:45) Jon asks what kinds of applications work best with OrigoDB. Robert describes the problem it solves: data access and databases are too slow, so we need to use caching to compensate for that. Traditional relational databases were a good fit when memory was scarce, but now your entire application’s data can fit in memory. Also, historically, relational databases reflected the entity model and allowing business users to run queries; now we’re mapping back and forth between models which don’t match. If you keep all the data in memory, everything’s in one place and the data access is incredibly fast. That allows you to do everything single-threaded, really quickly – on Robert’s laptop he can do 50,000 ACID transactions per second.
    • (07:35) Jon asks what the difference is between using OrigoDB and just using his own in-memory structures. Robert explains that’s how OrigoDB works – you use your own in-memory structures and do LINQ queries against them. OrigoDB adds in snapshotting, persistence, etc.
  • OrigoDB support for clustering
    • (08:22 ) Robert says that in addition to the embedded engine, they also have an out-of-process server product that supports clustering with replication, load balancing, and off-site backup.
    • (09:12) Jon asks how the clusters communicate. Robert says that it’s via TCP, inspired by SQL Server clustering.
    • (09:50) Jon asks what happens when the master goes offline. Robert says that there’s no automatic failover, but you can do manual failover using the web-based UI.
  • Other OrigoDB features: Web UI, cross-platform, cached queries
    • (10:29) Jon asks about the web UI. Robert says there’s a web-based UI that runs in the engine using Nancy. It supports some nice features, including ad-hoc queries using Razor syntax
    • (11:33) OrigoDB’s bindings for JavaScript and Protocol Buffers support cross-platform applications.
    • (12:35) You work with OrigoDB using commands and queries. You can also send in LINQ commands as text and they’ll be compiled, parameterized and cached.
  • How Much Memory?
    • (13:17) Jon asks how much memory it will take. Robert says over the past twenty years, the transactional workloads for the projects he’s worked on have all been under 200GB. You can offload your reporting data to a relational database if needed.
  • Business Model
    • (14:18) Jon asks if the server product is commercial software. Robert said they’ve tried the revenue model but haven’t had any sales. In the next release, they’re pivoting to everything free and open source and trying to build a support business.
  • Case Study
    • (15:15) Jon asks what kinds of projects he’s built with OrigoDB. Robert talks about a consulting job for a large healthcare company in Sweden. The customer was having really bad performance problems – each service would create business components, which would then create data components. Due to the business requirements, the data transactions were complex, and many were written using cursors. Robert said he traced some database use and found that a single transaction could make thousands of database round-trips. Robert did a proof of concept using simple C# collections in-memory and found they could do tens of thousands of transactions per second. Robert says that transactions in SQL Server require logging pages of data to disk, whereas logging an OrigoDB transaction is often just a few bytes since it’s just logging the command.
  • Snapshots
    • (18:18) Jon asks how many snapshots are maintained. Robert says he tries to avoid snapshots since they require a read lock. You can also use an immutable model (using multi-version cursor control).
    • (19:21) You can truncate events when you snapshot, but then you’re losing information. Robert and Jon discuss how this relates to event sourcing.
  • Other In-Memory Databases: SQL Server Hekaton, Redis, VoltDB
    • (20:02) Jon asks what Robert showed off in his talk. Robert says he normally does workshops that are a few days long, so squeezing everything into an hour is difficult. He does demonstrations showing OrigoDB, Redis and Hekaton, but his main message is that your application’s data probably fits in memory and memory is cheap.
    • (21:03) Jon asks about Hekaton. Robert explains how Hekaton works, pointing out that it supports a hybrid model in which only certain tables are in-memory. The advantage is that you can use your existing SQL Server tools, ecosystem and code.
    • (23:20) Robert mentions VoltDB, a startup that offers an in-memory, distributed relational database engine.
    • Redis is a key-value store. Most people use it as a cache, but the values in themselves are structures, so a value can be a hashtable, list, queue, sorted set, etc. There are predefined commands that kind of look like assembler.
    • (24:45) Jon says he remembers running into some objects that were difficult to serialize. Robert says that the default formatter for OrigoDB is the binary formatter, and you have to mark your objects as serializable. If you use Protocol Buffers require you to define a mapping.

Show Links:

Herding Code 208: Chris Klug on SOLID principles and migrating from Silverlight to Angular and TypeScript

While at NDC Oslo, K Scott and Jon talked to Chris Klug about pragmatically applying SOLID principles and on moving from Silverlight to HTML5.

Download / Listen: Herding Code 208: Chris Klug on SOLID principles and migrating from Silverlight to Angular and TypeScript

Show Notes:

  • Hello There
    • (00:18) Jon says hi to Chris Klug and mispronounces his name and feels bad about it but Chris is understanding and lets it go.
  • SOLID principles and Pragmatism
    • (00:27) Chris has been doing SOLID talks for a while, but this time he spoke about when to use SOLID principles and when not to. However, open-closed and Liskov substitution principle are both examples that might not fit into all scenarios. Chris says if you go full-SOLID you might never ship anything… and shipping is a good feature, too.
    • (01:28) Jon asks Chris if he gets some pushback. Chris says he does, from both sides.
    • (02:20) K Scott asks Chris about how he applies the Single Responsibility Principle.
  • Migrating from Silverlight to Angular and TypeScript
    • (03:35) K Scott asks Chris about the kind of projects he’s building lately. Chris mentions one project moving from Silverlight to Angular, another moving form WPF to Angular. He’s found that patterns he used in Silverlight development – MVVM, dependency injection, data services – translate really well to Angular. The only downside is moving from C# to JavaScript, so he’s using TypeScript a lot now. He recommends starting with JavaScript so you learn the platform, then move to TypeScript.
    • (05:44) K Scott asks Chris if he’d always use TypeScript. Chris says maybe not for simple projects, since TypeScript makes things a little more complicated due to the transpiler step. But the type safety’s really nice for any larger project.
    • (06:28) Jon says he thinks that Silverlight was a good bridge to HTML5 since it let you get started with things like video on the web long before HTML5 supported them. Chris kind of agrees, but points out that HTML5 video still doesn’t support DRM, and he still prefers XAML to HTML and CSS.
    • (08:22) Jon asks what Chris would recommend to developers who are starting on the transition from XAML to HTML5. Chris says just jump in and get started – probably using Angular and TypeScript. It’s really not that hard once you get started.
  • Kite surfing and being a rock star
    • (09:00) Jon asks Chris what he does for fun in his free time. Chris says he’s a kite boarder, which is a little tricky because he lives in Stockholm.
    • (09:42) Jon asks Chris about his picture from the NDC speakers page, and Chris tells the story.
    • (10:24) K Scott asks Chris about the best spots he’s been kite boarding, and Chris mentions a few in South Africa. 

Show Links:

Herding Code 207: Damian Edwards on ASP.NET 5, DNX everywhere, and experimental servers on Windows

While at NDC Oslo, K Scott and Jon talked to Damian about ASP.NET 5, running DNX cross-platform (including Raspberry Pi), the exploratory work he and the team are doing to make ASP.NET 5 run really fast, TagHelpers, and his favorite Redmond craft beers.

Download / Listen: Herding Code 207: Damian Edwards on ASP.NET 5, DNX everywhere, and experimental servers on Windows

Show Notes:

  • Hello
    • (00:18) K Scott asks about the general status of ASP.NET 5 and what Damian’s been up to. Damian mentions the work the team’s been up to and the two talks and two days of workshops he and Jon just completed.
  • ASP.NET 5 – portability and cross-platform
    • (01:16) K Scott asks about the advantages of running ASP.NET 5 on the Core CLR. Damian says the big advantages are portability (bundle the runtime with your app) and cross-platform – anything beyond that is secondary. It is lighter, more compact, etc., but that’s not the main goal.
    • (03:04) Jon talks about how he went through a depressing smackdown trying to talk managers into letting him upgrade corporate apps to ASP.NET 2.0 (when it first came out) and the "one framework per server" monster shut him down. Damian points out that you do lose some things in translations – there are some Windows abstractions and API’s that will only be available in the full CLR. So if you have a requirement for Windows-specific functionality (COM, directory services, etc.), you’ll need to run your application on the full .NET Framework.
    • (03:59) K Scott asks about the experience of bundling the runtime with your application. Damian describes how he demonstrated this during his presentation, bundling multiple runtimes (e.g. Linux, Mono, Mac, Windows) with an application.
    • (04:38) K Scott asks hosting on Linux. Damian describes Kestrel, the web host for ASP.NET 5 that runs on libuv (used by Node and other servers).
  • ASP.NET 5 HTTP Performance, Pipelining and HTTP 2
    • (05:50) K Scott asks how Damian expects ASP.NET 5 to perform relative to Node.js. Damian describes the performance testing testing he an his team have started looking at using the TechEmpower benchmarks. He lists several abstractions and implementations they’ve looked at, starting with the simplest benchmark that just tests plaintext response.
    • (10:15) Damian explains how pipelining works. K Scott asks about how HTTP 2 fits it. Damian talks about how both enable higher HTTP throughput.
    • (12:17) Jon asks if the useful the HTTP plaintext test is and Damian talks about this test focuses on efficiency in speaking HTTP, there are a lot of other tests and features they’ll need to look at as they move up the stack.
    • (12:55) K Scott asks why the ASP.NET 5 stack is nearly twice as fast as ASP.NET 4.x for plaintext responses. Damian talks about how legacy compatibility in ASP.NET 4.x extends all the way back to classic ASP, whereas ASP.NET 5 only pulls in specific features as needed by the application.
    • (14:44) Damian says they’ll also be looking at massive concurrency (important for realtime applications) and asynchronous background work.
    • (15:32) Jon incorrectly guesses that async background work will be important for pipeline scenarios.  Damian says that since pipelining only works over the same connection, so this is more relevant for HTTP 2.
  • Minimizing Kernel / User Mode Transitions
    • (16:56) Jon asks about the importance of switching between kernel and user mode. Damian explains why that transition is necessary and the performance impact it has. He talks about how some of the memory management techniques they’re looking at.
    • (21:12) Jon asks about some newer APIs Damian had previously mentioned to him. Damian talks about Registered IO (RIO) in Windows 8 and Windows Server 2012 R2. K Scott asks about HTTP.SYS kernel mode caching.
    • (24:07) Jon asks if there are some performance impacts he should pay attention to as a web developer. Damian talks about the process ASP.NET MVC pipeline and how content could be double or triple buffered in some cases as it moved through the pipeline. That’s an issue when you’re looking at client-side optimization, because bytes aren’t flushed to the client as early as possible. In ASP.NET MVC 6, it’s now possible to explicitly flush content, and the double and triple buffering has been removed.
    • (27:14) K Scott asks about the baseline KB allocated per request. Damian said that previously it was 15-40 KB per request before your application code did anything; now it’s under a KB. There are some other buffers they can probably pool as well.
  • TagHelpers
    • (28:22) Jon asks TagHelpers. Damian explains some of the problems with HTML Helpers, especially when you need to control the HTML that’s being output. TagHelpers allow you to call HTML Helpers using an HTML-like syntax (tags and attributes) so you get all of the benefits of working in the HTML editor.
  • What do you do for fun?
    • (30:33) K Scott asks Damian what he does for fun. Damian talks about craft beers in Redmond and his new espresso machine. K Scott asks Damian about some of his favorite craft breweries; Damian talks about Kilty MacSporran and Hogus Maximus from Postdoc Brewing and Black Raven Brewing Co.  

Show Links: