Category: Uncategorized

How to onboard new employees

Most people seem to focus on the stages of the recruitment process before the employee starts. This is understandable given how difficult it is to source candidates, and then run the interview process.

However, it’s just as important to consider how new employees are onboarded into the new role. The first few weeks and months are critical to set people up for success, and determining if you actually made the right choice in hiring. There’s a reason most employment contracts have a probation period – this is the time to discover if your hiring process worked.

What does success look like?

The biggest mistake I’ve seen is that the job specification only describes the characteristics of someone who you would like to do the job, but no criteria for checking if that person is succeeding or failing. It should include a detailed description of what you expect the person to be doing and how you measure that output.

Indeed, how do any of your team know they are achieving what is needed for the business? There may not be quite as direct a correlation as with sales quotas, but there should be some objective criteria for every role which determines whether that person is performing or not.

This is idea behind OKRs – having objective measures which are regularly reviewed. We didn’t actually manage to make them work at Server Density but we still had metrics that we reviewed on a weekly basis to ensure that we were tracking towards our overall objectives, which were then discussed at the monthly board meetings.

For SaaS businesses, the key metrics are well understood. MRR, churn (net MRR churn and customer churn) and NPS are typically the three key numbers relevant for all areas of the business. Then each team might get something more granular e.g. cashflow for finance, funnel metrics for sales, critical out of hours alerts for ops, ticket satisfaction for support, CPA/payback rate for marketing, etc.

At Server Density, our new engineers had a goal of deploying to production by the end of the first day, although our expectation was that it would be a good 3-6 months before they were fully up to speed to work independently on the codebase. These short term vs long term goals really help set expectations.

Operational onboarding

This is where most onboarding starts and ends. It consists of setting up the mechanics of how the employee will actually get work done: laptop, email, chat tools, etc. This is obviously important, but should really be the easiest aspect of onboarding, and the one that is the most optimised.

That means all the crucial accounts, applications and systems are already set up and the computer the employee requested is ready to go. So much time is wasted with initial setup of laptops, creating email accounts, etc when this can easily be automated and be completed in advance of the first day.

Managerial onboarding

Everyone has a manager. This is someone who will help you get your job done, remove obstacles, act as a career mentor and hopefully offer proper feedback in one to ones. Getting off to the right start with your manager is often daunting – you don’t know how they work, what they’re like and how best to get what you need from them.

I like the idea of a Managerial Manual which describes exactly this. My favourite example is How to Rands, something I’ve seen implemented by several managers at my company, StackPath, as well.

Cultural onboarding

As the team grows, the culture will diffuses from “how the founders do things”, so the values and mission of the business need to become more formalised. This was something I wasn’t really aware of when I started Server Density – we were very informal about explaining how we did things and only wrote down our approach when we faced a problem with how something was being done.

The Netflix culture is probably the most famous and it is worth reading about what they mean by the term, but every founder and every company has a different approach to things. If I was starting a new company today, one of the first things I’d do would be to write out the mission and core values I wanted to build out. Values are what you hire against and describe how you operate. The mission is a short statement describing why the business exists through what it aims to achieve long term.

You also have to be careful of “culture fit” meaning “same as me”:

Finding the right people is also not a matter of “culture fit.” What most people really mean when they say someone is a good fit culturally is that he or she is someone they’d like to have a beer with. But people with all sorts of personalities can be great at the job you need done. This misguided hiring strategy can also contribute to a company’s lack of diversity, since very often the people we enjoy hanging out with have backgrounds much like our own.

How to hire, HBR

But you do need to find people who both understand and believe in the mission as well as agree that the values are the right way to achieve it.

Onboarding gimmicks

In addition to considering all the above, there are some impressive but gimmicky things you can do to make new employees feel welcome. I use the word “gimmick” in the sense that these are good ideas to impress someone on their first day, but aren’t really part of “onboarding” in the same way as the more important things above.

  • Swag – have a parcel of tshirts, hoodies, socks, stickers, etc ready and waiting for them on their desk. All correctly sized, of course.
  • High quality printed version of the company culture deck/values/mission.
  • Handwritten welcome note from their manager / the CEO.
  • Lunch / dinner with the management team / CEO.
  • Announcement introducing the new employee to the whole company.

Isn’t this a lot of work?

Yes. That’s the point. You can’t achieve anything without a great team, and it is absolutely necessary to spend a significant amount of time on the full end to end process: hiring, onboarding and then ongoing management.

To quote a good First Round Review article on this:

You say you’re too busy. You work at a tiny startup after all. But that’s no excuse, Guthrie says. “You’re not too busy. You just spent all this time, energy and money getting this new person to join. Blowing it is going to cost you even more when you have to start the hiring process again from scratch. Don’t make someone feel like you’re too busy to make them feel good about choosing to work with your company.”

Your goal should always be to make people believe on a gut level that your organization is so amazing that they couldn’t possibly work anywhere else. Building that attitude starts immediately once an offer letter is signed. And if you do it right, when the phone rings and it’s a recruiter on the end of the line offering the next big thing, they’ll say, “Sorry, I’m happy where I am.”

Questions for when things aren’t working

There are many times when you have a goal, you try something to achieve that goal, but it doesn’t seem to work. Whether it is in trying to grow a startup, writing a piece of code or in a more personal area of life, this will be a regular occurrence. It’s often called iteration and improvement, experimentation or trial and error.

But how far does that approach go? When does the trial become an error? What do you do when things aren’t working?

1. Define what “working” looks like

This should be obvious but unless you know what the end result should look like, how can you evaluate whether whatever it is you’re trying is “working”? The best way to do this is with a single number you can measure easily. That is not always possible but the closest you can get to a quantitive metric, the better.

2. Set a timeline

You can’t continue indefinitely. You need to see progress within a reasonable period of time. You already defined what the end goal looks like, but you also need a measure of “progress” towards that goal.

This has to have a time frame because how do you know whether to keep going or stop? Knowing when to stop something is just as important as deciding when to start something.

Are you almost there or do things just keep creeping forward? Is there an illusion of progress if you look at things in isolation, but when you zoom out on the timeline then you really haven’t come far at all?

3. Get a second opinion

It’s difficult to be objective when you are deep in something day to day. You have the full context to make micro, operational decisions but you also need someone who can consider things more independently. In business, this is usually the role played by the board. Talking to close friends is similar in a personal situation.

You need people who will listen and understand, but also who are not afraid to give their true opinion.

Sometimes you can make a decision by yourself but asking a few, experienced people what they think gives you additional data points to align with. You might decide to discount their opinion this time round but each time you come back with the same result, your threshold for dismissing their opinion should increase.

4. Repeat, but not too many times

Whether or not it was Einstein who said it, the old quote applies:

the definition of insanity is trying the same thing over and over and expecting a different result

Going back and tweaking variables, making changes, taking different approaches. These are all normal responses when things aren’t working. This is how debugging works! But there is a limit to how many things you can try. Ask yourself:

  • How is it different this time from the last time I tried it?
  • Is it actually different, or just a minor tweak on something we already tried?
  • Do I need to make a minor tweak or should we have a major rethink?
  • If I am trying the same thing again, why would it be different this time?
  • What needs to change in my approach to get an order of magnitude difference in result?

5. What are the consequences of making the decision?

Jeff Bezos has an approach to decision-making based on whether the result is reversible or not.

Some decisions are consequential and irreversible or nearly irreversible — one-way doors — and these decisions must be made methodically, carefully, slowly, with great deliberation and consultation. If you walk through and don’t like what you see on the other side, you can’t get back to where you were before. We can call these Type 1 decisions.

But most decisions aren’t like that — they are changeable, reversible — they’re two-way doors. If you’ve made a sub-optimal Type 2 decision, you don’t have to live with the consequences for that long. You can reopen the door and go back through. Type 2 decisions can and should be made quickly by high judgment individuals or small groups. 

Jeff Bezos, Letter to shareholders

Is it usually easier to take the path of least resistance and just “keep going”. Repeating with minor tweaks appears like it should be a type 2 decision, and it probably is for a while.

However, the cost of repeating compounds over time. Each time you try one thing, you’re not trying something else. This is the opportunity cost, and a type 2 decision can quickly morph into a type 1 decision.

6. What is the opportunity cost?

The expression in your while loop above needs to be determined by your opportunity cost. What would you otherwise be doing with your time/money/team/life if you weren’t doing the above?

Everyone has a different understanding of their opportunity cost but the cost is never zero. Nobody has unlimited time or money. And it’s not just your own opportunity cost you might be wasting – what about everyone else who is involved? What do you care about? What do they care about? Would you time be better spent pursuing those goals in other ways than what you are doing right now?

I think opportunity cost is the one most people under appreciate. It has the highest impact yet is the most difficult to see, because it only becomes obvious over a long period of time.

Startup holiday policies – unlimited time off doesn’t work

As an early startup founder, you have unlimited flexibility to work when and where you want, and for as long as you want. It only makes sense to extend that flexibility to the whole team, right? Indeed, that’s where the idea of “unlimited” holiday came about. I adopted this as our holiday policy when Server Density was founded in 2009.

However, it didn’t really work out as intended. The problem is that there is always something else you should be doing. It’s never a good time to go away and you certainly can’t disconnect completely. I travelled quite extensively in 2010 both to the US for the company and my first ever trip to Japan on holiday. But I didn’t ever have a true “holiday” in that I could completely disconnect from work.

If the founder and CEO can’t take real time off, how do you expect the rest of the team to be able to interpret the “unlimited holiday policy”?

Over 9 years at Server Density, we tried several different approaches to holiday policy. I’ve previously written about startup financial incentives so this post is about the different holiday policies we tried:

Unlimited holiday policy

With this policy, there is no quota and anyone can take as much time off as they like, subject to manager approval. The company does not count the days off each person takes, although it is of course still recorded so everyone knows if someone is off.

Pros

  • It sounds good, especially to those new to startups. Everyone likes the idea of “unlimited” because it means they don’t need to worry about whether taking a day from their allowance is a good use of time, and won’t miss out on events because they’ve used all their allowance.
  • You can use it to make the most of travel opportunities. The further away you go from home, the longer it takes to get into the timezone and so the more time you want to spend in a destination. I know 2 weeks is the minimum amount of time I would want to take for a trip to Japan.

Cons

  • There is no guidance about how much is “reasonable”. You will probably not hire someone who will take advantage and disappear for 3 months, but what is the right amount? Expectations are ambiguous and if the founders/senior team are not taking much time off, the rest of the company may feel pressure to avoid holiday.
  • Without guidance, some people will take off more time than others. Perhaps those will fewer deadlines or less pressure will find it easier to take holidays. Managers will be reluctant to approve holiday, especially longer trips. And when holiday is taken there will be an unspoken pressure to “check in”, with email or Slack.
  • If you don’t actually track time off, you can’t tell if someone is abusing the system or let other people know people are off. So you do have to track it, which means it is not “unlimited holiday” but “no measured allowances”.
  • The startup marketing doesn’t match the reality, which hurts your long term reputation. If a new candidate asks your team (or reads a Glassdoor review), will they be enthusiastic about the reality of working there?

Bonuses to encourage people to take time off

With the realisation that the pure “unlimited holiday” policy wasn’t necessarily working out for everyone, we decided to introduce an incentive to take time off. This amounted to a £1000 cash bonus at the end of the year if you took at least 25 days in the calendar year. This was originally a 50% salary bonus in the final month of the year but we adjusted that to a fixed amount because it became expensive.

Pros

  • Using a financial incentive to encourage behaviour is a well understood approach. It is used in all sorts of areas of life, from sales commissions through to encouraging people to give up smoking.
  • It makes it clear what the holiday policy expectations are. The actual number of 25 days could be changed but it sets a clear guideline for how many days you are encouraged to take.

Cons

  • It assumes everyone is incentivised by money.
  • How do you set the amount? £1000 means different things to different people but a percentage of salary is also somewhat arbitrary and can prove expensive.
  • How do you deal with edge cases e.g. someone who takes 24 days? Is it fair that someone who takes 25 days gets the bonus whereas the person who just misses it does not?
  • If someone suddenly realises towards the end of the year that they need to take time off, what happens if it is not a good time or lots of people are in a similar position? You could see lots of holiday requests right at the end of the year.
  • How do you deal with people who join the company part of the way through the year? Pro-rate the 25 days? Pro-rate the bonus?
  • It introduces a large cash outflow in the final month of the year. As your team grows, this could become a problem for cashflow.

Setting an expected minimum with active encouragement

After trying the above, this is the approach we were using before Server Density was acquired. We had an unlimited holiday policy, tracked all time off but set an expectation that everyone would take at least the UK statutory minimum – 28 days. Anything over that was generally allowed, subject to sufficient notice, availability of any cover and in consideration of occasional deadlines. In practice, we rarely denied requests for time off and most people were in the 30-35-days range each year. We also removed the bonus.

Pros

  • The expectation is clear and it follows the legal requirements. The UK fits nicely in-between the crazily low allowances of the US and the quite-high expectations common in Europe, so it feels like a reasonable middle-ground.
  • It is a good benefit to talk about when recruiting because it’s still unusual to have “unlimited” holiday in the UK, outside of tech startups.
  • It is down to managers to ensure their team take the minimum time off, spread throughout the year. This removes some of the responsibility from the team member and makes the manager think about any pressure they might be applying to work under unreasonable pressure.

Cons

  • It is still “unlimited” and so the value of having a day off is less than if each person had an annual allowance. To me, this means taking paid holiday doesn’t have the same feeling of value. If you can have as many days off as you like, you value them less. Using something like time off in lieu therefore has no effect and individuals may appreciate the holiday time less.
  • There is an upper limit. As a founder, you have to set the example and what if you want to go on a 3 week holiday? That’s almost all your allowance gone.

Conclusion – how would I do it today?

Unlimited holiday by itself simply doesn’t work. Many others have found this, too.

Introducing financial incentives also doesn’t work. The differences are marginal and it can end up being expensive for the company.

However, the motivations for exploring these kinds of policies are still essentially valid. You want employees to feel trusted to balance the needs of the business and their desire for time off, give them the flexibility to enjoy life and make expectations clear so that nobody feels unnecessary pressure to work so much they burn out.

If I was creating a new company holiday policy today, I would go with fixed allowances. This would probably start at the UK statutory minimum during the employee’s first year and then gradually increase the longer they have been with the business. This is similar to how Basecamp run things – 3 weeks plus a few extra days here and there if needed.

I would also be flexible e.g. if someone had used all their days off but wanted a day or two more for an important event, it seems unnecessarily mean to deny the request. I would also consider allowing some unused time off to be rolled over into the next year. This would encourage employees to think about time off as something of value, but also that the company encouraged people to use it.

Ultimately this is about balancing the requirements of the business and the needs of the team, which should align almost exactly most of the time. If employees feel rested and relaxed, and feel like they don’t need to strictly count time off, the business benefits as well as the employee. The advantage of early-stage startups is that you can experiment with these things.

My favourite books of 2018

In December 2017, I decided to set myself a goal of reading 1 book every week, reviewing each one once I finished it. I’m pleased that in 2018 I managed to read 57 (with an average length of 306 pages).

Looking back on the 9 years of running my own company, one of the three lessons I write about after the acquisition was that I would have liked to read more books. Although I can’t possibly remember all the details, over the course of the last year I have felt my thinking being influenced by what I have read even if I don’t remember specifics.

Taking the time to rate and review each title has also been useful. These are all tracked on my Goodreads profile along with reviews for each non-fiction title (I decided to stop writing reviews of fiction because they are so subjective, although I still rate them). Being able to see all those books on a single infographic style page is a nice feature and I often use it as a reference to return to cite something.

My “to read” list has stubbornly remained at around 100 books because I keep adding titles that get cited in other things I’m reading! I’m aiming to keep up the same pace so at that rate, I have at least another 2 years of reading to go!

So to end 2018, here are the 12 books I rated 5 stars:

Fiction

I particularly enjoyed historical fiction in 2018, and Robert Harris is the only author I read to achieve 5 stars in 2018:

I would also recommend Fingersmith by Sarah Waters and The Moon is a Harsh Mistress by Robert A. Heinlein, both of which I rated 4 stars.

Non-Fiction

Silent Spring, Rachel Carson

This book was published in the 1960s and represents the situation at the time, so things have certainly moved on since then, but it is amazing to read about the ignorance and willful disregard for evidence resulting in the extreme damage that humans have done to the environment.

There is no such thing as an entirely pure free market economy. Where negative externalities come into play, such as with the environment, regulation is a necessary component. The degree of carelessness and blindness to any evidence highlighted in this book shows why there are certain areas which always need effective government oversight. 

Whilst there is sometimes too much of a play on emotion, this is a great example of how science can be communicated effectively to a general audience, not only to help teach but also to push a policy objective. It’s no wonder that this book kick-started the environmentalist movement and helped policy-makers truly understand what their role should be.

The Great Degeneration, Niall Ferguson

Written in 2012, it’s interesting to see how the degeneration model has played out over the following ~5 years. This has come to a head with Brexit, and the election of Trump. 

China continues to grow and the idea that it will adopt a Western approach has basically been discarded. But can it maintain its approach without the institutions and structures that made the West so successful? Perhaps it is developing its own model.

I am increasingly of the view that a proper study of history is the most important thing for anyone who is interested in politics or business, and wants to avoid the mistakes of the past. The specifics certainly aren’t predicted by Ferguson, but the direction is. All it requires is knowledge and understanding of the key events of history. 

This book isn’t just about historical events, it is about the importance and relevance of today, and to the future. Applied history.

Essays In Love, Alain de Botton

Anyone familiar with The School of Life will notice the beginning of many common themes. A clever merger of romantic philosophy, relationship advice and a love story, even more impressive knowing it was written when de Botton was only 21.

Universal Healthcare without the NHS: Towards a Patient-Centred Health System

It’s clear that the current approach to funding the NHS is not going to work in the long term. The UK tax burden is already the highest it has ever been, significantly higher than global averages. With changing age demographics, funding for health is only going to become even more of a problem. That doesn’t even consider social care. How we go about addressing this at the same time as maintaining the crucial principle of universal healthcare is the subject of this book.

The instant reaction to any discussion of changes to NHS funding is to cry about the extreme inequity of the US system. Yet most healthcare systems, especially those in similar-income countries globally, and within Europe, use some form of social health insurance. These are much better comparators as likely alternatives than the US. The outcome measures tell you everything you need to know about those systems – the NHS ranks near the bottom and whilst it has improved significantly over time, the current snapshot of performance today is still poor, relative to alternative systems.

This is a discussion that needs to be had. Unfortunately, given the inability to have any form rational debate about the NHS within the UK, it will take a brave politician to set about implementing the reforms the system badly needs. This book does a good job at describing how we got to where we are now, how we compare to alternative systems in Europe and what changes might be workable given the religious nature of anything to do with the NHS. It serves as a good basis for starting that discussion.

The Road to Serfdom, Friedrich Hayek

In a very concise and logical manner and by using real examples from the time it was written (1940-43), Hayek convincingly and systematically dismantles the idea of socialism as a realistic form of government. Walking through the key concepts of individual freedom, planning, the rule of law, security, democracy and truth, Hayek demonstrates that socialism inevitably leads to totalitarianism at the hands of the few.

Hayek provides an important understanding of his beliefs about the role of government and what the idea of traditional 19th Century liberalism means (liberty and freedom), which is quite distinct from liberalism as the views of a progressive political party. In writing before the establishment of the welfare state in the Britain, his definition of socialism is in relation to the control of the economy. Most of his examples are from Nazi Germany and the 20-30 years before, whereas for many readers today, examples of socialism which come to mind are more likely to be post-WW2 East Germany and Soviet Russia. He assumes the historical context is known which means it might be difficult for the modern reader to relate, and to apply his theory to how the state should provide social security, healthcare and market regulation today.

Although I started the book already believing that socialism doesn’t work, Hayek’s style and straightforward logic has helped me significantly develop my thinking of the topic. It offers a level of clarity that is so often missing from contemporary debates. There are so many parallels with current events that you can easily replace the historical examples with more modern instances.

The final chapter discusses how an international federal system might help to prevent further war and how it could be structured to avoid socialist control. I found the parallels with the early to mid stages of the EU project fascinating. I feel the historical context of why the EU was created was missing from Brexit the referendum debate, and Hayek only serves to highlight concern about might happen following any breakup of the EU. Parallels with the WTO are also interesting.

I will now be looking for both essays which can help to add detail to the arguments Hayek makes, but also look to discover dissenting opinions and counters to his arguments. 

Capitalism and Freedom, Milton Friedman

Friedman is not quite as easy to read as Hayek. Whereas I found I could hardly put The Road to Serfdom down it was so compelling, Friedman was more of a challenging read. He gets very technical but this is because he uses real world examples of how to apply his philosophy to the current state of things (as it was in early-1960s America, which is mostly still relevant today).

I like to understand how the philosophy I read can be applied to real world situations and so I enjoyed how Friedman combines both the theoretical and practical. He doesn’t waste time repeating things unnecessarily and his logic is often very straightforward and comes to a rapid conclusion. For example, he dismisses Marx in just a few sentences, entirely effectively as well!

Friedman’s thoughts on inheritance particularly stood out for me as it instantly changed my opinion on the merits of inherited wealth vs “working hard/earned wealth”:

It is widely argued that it is essential to distinguish between inequality in personal endowments and in property, and between inequalities arising from inherited wealth and from acquired wealth. Inequality resulting from differences in personal capacities, or from differences in wealth accumulated by the individual in question, are considered appropriate, or at least not so clearly inappropriate as differences resulting from inherited wealth. This distinction is untenable. Is there any greater ethical justification for the high returns to the individual who inherits from his parents a peculiar voice for which there is a great demand than for the high returns to the individual who inherits property?

Regardless of your political viewpoint, this is a key text to understand the capitalist ideal. Friedman has chapters on taxation, education, discrimination, licensing, social welfare and others. In all he provides solutions that he thinks will much better tackle the problems at hand. I would particularly like to implement his flat rate negative income tax and simplify the entire system of taxation. This book should be required reading for anyone in government, politics and probably even society in general.

Poverty Safari, Darren McGarvey

There are 2 parts to Poverty Safari. The first is a series of auto-biographical stories which provide the sad backstory for Darren McGarvey’s upbringing and experiences growing up and living in poverty in Glasgow. 

It is a brave thing to do to recount such personal tales. Whilst it certainly helps to share these experiences so others can try and understand how many people live, I feel their real purpose is to provide legitimacy and authenticity to what I feel is the main, and second part of the book: McGarvey’s commentary and analysis of the current state of poverty politics.

This is important because McGarvey’s assessment is no doubt controversial and without understanding his background then it would be right to ask how he is qualified to judge the current state of political discussions. Of course, the stories themselves are important to help the reader understand what life is like struggling with poverty but I get the feeling that society is becoming somewhat immune and numb to such otherwise emotive events. The specific, harrowing details might surprise the reader, but the fact that this happens in general probably does not. As such, the dual purpose is crucial: factual information as well as providing credibility for the author to attack the current political landscape.

It is refreshing to read an assessment of the current state of things that to me sounds entirely accurate. Not only does it criticize the entire strategic approach to dealing with poverty:

There is a big disconnect between the grand social engineering agenda of government and the far simpler, unglamorous aspirations and needs of local people, many of whom are not fluent in the ways of jargon.

but it also provides a compelling critique of how the problem is even discussed. Not only does McGarvey examine his own beliefs, but he asks questions of everyone involved:

I always just thought the aim was to dismantle poverty. However, once you see the mechanics of the poverty industry up close, you realise it’s in a state of permanent growth and that without individuals, families and communities in crisis there would no longer be a role for these massive institutions.

It is also good to see acknowledgement of the difficulties in “solving” the problem. We are stuck in a blame narrative which is designed to score political points which only serves to distract from solving the real problems:

Blame for poverty is always ascribed to someone else; an outgroup that we are told not only enables poverty and benefits from it, but also gets a kick out of people being poor.

For anyone who is interested in learning about what it is truly like to experience poverty but is frustrated by how things seem to be progressing (or not), you will find this relatively short book an enlightening read regardless of your political views. And especially if you have political views (left or right), it cuts through the challenges of having a real discussion in our current age of outrage.

Who has the best serverless platform?

Back in August 2017 I wrote a post about: Who has the serverless advantage? AWS Lambda vs GCP Cloud Functions vs Azure Functions. We were three years into commercial serverless offerings and there were still a lot of limitations. My main observation was:

What really matters is the availability and consumption of other services within the cloud provider ecosystem.

Being able to execute functions in response to events is only as useful as what you can actually do within the execution pipeline. This is where the services differ — their ability to to pass data to backend services, perform calculations, transform data, store results, and quickly retrieve data.

AWS benefits from being the leader in cloud by the sheer size of its product portfolio. The core services of compute, storage and networking are commodities — the differentiation is what’s built on top of them.

At the time, Azure Functions and AWS Lambda were similar in functionality and significantly ahead of Google Cloud Functions, which was still in Beta. So how have things progressed for each provider since then, and what does this say about their approach to serverless?

AWS Lambda

It is an interesting coincidence that both Alexa and Lambda were released in Nov 2014, which suggests that the requirements of one product may have influenced the other. Alexa, a deployed on the Echo devices and in other hardware, in an inherently event-driven product. It is only doing things when you trigger it. It fits the Lambda execution model exactly.

If we assume AWS created Lambda to service Alexa as the first customer, this is important because it means the product development is driven by internal stakeholders who provide the initial demand and use case validation well in advance of input from the general market, rather than copying competitors.

Given how well formed Lambda was on release, I think this is a reasonable assumption. The continued development of the product since 2014 means it has a significant headstart, and the internal use case meant there was always an incentive to invest even in advance of adoption by external customers.

This means that Lambda essentially had no competition for 2 years until Azure Functions and Google Cloud Functions came out in 2016, and Google wasn’t really in the game until March 2017 due to the gap between the Feb 2016 Alpha and March 2017 Beta.

Today, Lambda supports runtimes in Java, Go, PowerShell, Node.js, C#, Python, and Ruby and has added complex functionality such as Step Functions and integration into other AWS products. Many other AWS products generate events which can be processed in Lambda which is proving how you can tie together the full ecosystem to deal with logging, metrics and security in particular. The original Alexa use case is still valid, but many of these improvements are clearly driven by external customers.

Lambda@Edge is also interesting because it allows you to execute your functions in CloudFront CDN locations, allowing for low latency applications close to the user. However, it does have some major limitations (such as 1MB response limits and a maximum of 25 deployed functions) so is clearly still an immature product. But what we know about AWS suggests this will progress rapidly. As nicely visualised by CloudZero, serverless is truly a spectrum:

Serverless spectrum

Azure Functions

Azure offers very similar functionality when compared with Lambda, with more language support in experimental runtimes e.g. Bash and PHP.

A big difference is the availability of on-premise functions, where you can run the workers wherever you wish (so long as you have a Windows server). Until the recent release of AWS Outposts, this was a major difference in product philosophy between AWS and Azure. The former was focused on pure cloud, with functionality to help you move there whereas the latter adopted a hybrid approach from the beginning. This makes sense for Azure given Microsoft’s history of on-premise software. Now we see AWS expanding to try and include the relatively small but still significant share of workloads which will always run on-premise.

A famous use case for Azure Functions is Troy Hunt’s Have I Been Pwned (HIBP) project, entirely delivered using Cloudflare (who have their own serverless product, covered below) and Azure.

But it’s not just functions being used – as I mentioned in the original article, the rest of the Azure product portfolio is just as important. HIBP makes extensive use of Azure Table Storage, which you could call another “serverless” product because it is a database as a service – no need to deal with running SQL Server. This is where AWS has often had a lead in the past because of the range of products they have. That’s less relevant today because both Azure and Google have products in all the key areas: storage, compute and databases.

Google Cloud Functions

GCF has only been Generally Available since July 2018 which makes it the youngest platform of the major three. However, it has actually been available in public since 2016. This is something I’ve called App Engine Syndrome in the past:

Cloud Functions seems to be suffering from App Engine syndrome — big announcements of Alpha/Beta features, followed by silence/minimal progress until the next big announcement the following year. The focus of Google’s serverless ambitions seems to be Firebase, not Cloud Functions.

The release notes show regular updates and new functionality during the beta but there was nothing between July 2018 and November 2018.

This is somewhat frustrating because at Server Density we made extensive use of Google Cloud Platform and it is my preferred vendor of the big three – they have the best console UX, documentation and APIs, and I’ve found all their GA products to be well designed and robust.

Indeed, GCF is a good product that is easy to work with through the command line and console. It has good integration into the rest of GCP whether as direct triggers, through pub/sub, storage, monitoring and most recently scheduled functions. The lack of development on the core product is somewhat misleading because of how it continues to be built into the rest of the platform, but it is concerning that the development velocity of what is a growing technology in the industry is seemingly being neglected. At least publicly, and in comparison to the rapid progress of AWS Lambda.

My criticism of Google Cloud has long been that not enough of Google’s own products use it. Of course, they use the underlying technologies and the physical infrastructure, but unlike Amazon.com using AWS for their critical retail operations, it is unclear how much of Google is using the same platform that GCP customers are using. If Google had a use case similar to AWS and Alexa, that might provide incentives to increase the velocity of development in addition to any GCP customers. Maybe they do. But we still see Google falling further and further behind.

Other platforms

AWS, Azure and Google have the position of being the main three cloud providers. Indeed, they are the only ones that matter for general use cases. Their advantage is the vast resources each company can invest in infrastructure and product development. However, that doesn’t mean they are the only players in serverless. There are other, specialist providers who have particular use cases in mind.

Cloudflare Workers

I mentioned Cloudflare in the context of Have I Been Pwned above. Whilst all of the main processing for HIBP happens on Azure, over 99% of traffic is actually being served by Cloudflare’s CDN and their Workers product.

Moving caching and request response serving to the edge reduced HIBP’s cost from $9 per day to $0.02 per day. This was not just by being able to serve many requests within Cloudflare’s free tier but also by eliminating almost 500GB of Azure network traffic:

StackPath EdgeEngine

As HIBP shows, avoiding network traffic is a second order benefit of using serverless. Colocating other services on the same platform eliminates a large amount of traffic so all you need to deal with is internal networking. Serverless certainly means not having to deal with scaling server infrastructure and so spending only based on what you use. But it also means that where you use other products like CDN, you can avoid costly traffic back to the origin.

This is a big part of the use case for why we launched EdgeEngine at StackPath – we have huge volumes of CDN traffic where we serve as the edge delivery provider, but requests still have go to back to the origin for dynamic processing. One of the major selling points of StackPath CDN is the diversity of PoP locations around the world. Your origin might be in AWS US East but if you are serving traffic to Spain, you will still have a significant volume of requests needing to go back to the centralised cloud provider.

Network costs at cloud providers are one of the biggest hidden taxes on using public cloud, especially if you use third party services like a CDN. So if you can eliminate those requests entirely you can not only provide better application latency but also save on your bill. Now we have our EdgeEngine product, you can do things like API token validation without it ever having to hit that central infrastructure.

AWS Lambda vs Azure Functions vs Google Cloud Functions

Lambda created the market for serverless and continues to innovate and lead on functionality. It benefits from the vast AWS product portfolio so is often a default choice for those already on AWS.

Azure Functions are just as mature as Lambda. It’s the Lambda equivalent for Microsoft fans, so is an easy default if you use Azure. .NET languages are well supported, as you would expect.

Google Cloud Functions still has the same problems of slow product development but the overall platform portfolio has improved significantly. It’s likely to the first choice for developers starting from scratch, just because the overall experience of working with Google Cloud is better than AWS or Azure. Google is also innovating in other areas, such as with the Cloud Spanner database product. GCF may benefit from those who want something in the platform they chose for other reasons.

Ultimately, serverless is not just about functions. If you want more than just simple request manipulation or one time processing then you need to be able to connect to a datastore and other services like logging and monitoring. It’s been possible to build full applications using only serverless, like HIBP, for a long time. How sophisticated those applications get will now depend on what other services appear to support serverless functions, and what the role for low latency edge deployment plays in adding to the use cases. To quote Troy Hunt of HIBP:

So, in summary, the highlights here are:

  1. Choose the right storage construct to optimise query patterns, performance and cost. It may well be one you don’t expect (as blob storage was for me).
  2. Run serverless on the origin to keep cost down and performance up. Getting away from the constraints of logical infrastructure boundaries means you can do amazing things.
  3. The fastest, most cost-effective way to serve requests is to respond immediately from the edge. Don’t hit the origin server unless you absolutely have to!

The organisational commit log

As a company grows, I’ve found communication to be the most difficult challenge. Whether it is explaining the company vision to new hires or finding the reasoning behind a decision made 6 months ago, recording knowledge and keeping it up to date requires a lot of effort.

When everyone is in the same office, informal chats are the norm. Even if everyone is remote, a small team can be in almost-perfect sync with simple one to one chats or a basic chat room. But problems occur as more people join the team.

  • Who is working on what? Who might be able to help with current challenges or problems and what experience and knowledge can be shared?
  • What tools, systems, processes and technologies are being considered, implemented or used? Is there relevant experience elsewhere in the company that can help?
  • What decisions have been made about products, customer issues, sales pitches, technical approaches? And why?

Just as a code commit log shows the reasons behind a specific change, where is the organisational commit log?

Email has the advantage of being asynchronous, user customisable (your own clients, filters, etc) and easily searchable, but it shouldn’t be a database for important information and is only useful for the participants of written discussions. Mailing lists could address that and are well understood particularly for open source projects. Stripe has successfully used email with internal mailing lists to improve transparency but have had to made adjustments as they scaled.

Group chat is a good way to communicate with individuals or small groups but has a lot of major issues, particularly with interruptions, and requires everyone to pay attention to every message and decide if it’s relevant to them.

Unlike email which has subjects and forced threading, chat tends to be more of a stream, very often with multiple discussions interleaved into the same chat. If you have teams in multiple timezones, it’s very easy to wake up to hundreds of messages across many different subjects (because people don’t actually use Slack threads!) which may or may not be relevant. I like the ability to pipe messages from other system so there’s only 1 place to catch up e.g. Zendesk tickets, but it doesn’t seem like a sustainable way to manage a large organisation.

Meetings are another place for knowledge to become lost. Unless you have a disciplined approach to note taking, recording decisions and the reasons behind them is easy to neglect. We’re starting to see some “AI” driven attempts to solve this, particularly with the Microsoft Teams Meeting Transcription product. Having meetings transcribed means that discussion now becomes searchable and can be referenced in commits, designs, comments and other documentation.

Documentation itself has to become a core part of how a company operates. Something like Confluence, a central wiki that everyone has access to, is more efficient than having thousands of individual documents in various cloud storage systems. I find the daily summary of changes of the StackPath confluence a valuable way to passively see what other teams are doing. But it’s only really effective if every team is deliberate in writing everything up and keeping it up to date.

Wasn’t this supposed to be a problem that was solved by the Google Search Appliance? And then Google Desktop. And Google Drive. And now Google Cloud Search.

These are all things which tend to only become an issue as a company grows, by which time it’s almost too late to have writing organisational history be part of the culture. Forcing everyone into the mindset later on is difficult, which is probably why communication remains the biggest challenge every organisation faces.

So you want more employee company ownership? The answer seems to be less government intervention – not more

Originally written for ConservativeHome.

Employee ownership of company shares is a good idea. As an employer, you want your team to be incentivised towards the success of the business. That’s why it’s fairly standard for startup companies to offer generous stock options as part of the compensation package when competing in the hiring market. Linking employee effort to shareholder returns seems like a reasonable goal.

Unfortunately, it’s not quite as simple as that.

To begin with, options are just that: the option to purchase stock in the future at a price fixed today. You don’t actually own anything – only the potential to own something in the future. Options are used by new firms because the actual valuation of the company is often uncertain, and startups are typically unprofitable for some time, so dividends are not paid.

They are also used to encourage employees to stay with the business. It is usual to have vesting terms whereby the employee must stay with the company for at least a year, and then a certain proportion of the options vest over a period of years. This encourages employees to stay, which is good for the company. But if they do want to leave, they only have 90 days to exercise those options – assuming they have the cash to exchange them for stock. In the US, that limit is ten years.

Options are a necessary instrument because ,if you do decide to give someone stock, that stock is an asset which has a value. And that value incurs a tax charge in a similar way that giving someone cash would incur income tax. It’s simply considered an employment-related benefit, and so HMRC says it must be taxed.

But what if you’re given stock in a company that isn’t yet public? A large company might have a private valuation, but not have floated on the stock market. The stock is not easily tradable and has no public value, so employees are left with a tax charge against a theoretical valuation that must be paid in cash.

Of course, there is a mechanism to protect employees from that tax charge today, deferring it to the future – the so-called s431 election that must be signed within 14 days of the grant. If you forget, or don’t know, you might find you have to pay a significant amount. Hopefully your accountant is paying attention. Assuming you have an accountant?

This is just the beginning of the complex rules around employee company ownership, all built up over time, no doubt, to try and balance the benefit of employee stock ownership with the challenges of tax avoidance through elaborate corporate structuring. Perhaps the reason why employee ownership is so low is because of the unreasonable complexity?

Simplifying the tax code is where we should be focused. Not with a possible future Labour Government forcing companies to issue up to 10 per cent of stock to a collectively managed fund. A ten per cent stake is significant, particularly for the largest companies, and would mean material dilution for other shareholders. The ownership makeup of UK shareholders is not just individuals (who only make up only 12 per cent) but unit trusts, pension funds, insurance providers, and over 50 per cent of stock market ownership is foreign investment. When there is a sudden, unexpected transfer of wealth forced by central government, how would investor confidence be affected?

The structure of Labour’s proposed ownership fund itself is also unclear. How would the managers of the fund get elected? What voting rights would they have? What kind of employee is interested in getting involved, and what would that mean for the kind of actions they pursue? Shareholders rarely have all the information available to the executive team, even if they are part of the ordinary workforce: indeed, that is why operational responsibility is delegated to executives. Active management by activist shareholders is unusual and often drastic. A more pragmatic proposal to involve employees in key decisions is through worker representation on boards – then they at least have information rights to understand and participate in those decisions. 

Linking employee benefits to the success of the business is a good incentive and gives workers a meaningful stake in the output of their efforts. But if this is indeed the goal, why is there a £500 annual cap? This is a clearly misaligned incentive: why put any more effort in once you hit the bonus level? There is a big altruistic assumption that employees would work more just for the benefit of the state, an assumption already disproved with the exodus of high earners during the period of extreme income tax bands in the 1970s.

When something isn’t working, the solution is rarely for the Government to force a solution that only involves first-order thinking. Employees don’t own enough of their company? Let’s have the government force them!? No. We must think deeper and consider the root cause. If the rules were a lot easier to understand and the tax implications less onerous, aligning incentives would be much more appealing, making employee ownership easier and more popular. Rather than more government intervention, the answer seems to be the opposite.

Things they don’t tell you about M&A

Startups exiting through M&A is incredibly rare. Indeed, you can expect at least 70% of companies that raise money to either fail or become self sustaining. There are also different stages where you can sell for an optimum outcome – miss the stage and it becomes more challenging to achieve valuation goals.

But if you are lucky and do go through an M&A transaction, it’s worth being aware of what’s going to happen. The process is set up for the buyer’s advantage. Just like buying a house, the founder/CEO probably only experiences M&A once or twice in their lifetime, but the other parties do it regularly. With the possibility of a life changing financial outcome, the psychology of selling is important to keep under control.

There’s quite a lot of M&A advice online, but it tends to focus on the vagaries of “how to get acquired” or what the formal process is. Having gone through this earlier in 2018, I thought I’d share some of the specific things I’ve experienced that might help other CEOs. This post is about the transaction itself. It’s too early to comment on post-close integration but this post by Steven Sinofsky with lessons from Microsoft is great.

Signing the LOI/termsheet is just the beginning

It’s easy to fall into the trap of thinking that once you’ve negotiated and signed the termsheet, the rest is just a formality. Actually, it’s just the beginning.

Due diligence is about the buyer confirming what you’ve said is true, and trying to find reasons not to do the deal. Of course, everyone is acting in good faith but this is a formal legal discovery stage to understand risk.

We only hear about the successful transactions. Deals fall apart all the time, even at the very last moment.

If there is anything unusual in your business it will be discovered in due diligence, but that is the worst place for surprises. Disclosing anything upfront that you think might worry the buyer allows you to control the messaging and avoids surprises and questions about why you didn’t reveal it sooner. The whole exercise is about trust, which is all too easy to break.

You have to be a project manager

There are so many people involved in a transaction that it can be easy to assume someone else is driving things. You will have teams of lawyers from both sides, tax advisors, executive and technical team members doing due diligence and probably your own M&A advisor as well. There’s a lot of keep track of.

However, just like when you are buying or selling a house, it is up to the principals to keep on top of everyone and make sure things are progressing speedily. Advisors are paid by the hour so their incentives are somewhat misaligned with your own.

Your goal as the seller is to close as quickly as possible so you must keep things moving at all times. Be relentless in chasing your own team as well as the buyer.

Lawyers will only follow your instructions

Your lawyers are paid to execute your instructions. They are usually good problem solvers but will not necessarily come up with creative or wildly different solutions.

We had an instance where our lawyers discovered a problem with a particular aspect of the transaction. They spent half a day going back and forth with the buyer’s lawyers and came to the conclusion we needed to engage additional external advisors, pushing back close for several weeks. This was on the Friday before we intended to close the following week!

Upon being informed, I was able to instantly change the instructions to our lawyers because this wasn’t actually important to resolve right away, and we could deal with it in the post-close process.

The lawyers were doing their job – trying to follow instructions – but they can’t adjust those instructions in the same way the principle can.

Maintain open communication with your counterpart

I was in constant communication with the main person on the buyer side by phone and text. Providing status updates and asking/answering quick questions using informal messaging really helped us keep things moving.

This helped when there was a particular check the lawyers were doing to verify signing authority from one of our investors. It was clear that this investor had authority by looking at the public company records, but it wasn’t quite sufficient to prove authority for the lawyers. Strictly speaking they were probably correct but I was able to get the buyer to instruct their lawyers to lighten up the requirements thereby unblocking a sticking point.

There will be lots of little questions and problems that should be dealt with directly and not via lawyers or advisors. Be careful not to become too friendly because you are still in a transaction, but having a trusted counterpart you can work with closely really helped us close quickly.

Investors vary

You will need to engage with all of your shareholders because you have to get them all to sign, probably physical papers as well as electronically.

We did this in stages.

A few weeks after the LOI was signed as we were getting towards close, I sent out a general message to let everyone know that a transaction was in progress and what the intended close date was. Everyone was then kept up to date because the last thing you want is for someone to disappear on holiday or be away from connectivity.

You also want to minimise any unnecessary involvement, especially from minority investors. It’s difficult enough to close a transaction without random investors trying to negotiate with their own agenda.

This is where it is helpful to have professional investors who have seen transactions before, and angels who have been CEOs. They know what it’s like and keep out the way, being quick to reply and helpful when needed.

Others cause problems. They think they should be involved as if they were board members or majority investors. All that does is make you never want to work with them again. The last thing you want to do is use your drag-along to force the deal: it’s a red flag to the buyer, creates legal risk and ramps up your fees.

The lesson is: reference check your investors. Find out what they were like for the entire lifecycle of an investment.

How to learn product management

For the last few months since the StackPath acquisition, I have been shedding all the administrative tasks of a CEO of a small startup and focusing more and more of my time on product.

This has been initially scoped to integrating Server Density monitoring into the StackPath platform but has been broadening to multiple products across the platform.

I am used to shifting between many different tasks and responsibilities so focusing entirely on product has been a new experience for me. As a result, I have been spending as much time as possible learning about what it means to do product management.

Learning something new is a great time to write about the experience. There are valuable insights that can be shared from a beginner mindset. Once you “know” something, you think about problems in a different way.

So this post is a collection of the resources I’ve found useful in learning about running product engineering ~6 months into the role.

Books for product managers

Product management podcasts

I have yet to find a good podcast that is just about product management, so here are some specific episodes from more general podcasts that I’d recommend listening to.

  • Masters of Scale: Marissa Mayer – I find this podcast series very difficult to listen to because it is incredibly over-produced, but this one made me do further research into how Google runs product management and so was valuable in that sense!
  • The A16Z podcast as a whole is worth listening to, but specifically related to product I would suggest High Growth in Product (and tech) which is a podcast interview with Elad Gil of the High Growth Handbook mentioned above. Also listen to The Basics of Growth Part 1 and Part 2.

Events for product managers

Generally I don’t find attending conferences to be a good use of time. The travel, disruption to routine and low signal to noise ratio of talks means I’d usually much rather watch the videos after. However, I have found these to be worth the time:

  • Mind the Product Conference is the main conference but I attended only the Leadership Forum, which was worth it because of the small number of attendees.
  • Every industry has their own niche conference which is worth attending just to understand the overall landscape. For monitoring, it’s Monitorama. And for SaaS in general, it’s SaaStr. Be very picky and very specific.

Product management blogs

Where specific articles but not the whole blog is useful, they’re listed in the next section. These blogs are worth subscribing to in their entirety.

Articles for product managers

Product management videos and talks

  • Customer Obsession – from ProductTank San Francisco, this talk outlines: the balancing act of delighting customers in hard-to-copy margin-enhancing ways; how “customer obsession” helped Netflix to create a highly personalized experience; and the principles of customer obsession through a case study — “Should Netflix send a free trial reminder to its customers at the end of their four-week trial?”
  • Mastering the problem space for product/market fit – this is a framework covering the universal conditions and patterns that have to hold true to achieve product/market fit. Each layer in the pyramid is a key hypothesis that you need to get right in order to build the next layer and ultimately achieve product/market fit.

Good quotes on product management

Some select quotes from the linked content above that’s worth highlighting by itself.

In the 10+ years since AWS’s debut, Amazon has been systematically rebuilding each of its internal tools as an externally consumable service. A recent example is AWS’s Amazon Connect — a self-service, cloud-based contact center platform that is based on the same technology used in Amazon’s own call centers. Again, the “extra revenue” here is great — but the real value is in honing Amazon’s internal tools.

If Amazon Connect is a complete commercial failure, Amazon’s management will have a quantifiable indicator (revenue, or lack thereof) that suggests their internal tools are significantly lagging behind the competition. Amazon has replaced useless, time-intensive bureaucracy like internal surveys and audits with a feedback loop that generates cash when it works — and quickly identifies problems when it doesn’t. They say that money earned is a reasonable approximation of the value you’re creating for the world, and Amazon has figured out a way to measure its own value in dozens of previously invisible areas.

^ Why Amazon is eating the world

Perhaps most importantly, the product manager is the voice of the customer inside the business, and thus must be passionate about customers and the specific problems they’re trying to solve. This doesn’t mean the product manager should become a full-time researcher or a full-time designer, but they do need to make time for this important work. Getting out to talk to customers, testing the product, and getting feedback firsthand, as well as working closely with internal and external UX designers and researchers, are all part of this process.

^ Product Leadership

Many books emphasize the first two points—corporate strategy and culture setting. However, you will find that in practice you have little time in a high-growth, rapidly scaling company to think deeply about those points until you hire a strong executive team and manage your own time properly.

^ High Growth Handbook

There’s no point in defining what to build if you don’t know how it will get built. This doesn’t mean a product manager needs to be able to code, but understanding the technology stack — and most importantly, the level of effort involved — is crucial to making the right decisions.

^ Product Leadership

Another lesson that I learned from Brian Chesky—one way to think about when to upgrade executives—is that a really great executive is about six to twelve months ahead of the curve. They’re already planning for and acting on things that are going to be important six to twelve months in the future. A decent executive is delivering in real time, now to one to three months in advance.

^ High Growth Handbook

The trick to creating a great product team is to think of them as the product. This is not an objectification but rather a thought exercise. After all, they are the product that creates the product. Without them, there is no product. Amazing teams make amazing products. Seen from this perspective, the task of how to hire, onboard, train, and develop them becomes another product design problem. The approach that successful leaders take to creating great product is the same approach they take to creating great product teams.

^ Product Leadership

Often the hardest part of the communication is communicating the “why” behind the product road map, prioritization, and sequencing. Part of this will be creating a framework that establishes why some things are prioritized higher than others—and it’s important that all other functions buy into this framework.

^ High Growth Handbook

Out of the goals will come the specific features for development. Like a ripple effect with the vision at the center, the objectives or goals are generated and they in turn generate the features that support those goals. Never start with features. Even if your business or product is based on a “feature concept,” ask yourself what the bigger problem is and why it needs solving. Any feature shouldn’t be considered, prioritized, or delivered in a vacuum. Without a vision to guide the product creation, a project can quickly become a collection of cool solutions lacking a core problem to guide them. Features need to be directly tied to the product or organization’s strategic goals.

^ Product Leadership

For example, if you as the designer/manager discover that you as the worker can’t do something well, you need to fire yourself as the worker and get a good replacement

^ Principles: Life and Work

If you are not evolving your organizational design, it might be an indicator that your product strategy is getting stale. In our experience, most rigid organizational structures are built to create processes for predictability, not successful outcomes.

^ Product Leadership

As GV’s Ken Norton says, “I like to start with the problem. Smart people are very solution-oriented, so smart people like to talk about what the solution is going to look like to the problem. But successful people think about the problem. Before I talk about this product, or this feature, or this device I’m going to build, I must understand the problem at a deep level. Then success is easy to articulate, because you know what it’s like when that problem is solved.”

^ Product Leadership

“By-and-large” is the level at which you need to understand most things in order to make effective decisions. Whenever a big-picture “by-and-large” statement is made and someone replies “Not always,” my instinctual reaction is that we are probably about to dive into the weeds—i.e., into a discussion of the exceptions rather than the rule, and in the process we will lose sight of the rule.

^ Principles: Life and Work

How to hire engineers: the interview process

Originally written for the Seedcamp resources website.

Earlier this year, I wrote about the first step in hiring – how to source candidates. Once you have applications, then you need to evaluate them to decide who you might want to hire.

Regardless of how urgent the need is to fill the position, finding the right people, not just for the role today but for how your business will change in the future, is crucial to success. This post will take you through how to create a robust selection process for hiring engineers.

The goals of the process

You have to remember that you are still in a sales process. You are not just trying to match applications against your person spec but you are also trying to convince them to accept the offer you might make at the end. This means there are several goals to consider:

  1. Evaluate applications against what you are looking for in team members now, and in the future. You need to balance the requirements of the job today with an ability to adapt as the business changes. This is particularly important in early-stage startups. Past experience may be relevant to demonstrate ability to execute, but knowledge of specific technologies is probably not – the best engineers can learn new skills, languages, frameworks and systems.
  2. Continue to demonstrate why your business is a great place to work. This comes in multiple parts, the first of which is well before you even get applications. Building your profile and supporting website materials is important for getting applications in the first place. It is just as important that the interview process runs smoothly, the candidates always know where they are at, what they need to do next and what the timeline is. You need to provide regular updates and fast responses. Their time must be valued more than your own and you need to explain to them why they should be joining the company if you make them an offer. You can never take for granted that just because they have applied to you, they will actually accept any offer.
  3. Build a diverse team. This is assisted by the design of the process but also requires you to have the appropriate HR policies in place e.g. flexible working, generous holiday allowances, clear maternity/paternity policies, etc. Thinking about this from the beginning and designing your processes to consider the challenges of diversity means you do not need to do things like positive discrimination, which I do not think is a good way to tackle the diversity problem in tech. The goal is to increase the diversity of the application pool and run an unbiased process to select the best candidates from that pool. Google has some useful guides on diversity in general and there are several good resources for working on gender diversity.

The basic foundation for running a good engineering interview process is valuing the time of the candidates. They likely have full time jobs and/or consulting gigs, so you cannot ask candidates to spend many hours on the phone, doing coding tasks or building projects. Of course they will need to give up some time to dedicate to the process but you should work hard to minimise it.

Step 1: Application

The usual application is a simple form which asks the candidate to submit their basic details, a CV/resume and a short cover letter explaining why they are interested in the job. The cover letter is the most important aspect and the only element that is actually examined at this stage.

In the job ad I include an instruction which asks the applicant to mention a keyword in their cover letter. If the keyword isn’t present then the application is instantly rejected. This is specifically to filter out mass, shotgun-type applications and to test for attention to detail.

The best people will usually only ever apply to a small number of positions. You want to find people who take the time to consider the company and role well in advance of ever applying, which means reading the full job ad and description.

Where possible, this step should be automated. Only collecting the minimum amount of information e.g. email, cover letter means you can systematically ignore any other details of the application, such as the CV, name, which might introduce bias. Be aware of protected characteristics and things you cannot ask.

Just like college degrees being mostly irrelevant for engineering positions (unless you have some very specific scientific knowledge you require), some companies are now excluding CV submission entirely. This is worth considering as another way to remove potential for bias. The only thing I find CVs useful for is to research interview questions in advance, but everything you need to know you can simply ask the candidate when you speak to them later.

Step 2: Writing exercise

I have found there is a good correlation between ability to write well and coding ability. Programming is all about clear and accurate communication, whether that is directly in code itself or communication about the project with real people!

I test this by requiring candidates to do a short writing exercise whereby they have an hour to research the answer to a particular question, and write up the response. The question should be relatively easy because the focus is on their written answer. You are simply looking for accurate spelling and grammar. Any mistakes should mean an instant rejection – if they are unable to write such a short piece without mistakes or proper proofreading then that indicates a lack of care and attention.

The task should take no more than an hour and you are not looking for technical accuracy of the response. This is purely an assessment of clear and accurate communication.

Step 3: Coding exercise

Designing a good coding exercise is tricky. It needs to be representative of the kind of skills you need for the role. It should allow the candidate to demonstrate a wide range of skills, from writing clear code to tests and documentation. And it should be straightforward to build in a short period of time – a couple of hours is ideal.

One of the more successful exercises I have used in the past is to ask the candidate to build a simple client for a public API. This tests many things such as working with real world systems, understanding credential management and dealing with network issues and error handling.

Whatever you pick, you want the candidate to be able to create a self contained package or repository, with some basic installation and setup documentation so that you can evaluate both whether it works, and the implementation itself.

Before starting this, as an engineering team you need to create a list of objective criteria that you can score the exercise against. These can include things like checking the documentation is accurate, test coverage, code linting, etc. You can determine your own criteria but they should be as objective as possible so that each evaluator can compare their conclusions.

Once the candidate sends you their completed exercise, the code should be given to several of your engineers to evaluate. This should be done blind so the evaluators only see the code, and they do not discuss the details with each other. This gives you several independent evaluations and avoids any bias. Be sure to instruct the candidate not to include any identifying information in the package e.g. a Github URL or their name in an auto-generated copyright code comment.

Step 4: In-person pair programming

At this point you have done most of the evaluation and believe the candidate has the skills you’re looking for. The final stage is to evaluate actually working alongside you in a more realistic situation. For this, I prefer to meet candidates in person and have them work alongside their potential colleagues.

I have done this stage remotely in the past but have found that it is more effective to meet someone in person. You can then evaluate what they are like as a person. However, this is also the stage where there is most risk of bias. You can mitigate this by involving multiple people from your team so that one person doesn’t have a veto.

In the interests of speed and efficiency, I try and schedule all final interviews within the same week. This may not always be possible but I try to batch them as closely as possible. This makes the best use of your team’s time and means that candidates can get a response quickly.

You should cover all travel costs for the candidates, booking tickets for them rather than making them pay with reimbursement – they shouldn’t have to loan your company their own money! If they have to travel a long distance, offer overnight accommodation, transfers and food. Also ensure they have a direct contact who is available 24/7 in an emergency. You want candidates focused on the interview, not worrying about logistics.

Again, you need to determine what the best approach to evaluating their capabilities is. I have found that getting them to actually work on your codebase is a good way to see how they deal with an unfamiliar environment and start to learn a new system. You can ask them to fix a known bug, or introduce a simple bug into the code and work with them to fix it. You are not testing them on their knowledge, but on how they approach the problem. Whether or not they fix the problem isn’t important.

Remember that this continues to be a sales process. Take the time to introduce them to key members of team, show them around the office and, if they’re not local, the area where they’ll be working. Be sure to show off and explain why you want them to join. This is the job of everyone on the team – multiple people telling them about the company is a lot better than just the hiring manager or CEO!

Step 5: The response

Anyone who gets past step 1 should receive a response to their application whether they are successful or not. One of the worst things about applying for a job is not knowing what the decision was.

The challenge with giving a negative result is that candidates will often ask for feedback and may argue with it. It is up to you whether you want to do this at all, but I usually offer detailed feedback only if a candidate reaches step 3 or 4. Failing step 2 is only for poor spelling/grammar, which you can build into an auto-generated response.

If you are going to make an offer, do it as quickly as possible. Include the key information about the compensation package, start date and anything else you need from the candidate. Be sure to review the legal requirements for a formal job offer first.

Don’t use exploding offers and don’t pressure the candidate. During the step 4 interview, you may want to ask them what their evaluation criteria are and whether they are looking elsewhere. Asking them when they think they will be able to reply to you is probably fine, but  don’t ask about salary expectations.

What not to do

You may notice that certain things are not present in the above process.

  • No questions about their background and experience. It is not necessary – you are evaluating them based on their skills and how they apply to them today, not what they claim to have done in the past. That said, in step 4, you may want to ask a few questions about how they may have tackled similar problems in the past, or what interesting challenges they have solved if you are hiring for a very specific problem area. But really, you want to put as much time as possible into designing your coding exercises so they are representative of the problems the candidate would have to solve if they were working at your company. Let them demonstrate their ability, not talk about it.
  • No knowledge questions or puzzles. The ability to recall function definitions or solve theoretical problems is not particularly useful for evaluating whether someone can write good software.
  • No whiteboarding. You may want to use a whiteboard to explain specific system architecture but there is no place for actually coding on a whiteboard, on paper, or anywhere that isn’t a modern IDE or code editor of the candidate’s choice. Nobody codes in isolation without access to the internet to look things up. Everyone has their own preferred coding environment and the coding interview will likely place them in an unfamiliar setup without their usual shortcuts and window layout, so be sure to make allowances for this too.
  • No phone interviews. Again, get the candidate to demonstrate their ability through real tasks, not be explaining what they might do or have done.