Hiring for continuous improvement

When I was CEO of Server Density I was always thinking about how I might be able to do my job better, and how we might operate better as a company. It felt like there were improvements to be made and new things to try.

Whether it was a new post on SaaStr, the latest book I read or something posted online by a company I admire, the idea of continuous improvement was a regular feature in my thinking about the company.

The most recent thing I’ve seen is Shape Up by Basecamp.

This book is a guide to how we do product development at Basecamp. It’s also a toolbox full of techniques that you can apply in your own way to your own process.

Whether you’re a founder, CTO, product manager, designer, or developer, you’re probably here because of some common challenges that all software companies have to face.

Shape Up, Introduction

Product and software development is challenging, especially when the business scales with more engineers. Most people know about agile and scrum but are they the most effective way of doing things? Projects are still late. Management still ask for engineers precise estimates. Engineers still get frustrated.

Having read much of Shape Up, there are principles there that I recognise from how we did things at Server Density, but also many new ideas that I would have liked to try. Their approach is exactly the kind of thing I would have considered proposing to my team.

That’s one way of introducing new ideas – having the CEO / Co-Founder suggest something. Often, a “suggestion” from the CEO is actually an “instruction”. It’s all very well having the authority of a company founder but it’s not always the best way to do encourage improvement. You need to be careful of forcing things through just because you’re the boss. One way to do this is to frame the discussion or proposal properly with the rest of the team e.g.

When I’m leading through a tough decision, I try to say, at the outset, “I want all of your opinions, but I’m going to be the one who ultimately makes the decision.” Or in some cases, I will say, “I don’t know if I’m the right decision-maker. I need help exploring what the decision vectors are, and I need all of your help. And then I will let you know how we’re going to make the decision once we’ve talked about it.” If you don’t give people that guidance, which is I think a common mistake, you’re likely to run into trouble.

High Growth Handbook, Elad Gil

Improvement as part of being a “good team”

In discussing some of the ideas in Shape Up with my co-founder, Harry, he made the good point that the real barrier to introducing change (even just experimentally) is often within the team itself. You have to get other team members to buy in (to the principles, or at least to the idea of trying new things). There is value in having process (even at a small company) so long as it helps rather than hinders the way people get things done.

Sometimes team members just don’t care or don’t understand why things are worth improving. They may not willing to put in the work to form an opinion or they may just be happy with the status quo.

This is different from team members not having the time or being so pressured there is no room for thinking about how they improve their workflow.

It’s also not the only thing that is important, and may only come up occasionally. There needs to be some predictability in daily work – you can’t be changing things all the time. The person who turns up with a new idea every day can become quite tiresome!

But equally, the definition of what makes a “good team” should include some degree of willingness to read, research and suggest ideas for how things could be better. Many other professions have the concept of Continuing Professional Development. It’s often a mandatory part of remaining qualified for the likes of lawyers and in many scientific or medical roles.

In a competitive industry, you want to find people who share a similar desire to improve workflows and outcomes for the whole team, not just working for their own selfish benefit.

How do you hire for continuous improvement?

This is a new realisation for me about what makes a good team. I’ve always has the luxury of the founder authority to propose ideas, but really you want the team to come up with their own suggestions for how they can improve their own work. The founder (or manager) can then support them in their experiment.

So what are some questions you could use in interviews when hiring for this characteristic, or at least an open willingness to consider new approaches?

  • In your own projects outside of work, how do you learn about new ways of doing things or improvements you could try out? This doesn’t have to be in programming, it could be in a different hobby, family life or their own approach to doing things. Good answers might show a range of sources for exploring new ideas e.g. blogs, reading, Twitter, journals, etc. You are looking for someone who is interested in their industry enough to review other ideas and opinions, and have sources of input they keep an eye on. It’s not the same as always staying up to date on the very latest technologies or frameworks – it’s more of a broad curiosity for their field in general.
  • What do you think of the processes at your current company? Where could they be improved? This question is looking for a critical awareness of how things work, what does/doesn’t operate well and, crucially, any ideas for improving things.
  • Tell me about a time you tried to change something at work. How did you approach it? Was it successful? You want people who have thought of ways to improve things and probably tried to get it implemented. Failure is not a problem if they understand why. And indeed, not suggesting changes is also a valid answer e.g. they may have felt they would have been ignored, or weren’t asked, or some other structural issue that prevented them from speaking up. Some people are more willing to put ideas out there than others, and will do it in different ways.
  • If you could design the ideal process for X, what would it be? Again, this shows they have thought about improvements and considered challenging how things are done. You may want to probe them on “ideal” vs “pragmatic”. It’s fun to discuss the perfect world but there are always constraints. It’s important to understand that.

We often think about technological change, improvement and progress in terms of the raw technology itself. Let’s not forget that it’s the human processes that actually get things done, using that technology. Don’t just get excited about the latest framework or cloud service, also think about ways to improve how things get written, delivered and deployed.

The right way to replace team members

Sometimes people need replacing. Whether it’s because a manager is no longer able to scale to meet the requirements of the business or perhaps their personal circumstances have changed so that they don’t want to take on additional responsibilities, it’s fairly common to find you need to bring in more experienced help.

This is different from firing someone. Although some of the elements might be the same, removing someone entirely from the organisation is a different challenge. Replacing someone implies that the person is perfectly capable of doing their old (or a different) role and will ideally remain as an important contributor the business. That’s what this post will discuss.

What you need to understand as the manager

Before anything happens it’s important to understand and clarify your own expectations of the role. You need to know what you want the responsibilities to be and what the position should deliver for the business. This will provide clarity and precision about where the person is failing or is unable to meet those requirements. You also need a good sense of why they can’t grow into the role now and how they might be able to do so in the future.

The outcome you’re aiming for is not just to find someone who can execute on the role today, but to coach and develop the existing team member so they remain with the organisation and can take on new roles in the future. If they feel like they have hit a ceiling then they will leave. And if it’s not clear why they can’t perform the role today, they will (rightfully) feel aggrieved.

How to approach the team member

The first step is to have a discussion with the person concerned. If you have a good one-to-one feedback process, they should already have a sense that they’re not suited to the role. Nevertheless, this is an opportunity to set out the situation, explain what you are proposing to do and see if they have any suggestions for alternative ways to approach it. It may feel like you have already made a decision but taking the opportunity to discuss the situation may reveal something you have overlooked.

Then you need to discuss a transition plan and timeline.

  • Are you going to hire externally? Announcing that you are hiring for a role that someone is already doing makes it obvious they are being replaced. In this scenario, communication needs to be approached particularly sensitively.
  • Are you going to hire internally? Do you already have someone in mind? Does the person being replaced have any recommendations?
  • What role do you want them to play in choosing their replacement? Most likely the new person will become their manager. Usually, managers hire their team and involve their team in hiring their peers. The same principle applies in reverse: involve the team in hiring their new manager. This is important to ensure the team doesn’t reject the new person.
  • How should responsibilities be handed over? What needs documenting and how will the new person know what they should be doing? Certain tasks may be more important than others and a gradual transition phase may be appropriate.
  • What can changed administratively? Titles, compensation and formal processes around direct reports (expense management, holiday approvals and anything that is already in progress e.g. performance management or sabbaticals). Depending on geography, there may be legal considerations e.g. in the UK you cannot unilaterally reduce someone’s salary.
  • What will be communicated to the rest of the team? Defaulting to transparency sounds good but these changes are always sensitive. Agreeing how the change should be positioned will help everyone understand and avoid unnecessary rumours.

Communicating the change

Once you have figured out the right approach with the individual concerned, the next step is to communicate the change throughout the organisation. There are different types of concerned stakeholders.

  • Their team – new managers are always disruptive because they cause uncertainty. How will the new manager run things? What will their personality be like? There’s potential for a lot of change, which makes communicating what is happening (and going to happen) a crucial part of whether the change is successful or not. Discussing the change with the team as a group is the first thing to do after deciding on the plan. Everyone should be involved (and the person changing roles may even make the announcement themselves). Rumours start easily and spread throughout an organisation quickly.
  • The whole company – people will start to talk as soon as the announcement is made. It’s better to communicate sooner rather than later to keep control of the narrative. The size of the company and the seniority of the role will determine whether this is best done in an all hands meeting or via something like a regular whole-company newsletter. For example, a new CTO should be announced to the whole company regardless of size, whereas an engineering lead in an organisation of 1000 people where there are several other new hires at the same time may only warrant mention via email.

For both groups, the aim is to communicate:

  • What is changing?
  • Why is it changing?
  • What should everyone expect from the change?
  • When will it be changing?
  • Optionally: how to be involved e.g. putting yourself forward for the role.

This all requires effort and thought, and it may seem like you have no time to do this in a rapidly executing startup. Team change is always disruptive but getting this wrong will cause rumour and confusion to grow within the organisation. People will think: maybe I’ll be next. It may not be initially apparent but long term effects to morale result in a loss of trust and significantly reduced motivation.

How can you work effectively if you don’t know when new people might join or why existing team members have changed roles? What is the effect on morale when you feel like you’re out of the loop and perhaps missed an important opportunity for career growth?

Examples of things not to do

Communication is the hardest thing to get right in any relationship, organisation or team. People make mistakes but there are things which should seem obvious not to do.

  • Avoid discussing and clarifying the responsibilities of the role over a long period of time, despite repeated requests for clarification.
  • Inform the person who’s role is changing via email, whilst they are on holiday.
  • Publicly post the job ad on social media for the person’s replacement without informing them that the ad is going out, or involving them in drafting the job spec.
  • Ensure they only find out that hiring has started for their replacement when one of their colleagues ask them for their advice about applying to the role.
  • Involve team members in the interview process for the replacement but not the person actually being replaced.
  • Inform the person that their replacement has been hired by disinviting them from meetings they were previously attending. Just remove them from the calendar invite and have the automated email remove it from their calendar.
  • Inform the person affected that their role and reporting structure has changed by having Slackbot notify them they have been removed from the relevant Slack chat rooms.
  • Don’t introduce the new hire, describe their background or explain anything about their new responsibilities.

Consider how you would like to be treated and think through how to approach the situation, maybe seeking advice from someone who has had similar experiences in the past. Although this is often a difficult task that hopefully won’t happen often, it has an impact on the long term culture of your company. Toxic cultures still exist. Don’t end up being an example of what not to do.

What are the common causes of cloud outages?

The argument for using cloud services goes along these basic lines:

  1. The big 3 cloud providers (AWS, Azure, Google Cloud) invest more money in their cloud platforms every quarter that you will ever invest in your infrastructure.
  2. It is impossible to compete with the level of functionality, efficiency and security of the big 3 cloud providers for what are commodity IT services (compute, storage, networking), and the services built on top of them (e.g. databases, queues, email). The core offerings from the big 3 will always be best in class.
  3. Running IT infrastructure is not a core expertise that needs to be owned to differentiate what your business offers to customers.
Cloud provider capex.
Cloud provider capex from “Follow the CAPEX: Separating the Clowns from the Clouds

As such, all IT services should be outsourced to one of the big 3. Which one you pick will be more of a personal preference because they are mostly the same.

Trust their infrastructure but build your services using the usual recommendations for redundancy and reliability i.e. deploying in multiple zones and ideally multiple regions. Be aware of the cloud tax that is networking.

There is basically no argument for not using cloud until you get to extreme scale. Even then, only very specific use cases are worth considering – those which give you a true competitive advantage.

Netflix is the best example here: almost all their infrastructure runs on AWS, except for their content delivery where they have custom hardware. An example of a company making the wrong decision here is Dropbox moving away from AWS.Did they miss their opportunity as a result?

That’s why I actually find this announcement really disappointing. Apparently Dropbox has been devoting significant resources for at least two years to a project that will no doubt have a positive impact on the bottom line but a minimal impact on the top line…How might have the product and company evolved if the company had continued to rely on AWS and devoted its resources to fixing its product-market-business model fit problem?

Dropbox Leaves AWS, Should UPS and Fedex Be Afraid?

Trusting the cloud

Inherent in the decision to outsource all of your infrastructure to a third party cloud provider is that you must trust them to deliver. IT is mission critical for most businesses so you want to be sure that they are going to do a good job with the reliability of core infrastructure.

Any argument that assumes you can run your infrastructure more reliably than the big 3 cloud providers can be countered by highlighting the above investment numbers and then following architecture best practices: deploy across multiple-zones and regions and test your failover plans.

Cloud infrastructure is a commodity in the same way that electricity and other utilities are. Considering generating your own electricity is absurd. And if your operations are so critical as to require backup electricity generation, the cloud-native approach is to deploy across regions and perhaps even consider multi-cloud.

That said, no complex system can achieve 100% uptime. You have to assume failure, plan for it and test it.

So when things do go wrong, what are the common causes of cloud outages? What causes cloud downtime?

How do we know what causes cloud outages?

What is a cloud outage?

First we need to define “outage”. Some define it quite simply:

Cloud Outage simply refers to the duration when the cloud infrastructure service is unavailable for use. The unavailability may also refer to performance inadequacy of the service, as per the agreed SLA metrics.

What is a Cloud Outage? Cloud Outages Explained

But it is actually more complex than that.

Consider for example “monthly uptime percentage for a VM will be at least 99.99%.” How are we measuring uptime? On what granularity (seconds, minutes, calendar months, rolling 30 days,…)? What is ‘up’? The VM is provisioned? The VM is running an OS? The VM is reachable from the Internet? Is a performance brownout an outage? And so on.

For cloud providers things get extra complicated due to multi-tenancy, and the fact that the behaviour of their clients can also impact SLIs. As a simple example, an SLO around network throughput might rely on the customer running software capable of driving the network fast enough. Or a system availability SLO, for availability seen by the end user, may well depend on the user carefully exploiting the available redundancy.

Expressing available SLOs in terms of ‘nines’ also causes some issues: it hides the difference between many short outages and a few long ones, which is something many customers care about. It also treats all outages as equal, whereas an outage on Black Friday for a retailer is much worse than an outage on a quiet day of the year.

Nines are not enough: meaningful metrics for clouds

Methodology

For the purposes of this article I am going to look at the public post-mortems published by the 3 big cloud providers (AWS, Azure and Google Cloud) since 2016 for their core compute, networking and storage services only.

I’m excluding other services from those cloud providers because because the higher level services e.g. databases, will rely on these commodity low level primitives. Compute, networking and storage are the lowest level services that customers can build their own applications on.

The other cloud providers (e.g. Oracle, IBM, Alibaba) are not relevant because they are so small. The only reasons to choose them are political.

In each case I will try to categorise each into 1 or more generalisable root cause e.g. software bug or misconfiguratiom.

Cloud environments are complex so causes such as “a server failed” are highly unlikely. Outages typically involve a cascade of failures with multiple systems. Where possible I will try and find a single root cause category but it may be multiple problems that ultimately resulted in the outage.

Limitations

Cloud providers do not provide consistent reporting for their outages. They have their own criteria for which ones get a public post-mortem, how detailed it is and whether it covers everything that happened.

That means that this analysis is more anecdotal than scientific, but it should hopefully be useful for understanding what the common root causes are.

What causes cloud outages?

The data below shows that software bugs cause a majority of outages. Sometimes this is due to poor testing but mostly it is due to race conditions or unexpected situations triggering unusual code paths.

The actual number of outages is not important. I have not separated outages that affected a single zone (more frequent) from those that might have brought down the entire cloud platform (very rare). The lesson in the former case is to architect your application properly.

More detailed analysis about what those software bugs are can be found in a paper specifically analysing Azure – What bugs cause production cloud incidents? (or a writeup of the paper itself).

Among all incidents caused by bugs, the most common causes are (1) incorrect or missing detection and handling of component failures (31 %) and (2) inconsistent data-format assumptions held by different software components or versions (21 %). Timing bugs are also common (13 %), with many of them related to conflicting accesses to not only in-memory data but also persistent resources. Finally, incorrectly set constant values are non-negligible (7 %)

What bugs cause production cloud incidents?

Although this paper focuses just on Azure, engineering approaches are likely to be similar amongst these industry-leading companies. This means we can assume a similar situation at all cloud providers.

Outages before 2016

Given the speed of development of technology, I have limited my methodology to looking at outages since 2016. However, there is more data available historically such in the paper Why Does the Cloud Stop Computing? Lessons from Hundreds of Service Outages which looked at a broad range of cloud outages from 2009 to 2015 across 32 different cloud services.

AWS Outages

AWS does not provide an easy to access history of public incident post-mortems. The AWS Post-Event Summaries page only lists 2 major outages since 2016 but this list is not exhaustive. For example, the “Summary of the AWS Service Event in the Sydney Region” in June 2016 is not listed.

This lack of transparency makes it difficult to analyse the types of incident that AWS deals with and so we cannot make any conclusions.

Incidents

Azure Outages

Azure only provide 90 days of incident history and do not offer unique links to each root cause analysis.

Summary

  • Misconfiguration: 3
  • Resource exhaustion: 1
  • Inconsistent data replication: 1
  • Software bugs: 2

The sample size is small due to the public incident history limit. According to the paper “What bugs cause production cloud incidents?“, 40% of all incidents over a 6 month peweriod were caused by software bugs:

In this work, we systematically studied all the high- severity production-run incidents during a recent span of 6 months in Microsoft Azure services, which cover a wide range of services including computation, storage, data management, data analytics, IoT, media services, etc., and identified software bugs as the most common cause of cloud incidents (close to 40%). We then did an in-depth study of all the 112 high-severity production incidents that are caused by software bugs.

Incidents

  • 2019-06-14 – Virtual Machines and Virtual Networks (Management) – North Europe
    Root cause: misconfiguration.
  • 2019-05-22 – Service Management Operations – West Europe
    Root cause: resource exhaustion.
  • 2019-05-13 – Network Connectivity – Increased Latency
    Root cause: inconsistent data replication
  • 2019-05-02 – Network Connectivity – DNS Resolution
    Root cause: misconfiguration, software bug.
  • 2019-04-16 – Networking Degradation – Australia Southeast / Australia East
    Root cause: misconfiguration.
  • 2019-04-09 – Virtual Machines – North Central US
    Root cause: software bug.

Google Cloud Outages

Summary

  • Human error: 3
  • Misconfiguration: 12
  • Resource exhaustion: 4
  • Router failure: 1
  • Software bugs: 19

Incidents

A guide to remote working for startups

When I co-founded my company in 2009, I had no choice about whether I was going to be running a remote, distributed business.

I was living in student accommodation, halfway through my second year at university. Money was limited so I built the initial product with one of my friends, bootstrapping the company from savings and then customer revenue. We couldn’t afford an office, and were happy working from home anyway – much easier to get to lectures on time!

Over the following 9 years, we grew the company on a remote-first, distributed model. All our engineers worked remotely and we only opened a physical office after 3 years so that we could bring the London team together. Later this evolved to deliberately building the commercial team in the office, but still operating using a distributed model for engineering.

At the time (2009), there were very few companies doing the same thing. Remote work was weird and rare. Most people commuted to an office.

That is still the majority of cases, but there are now companies operating at significant scale in a mostly or entirely remote, distributed way.

It is becoming clearer that office-based jobs can be done more effectively from home. It doesn’t work for everyone and remote working is not a magical cure to all the problems of an office. Indeed, remote working has its own challenges but it is becoming more desirable with clear benefits to both the employee and employer.

I fell into this mode of working through necessity, figuring out what to do through trial and error. This post is about how that worked and the lessons learned scaling to 20+ people globally.

Does working remotely work?

Yes, there are many of examples of successful companies operating on a remote, distributed model. It is not an unusual way to work.

The number of remote workers in the US grew by 79% between 2005 and 2012. The US Government issues annual reports on its approach to teleworking. For fiscal year 2017, they report:

improved attainment of goals in almost every area: improved employee attitudes; emergency preparedness; recruitment; retention; reduced employee commute miles; improved employee performance; and reduced real estate costs

2018 Telework Report to Congress

Cost savings were also tracked:

Over a quarter of Federal agencies were able to track some form of cost savings due to telework (29 percent). Those agencies increasingly reported cost savings achieved through telework, especially in the areas of transit/commuting, rent/office space, and reduced employee absences.

2018 Telework Report to Congress

Similar effects have been studied in private companies, too:

Nicholas Bloom and graduate student James Liang, who is also a cofounder of the Chinese travel website Ctrip, gave the staff at Ctrip’s call center the opportunity to volunteer to work from home for nine months. Half the volunteers were allowed to telecommute; the rest remained in the office as a control group. Survey responses and performance data collected at the conclusion of the study revealed that, in comparison with the employees who came into the office, the at-home workers were not only happier and less likely to quit but also more productive.

To Raise Productivity, Let More Employees Work from Home

The evidence is there to show that remote working has tangible benefits. The list of companies implementing this at scale is impressive.

Examples of remote, distributed companies

One of the big challenges I faced growing my company was there were very few businesses that were operating remote-first. Emphasis on “first”. There are thousands of distributed companies – any business that has more than 1 office has to deal with employees in different locations. However, whilst they might use many of the tools you will find in a remote-first company, they still apply office-based thinking to solving problems.

Today, we not only have startups adopting a remote-first model, there are companies at large scale doing this successfully.

Examples of remote, distributed companies: Automattic, GitLab, InVision, Zapier, Buffer and Basecamp

Automattic, the business behind WordPress, Jetpack and many other projects, has been doing this for over 15 years. They are now approaching 900 people.

Much like WordPress sought to change the way we publish on the web, Automattic has set out to change the way we work. We are an entirely distributed company — with more than 800 employees working from more than 62 countries, and no physical headquarters. We are a company that works on, and for, the web.

About Matt Mullenweg

Another company doing this at scale is GitLab. They have over 650 people working from 50 different countries, all-remote. To make this work, they have also adopted a culture of transparency both internally and externally. This has proven extremely helpful for sharing how they manage the company and setting them up to be a shining example of how to run a remote business.

At GitLab, everything is documented in a 2000+ page public handbook. From their all-remote manifesto to how they manage hiring and how their customer success team operate is covered. Writing is key to success for all businesses and GitLab go one step further by sharing this so others can see what might work for them.

Remote working has an influence on how your own products develop. Automattic developed the P2 theme which is what they use internally for communication. InVision have over 1,000 people working for them remotely, and this has influenced features in their own products.

For the team behind InVision Studio, working remotely has helped them do more than just communicate and collaborate better—it’s allowed them to spot gaps in their own workflow and then apply those solutions to Studio as they build it. What better way to know whether your platform’s features are intuitive, accessible, powerful, or complete unless you were thrust into the exact scenario in which they’d be most needed?

Building InVision Studio with a fully remote team

On the smaller side but still quite big (and growing), Zapier has over 100 people all working remotely. Another tech startup offering a technical product, Zapier have written extensively about how their company works and how to succeed in a remote, distributed model.

It’s not just companies selling to engineers or those in the same industry. Buffer is 80+ people, all remote. And Basecamp, one of the original remote-first companies who’s founders wrote Remote all about distributed work, are currently at around 50 people.

So is the remote, distributed model the future of work? Matt Mullenweg of Automattic certainly thinks so:

Core principles for a remote, distributed company

The tools used to run a remote, distributed company are relevant but you have to apply some core principles first. A degree of process is necessary in any company but is particularly important where remote work is concerned.

Many companies focus too much on technology and not enough on process. This is akin to trying to fix a sports team’s performance by buying better equipment. These adjustments alone might result in minor improvements, but real change requires a return to fundamentals.

Why Remote Work Thrives in Some Companies and Fails in Others

Communication

Communication is the hardest part of running any company. It is important for everyone to have a clear sense of the company mission and values, how things are going and what their impact should be.

This is harder in remote-first companies because it requires more effort than simply talking to someone sitting next to you.

That said, many of the strategies for addressing this will significantly improve the working environment and culture of any company, remote-first or not. I’ve written about this is much more detail so won’t cover it here:

Establishing a regular communication and reporting cadence is a standard part of running a public company – financials and progress get announced in an expected format on an expected timeline. If public companies can do this on a quarterly basis, there’s no reason why smaller companies can’t do it too, and likely much more regularly.

Communication inside startups – what and how

Trust

There are industries where trust is not necessary. Those are often environments where the job is based on rapid, repeat assembly work. Factories or perhaps certain types of manual labour. In these roles, employees are tracked so closely to try to engineer trust out of the equation.

That said I’d still argue that regardless of the job, you want to hire people you can trust otherwise you end up depersonalising the role. That’s not a good working environment.

The context of this article is knowledge-work such as technology, engineering, science, research, i.e. roles that could be office-based.

If you can’t trust your people to do what they should be doing without you standing over their shoulder, you’ve hired the wrong people. That’s not a process problem, it’s a people problem. 

I Swear I’m Wearing Pants: 5 Misconceptions About Remote Employment

The worst kind of work is where you are simply following orders.

Sometimes issuing specific tasks is necessary e.g. preparing a spreadsheet of KPIs or fixing a reported bug. However, most knowledge-work is best driven by outcomes, leaving the individual to decide how to achieve it (guided by the company mission and values).

Managers should not be defining “how” and need to take deliberate steps to avoid being controlling.

Very few knowledge workers bill by time increment. Engineers, programmers and academics are usually paid fixed salaries.

Lawyers are a good counter-example. Some projects can be scoped to a fixed budget but legal questions often require research which can’t be time estimated. Such things require reading, writing, editing and conference calls plus discussion with colleagues. You are paying for their ability to locate and synthesise information so as to form a technical, legal opinion.

Another example might be software engineering. There are many freelancers who will charge based on number of hours. Whether this is the correct way to charge for development is debatable. The best consultants work with customers on the basis of the value they deliver, not the amount of time spent.

The main thing holding my rate down for the early years was personal discomfort with being “worth” the types of rates which I desired to charge. I dealt with (deal with?) impostor syndrome frequently and had little context for what alchemy one needs to work to justify professional rates.

Spoiler alert: there is virtually no difference in the mechanics of work done between $100 an hour, $200 an hour, and $30k a week — all of the leveling up there is in sophistication on who you go after, what engagements you propose and deliver, and how you package things for clients.

By the end of my consulting career, I pointed to a small pile of mostly satisfied customers and other evidence that I could do the work, sketched out a “Here’s how I plan to create a couple million dollars of value for your company” in proposals, and then just announced my rate. I received rather less pushback than I expected and virtually never negotiated on rate, using a trick from Thomas Ptacek (“If a client says you’re out of the budget, start talking about the scope of the engagement to find something they can afford rather than slipping your rate.”)

Talking About Money

Understanding whether someone is performing in a salaried role is more complicated than correlating minutes tracked, lines of code written, emails sent, bugs fixed, tickets replied to, or whatever arbitrary and irrelevant metric your manager might be able to come up with.

This is ultimately about working to outcomes and not micromanaging your team.

Being deliberate

Linked to communication, this means taking extra effort to ensure everyone is in sync. Symptoms of not doing this properly include:

  • Information disseminated by rumour.
  • Inaccuracies and mixed messages leading to confusion.
  • Incorrect understanding of roles, goals, methods of working and deliverables.
  • Not knowing how the company is performing.
  • Misunderstanding what everyone is working on and how you should be contributing.
  • Inability for teams to “do the right thing” and needing to wait for specific directions and task assignments.

The most common approach to address this is to write everything down, share everything, record/transcribe meetings, be as transparent as possible and not rely on informal methods of communication.

An engineering driven approach is to think about how to set up an organisational commit log.

This is where tools start to come in and becomes crucial for scaling past a couple of people. Do you use Slack as the primary communication method? Or email? What goes into a Google Doc vs a text file in a code repo? These are all important decisions but what you do and how you do it don’t really matter. It’s making a decision that counts. Everything should have exactly one “correct place” and everyone should know where that is.

Without formally documenting how the company operates, you end up with multiple channels and information all over the place. Sharepoint, Google Docs, Slack, Confluence, email, [other random tool used by just one team].

GitLab’s handbook offers a great example of how to they this. You could adopt the same approach or create your own. The point is to just make a decision, then document it.

How do you motivate a remote team?

There are 3 key elements to motivating remote, distributed teams:

  • Meaningful work: This is one of the main advantages of smaller startups – each person is a significant part of the company and has a large impact on overall success. As the organisation grows, roles necessarily become more specialised and the meaning of the work changes. In startups, you work on a large part of something used by a small number of people. In large corporates, you work on a small part of something used by a large number of people. This is leverage. Otherwise, it can be challenging to see what impact your work has. Bored people quit.
  • Care about your people: It might sound obvious but so many companies clearly do not care about their people. This can still be a competitive advantage in how your company culture, values and approach to management bring out the best in everyone. This isn’t about providing free lunch, unlimited holiday (which actually doesn’t work) or book allowances, it comes down to the fundamentals of managing humans. Do you feel like your manager cares about you as a person? Are they regularly making time to talk about not just work-related topics? Do you trust them and feel like they have your back? Companies are not families but they are quite close it it. The relationship is always going to be professional but it is in the interests of the company not just to care about revenue and profit because it is actually the people delivering those numbers. People don’t quit bosses, they quit jobs, but who is responsible for what the job is like? The boss.
  • Communicate regularly: This is a running theme of the post so I won’t add any more here other than continuing to highlight how important it is.

Notice how these are not unique to remote, distributed companies? You can apply them to any company. It is only the implementation that is different in a remote company.

Challenges of remote, distributed working

Personal connections

The most crucial lesson I learned was about how important it was to get to know the team in-person, and for the team to get to know each other.

Whenever we hired someone new at Server Density, I could separate their time with the company into two phases: before they met everyone, and after. Although new hires usually got on with everyone fine via chat and video calls, there was a noticeable difference in their engagement once they had met the whole company.

Early in the life of my company, full meetups were more ad-hoc and less frequent. But by the time of the acquisition we were trying to get the whole company to meet every quarter.

…even the best video chat experiences can’t quite match the fidelity of a face-to-face conversation in a physical space. There is so much nuance and context to human communication that is lost, especially when conversing in groups. It’s why distributed companies (like mine, Automattic) still have in-person team meetups and all-company gatherings — technology simply can’t replace the human connection of being together. 

For Years, VR Promised to Replace the Office. Could It Really Happen Now?

There are many ways to organise these meetups, from flying the whole company away for exotic retreats to just a few days all working together in one place. What matters isn’t so much the format but getting everyone together to work, socialise and get to know each other.

We try to get together every 9 months or so to get face-time with one another, build community, and get some work done! Since our team is scattered all over the globe, we try to plan a different location for each GitLab Contribute.

Contribute is not a mandatory trip or a department offsite, nor is it a vacation or incentive trip. It’s a chance for everyone to meet fellow GitLab team-members across all departments and regions: part team building, part education, part customer interaction, and hopefully all fun.

GitLab Contribute

This can be expensive, especially as the company grows, but is definitely worth the cost for the improvements to team morale and productivity in the long run.

Remote, distributed working is not a method of cost-saving. Money not spent on an office will instead be spent on travel expenses.

Buffer wrote about how they handle their meetups and also recorded a more recent podcast that goes into detail about how it all works.

Working on a single mission

Sometimes it’s necessary to bring everyone together for a sprint towards a single goal. Such occasions should be rare but where that shared sense or purpose is really necessary, it is beneficial to all be in the same physical location.

If I was starting a new company, I would include significantly more time working together from the same location. Critical times call for working in-person.The formation of a new startup is one of those times.

This is one of the arguments behind a tweet by investor, Arianna Simpson:

In a followup podcast she elaborated on the specific scenarios where she thinks it does make sense for everyone to be in the same location.

One thing I’ve noticed about startups that end up being very successful is often there is this strong sense of shared mission and shared energy, and that I find very difficult to replicate if you have people in different locations because you don’t have the same sense of camaraderie, the same sense of “You know what, this sucks but we’re gonna stay here until 10 PM and pound it out until we’re done ’cause we’ve gotta get this release out,” or whatever it is. So that is a really difficult intangible that’s hard to replicate if you have people in different time zones or different locations. It’s hard to muster up the same level of enthusiasm.

Is remote work ‘bull***t?’

Starting a new project is a similar situation where you might want to have the team together for the kick-off, planning, design and initial implementation.

The end of a project can also be a good time to have everyone together. Celebrating the launch and completion of a major project in person can have a major, positive impact on morale.

Remote, distributed-first does not mean exclusively remote, distributed-only.

Communication

You’ll note that communication is a common theme when thinking about remote, distributed teams. This is because it is so difficult to get right.

If the problem of working in an office is the constant interruptions, then the problem of remote working is the default to using real time chat. Slack is the most common example, but it applies to any kind of instant messaging.

I’ve written in detail about the challenges of communication in general and also about chat:

I personally dislike Slack (and any real-time chat) for anything other than general, social chat or deliberate collaboration on an ongoing project e.g. triaging a customer issue with a group of people in real time, doing incident response, or backchannel chat between participants during a sales meeting. These are all instances where a small number of people come together for a specific task and for a specific period of time, after which they disband. Real time chat is an awful tool for anything else because it implies everyone must be available and responsive immediately, at all times. Information is too easily lost.

Communication inside startups – what and how

My favourite article on the challenges of chat is “Is group chat making you sweat?“. Email is my preferred method of communication because of how flexible it is – I can choose my client, layout, set up filters and decide when to check it. It’s not perfect but it really does beat instant messaging:

So what about IMs? In short… they combine the worst of both worlds while having the strength of none of them.

In theory, IMs are supposed to be fast (“instant”), but in practice typing takes enough time, to make instant messages much slower than simply talking. Ironically, the expectation of a fast response forces everyone to write short and rather poorly structured short phrases that are hard to understand, especially out of the context of current circumstances.

In theory, IM communication is persistent, but in practice because of lack of structure, intertwined topics, and insufficient metadata, what have been persisted is mostly useless. Searching through the history of a Slack room is difficult, and rarely useful.

Kill Instant Messaging

That said, we have gone through a similar transition in the past – from no email to email – and similar problems arose. Email is certainly not universally loved.

In a 2011 study published in the journal Organization Science, researchers noted that while email was widely regarded as a “growing source of stress in people’s lives,” research also suggests that it affords people “flexibility and control by enabling them to communicate from anywhere at any time.” To attempt to address this contradiction, the researchers drew on interviews from nearly a decade earlier, conducted when email itself was still ripping through American offices, and producing its own stories of relief, ambivalence, and horror. Employees’ worries will sound familiar, and in hindsight maybe not unwarranted. “Although, in theory, email’s asynchrony should have granted recipients the leeway to respond at a time that was convenient for them,” the study said, “our informants described strong cultural expectations about not keeping senders waiting.” 

Slack Wants to Replace Email. Is That What We Want?

As with any workplace culture, it is important for management to set the example of what is acceptable.

Stephen R. Barley, a professor of technology management at the University of California Santa Barbara and a co-author of the paper, remembers subjects lamenting, nearly 20 years ago, the erosion of work boundaries as symbolized and enacted by email. “I think what they’re really expressing, and most white-collar workers would never say this, is that these technologies are appropriating time at the beginning and end of days, without any kind of payment,” he said in a phone interview. “It’s an encroachment of work into other spaces in your life.” 

But with Slack, there is no perfect equivalent to inbox zero; an instant message from your boss during the day might demand not just a quick response, but an instant one; it will be up to us, but mainly our bosses, to establish what a late-night Slack message means or demands, as compared to an email, and what noise or vibration it should cause in the phone that many of us have moved closer to our beds.

Slack Wants to Replace Email. Is That What We Want?

Chat is here to stay and so this means developing a simple set of rules for where communication should take place. One approach is to delete messages after a short period of time. This makes it clear that chat is ephemeral. How often do you actually search chat logs and find something useful?

Density of expertise

Despite the major advantage of remote, distributed work being that you can hire the best people anywhere in the world, expertise still clusters in specific locations.

My conclusion is that whilst you can quite easily build out product, engineering and customer support teams within Europe and the UK, it is very difficult to do the same with early stage commercial teams. There’s the occasional highly talented individual who has done it before in the UK, but otherwise it’s almost impossible to recruit for commercial roles here. The initial sales, marketing and customer success hires need to be in the US.

Challenges of running an enterprise SaaS business in the UK

Whilst you can build these teams remotely, it may place you at a disadvantage against competition who can leverage the benefits of a specific location.

If you are building a high-growth, venture funded startup in a very competitive industry, this will matter at very specific stages of the company lifecycle. It matters who the initial founding team are. It matters when you start to build out that first commercial team. And it matters again when the company goes into scale mode, leading into an IPO.

So why are industry towns so powerful?
1. Talent base. Great engineers, sales people, product people, etc. exist all over the world. However, the highest concentration of outstanding ones are concentrated in industry towns (just as the highest concentration of great actors in the USA live in Los Angeles). Similarly, the people who know how to scale technology companies are concentrated in the tech epicenters in the graphs above. Many companies come to the SF Bay Area to fundraise post product-market fit to access this talent base and know-how.

Industry Towns – Where You Start A Company Matters

Outside of those critical turning points, and for businesses that are operating on a different growth model, then this matters less.

It will be interesting to see how GitLab progress towards their Nov 2020 IPO date but despite being an amazing example of how to run a remote, distributed company, it is interesting that the CEO is based in San Francisco.

Tips and tricks for remote, distributed teams

At Server Density, we developed a number of mechanisms to help run the remote, distributed team. These evolved over time from 2009 to the acquisition of the company in 2018.

Weekly roundup

This started as a weekly standup for the whole company to report on what they had been doing that week but it eventually changed primarily to be a social discussion.

Meetings are a necessary part of working with multiple people but using them to distribute status updates is very inefficient and expensive. Status updates can be provided in writing, asynchronously. Meetings are for discussion and debate. But when you have a team all working remotely, regularly getting together to “see” each other also has a social benefit.

It was also the only compulsory meeting during the week. Unless you were on holiday, you had to be in this call.

At one point, we stopped the meetings for a few months and experimented with just using written updates. We quickly had requests to bring back the meetings so we could talk to each other!

The format we ended with was as follows:

  1. CEO Update (~5 minutes): Any announcements or important news as well as a quick update on the key business metrics, with a more detailed writeup shared later. I also highlighted one or two things I’d seen during the week that particularly stood out – deals closed, great responses to support issues, tricky bugs fixed, etc. The important thing was to mention one or two people and some specific piece of work I thought deserved everyone knowing about.
  2. Team updates (~1 minute each): Every person would say something they found interesting during the week. Updates could include work as well as something they had read or watched. We used a random order generator to mix up the order of speakers every week. After each person had finished speaking, there was an opportunity to ask questions. This often prompted some good discussions.
  3. Anything else: An opportunity for anyone to mention something not already discussed.

With 20 people we were usually able to complete this meeting within 30 minutes. It breaks some of the rules for meetings e.g. no status updates but we were deliberate in what the format was about – hearing from your colleagues about what they had been doing.

This worked nicely for us but I’m not sure how far this would scale. Even being strict about times, people tend to go over their 1 minute. The format we used above worked for us at 20 people but is quite different from what we used at 5 and 10 people. GitLab have blogged about how they changed their weekly 40-person meeting into a 10-15 minute podcast.

Random calls with team members

Working with people in an office usually means some socialising outside of work hours where you get to know your colleagues personally. This is more difficult for remote, distributed work because “out of hours” should mean disconnecting completely!

Every week, everyone was paired up for a 30 min call with someone random from the company. There was no particular agenda so they could discuss what they liked. Even on a small team, this helped connect team members who might rarely (or never) talk to each other.

This scales well even with large companies because as the team grows, you become more siloed in who you interact with on a daily basis.

What have you been working on?

Knowing what your colleagues are doing is one of the challenges of communication made more difficult by remote, distributed working. One fix we tried was a regular prompt for people to answer the question: “what have you been working on [today | this week | recently]?”

We used Basecamp to automatically ask this every day (and also tried it more infrequently e.g. a couple of times per week). It was never compulsory and different people wrote at different frequencies to suit them, and when they had done something worth broadcasting to the whole company. There are Slack bots available to do this too.

The important thing was that it was voluntary and non-specific. Anyone could write whatever they liked. Tracking of bugs, epics, tasks, stories and other precise units of work is for other tools.

This worked for a time but usage declined over time to the point where not enough people were using it to make it worth continuing. It was replaced by the weekly roundup (see above).

However, one thing we did find was that there was a weird political element to “likes” of posts. Feedback I received indicated that there was an almost addictive aspect of checking posts for likes, seeing who had liked them and whether your own post had more likes than others.

Not having to think about likes having ditched Facebook and Instagram is a nice side benefit I’ve noticed. It definitely influences your behaviour in negative ways. Basecamp discovered this themselves, replacing the feature with Boosts. A good move.

Final thoughts

There is no “one true way” of running a company that will automatically deliver total productivity in every situation. Whether you run entirely in an office, entirely remotely or a mixture of both, every organisation needs properly thought through processes.

Those processes will differ as the company grows. Smaller organisations need minimal overhead to remain agile and able to iterate quickly. Larger firms need to be able to coordinate multiple teams working on various projects. But no process at all is just as much a decision about working practices (probably in an environment of chaos) as writing a detailed handbook would be.

The challenge of remote working is that it needs a much more deliberate approach to running things. There are fewer organisations using remote, distributed working methods. Not as much shared knowledge exists. But that is changing.

Companies that do not allow remote and/or do not properly consider how to effectively incorporate remote working into the long-term are going to fall behind and fail to attract the talented people that are crucial to success. The form that takes will be different in every case so it is important to avoid “remote purism”. It is clear when things aren’t working but there are many ways to make it work well.

Should you encourage your team to write their own blog?

When I started my company, Server Density, the first thing that went public was a blog post. The majority of the posts were about our technical challenges, with only occasional product/company related articles. I wrote a post almost every single week and after 9 years built the readership to +100,000 monthly uniques and +25,000 email subscribers. All organically and without paid promotion.

Of course, this was my own company so my identity was linked to the business. There’s an argument to say you should put 100% of your effort into building the brand of your business, not your personal brand.

Perhaps this is true for founders, but is it applicable to non-founder team members, too?

Employees have very different incentives from founders. They might be aligned with the success of the venture (if they’re early) but really they are negotiating an exchange of time and expertise for compensation (cash, stock, etc). They can leave at any point and when it’s time to move on, they will need to pitch themselves to a new employer. Their tangible output might be owned by the company but what about lessons learned?

When hiring someone, their past experience is relevant. Skills and capabilities are the usual filters for any hiring process, but what about personal brand?

There are many great examples of high profile “internet people” who have built up a personal brand through writing. Some of those now work for well known organisations. Some run their own companies or consultancies. Some are investors and some even employ some of the others!

Being “internet famous” isn’t even that important, but writing about and sharing experiences is. Note how they all do this through their own website as well their job. It’s their own place on the internet independent of whatever they might be doing now.

There are plenty of high profile CEOs and founders who do not blog or write but there are just as many who do. The same applies to non-founder team members – writing about experiences and sharing stories is valuable, not just for the individual to get experience writing but for everyone to benefit.

It’s not reasonable for a company to expect to control 100% of employee output. Even founders aren’t really expected to do this. Very few run their business for life. Startups fail. They are acquired. Founders move on. You wouldn’t invest all of your money into a single project, so why do the same with your time? A healthy life balance means side projects, multiple investments, hobbies and other interests.

Indeed, there is tangible benefit to the company in actively encouraging team members to write, even on company time. Writing is a skill that needs to be developed. Doing so will have major benefits for communication internally. Not only that, gradually building a brand is valuable not just for the individual but benefits their employer too.

Everyone should have their own website with their own content. Tweeting is one outlet but the benefits compound from writing on a domain you control. Whether you are an employee or a founder, it should be encouraged. It’s not difficult to start.

Writing as a core company value

As communication is the hardest problem facing any organisation, writing well should be one of the core values for every company.

When I hired for engineers at my company, Server Density (now acquired by StackPath), the first part of the selection process was a writing exercise. We asked every candidate to spend no more than 1 hour researching the answer to a simple question such as “Compare and contrast MySQL and MongoDB”.

The purpose was not necessarily to test their technical skills but to assess their written communication. Errors with spelling or grammar meant immediate rejection but the ability to convey complex ideas was the main focus of the assessment.

Although anecdotal, I found there was a strong correlation between writing capability and engineering skill. Showing that you take the time to proof-read your work is a good indicator that you will put the same care into your day job. This is especially important for coding.

Chris Laffra on LinkedIn

I would now apply this to all roles, not just engineering. The ability to convey ideas accurately, precisely and in such a way that others can understand is a crucial skill.

Examples from successful companies

Some of the most successful companies follow this sentiment.

Amazon famously uses 6-page narratives in order to propose new ideas and read in silence at the beginning of each meeting. Not only does the exercise force everyone to consider the details, it also ensures a significant amount of time and effort goes into the idea in the first place.

We don’t do PowerPoint (or any other slide-oriented) presentations at Amazon. Instead, we write narratively structured six-page memos. We silently read one at the beginning of each meeting in a kind of “study hall.” Not surprisingly, the quality of these memos varies widely. Some have the clarity of angels singing. They are brilliant and thoughtful and set up the meeting for high-quality discussion…

…The great memos are written and re-written, shared with colleagues who are asked to improve the work, set aside for a couple of days, and then edited again with a fresh mind. They simply can’t be done in a day or two.

Amazon 1997 Letter to Shareholders

This results in Amazon consistently operating at a high level of efficiency:

Imagine for a moment that you could go into a meeting and everyone in the meeting would have very deep context on the topic you’re going to discuss.  They would be well-versed in the critical data for your business.  Imagine if everyone understood the core tenets you operate by and internalized how you’re applying them to your decisions.

How great would it be not to be constantly interrupted by clarifying questions?  How great would it be not to have the decisions in the meeting based on the social networking advocacy that happened before the meeting?  How great would it be if executives deeply understood your organization from your perspective before asserting they know better how to do it?  How great would it be to be able to review the core data going into a decision rather than have someone summarize it and assert that correlation is causality without revealing their work?

This is what meetings are like at Amazon and it is magical.

The Beauty of Amazon’s 6-Pager, Brad Porter

It’s not just Amazon. If you have ever used any Apple apps on iOS or macOS, you will have noticed how consistent they are in design and style. This is because they have strict, detailed Human Interface Guidelines for all their platforms.

Design consistency is an indicator of quality and the same applies to writing. Apple has a style guide which is just as relevant as the Interface Guidelines:

The Apple Style Guide provides editorial guidelines for text in Apple instructional materials, technical documentation, reference information, training programs, and user interfaces. The intent of these guidelines is to help maintain a consistent voice in Apple materials.

Writers, editors, and developers can use this document as a guide to writing style, usage, and Apple product terminology. Writers and editors should thoroughly review the guide to become familiar with the range of issues involved in creating high-quality, readable, and consistent materials. Apple developers and third-party developers should follow these guidelines for user-facing text.

It’s not just the large, successful companies that take this approach. It applies to smaller, successful businesses as well. Basecamp, a ~40 person company, uses a very similar written pitch to propose and discuss ideas for feature development:

Why don’t we pitch in person? For a few reasons:

1. We feel it’s better to write something up completely. This forces the floor — the person who’s making the pitch can’t be interrupted. They are guaranteed to be able to present their story completely, exactly as they want.

2. Further, we believe writing things out makes you consider them at a deeper level.

3. We’re big believers in asynchronous communication — write it down now and other people can absorb it later when they’re ready. Real-time communication in person or virtually forces synchronization of schedules which is highly inefficient.

4. And last, when it’s posted to Basecamp as a message, all feedback and follow up questions are automatically attached to the original post. This keeps all communication about the pitch centralized in one place on one page so everyone has access to the same story. One version of the truth.

Pitching an idea

Indeed, the importance of writing skills is something that Basecamp consider as part of their hiring process:

Our top hiring criteria — in addition to having the skills to do the job — is, are you a great writer? You have to be a great writer to work here, in every single position, because the majority of our communication is written, primarily because a lot of us work remotely but also because writing is quieter. And we like long-form writing where people really think through an idea and present it.

Jason Fried of Basecamp on the Importance of Writing Skills

Writing really is crucial to successful communication, and success in general. There are too many examples of poor writing. Why not make it one of your core values?

Parachuting tasks because available time is fixed

There are 24 hours in a day. 8 of those are the standard length of the working day, of which an even shorter total make up the time we can spend actually doing focused work.

The available time does not change. You might be able to develop a system to increase the number of focus hours but you cannot increase the number of available hours.

If you read that paragraph in isolation then it is an obvious fact. Of course you can’t increase the available hours.

So why is this ignored when we’re actually planning work and progressing through a task list?

An interrupt-driven culture whereby new tasks appear without reducing the pool of tasks the team are expect to complete is something I see on a regular basis across many companies.

This is especially prevalent in engineering teams. A sprint or development cycle will begin with a set number of tasks, and then mid-cycle new tasks will be dropped in due to critical bugs, customer complaints or other reasons. This doesn’t necessarily have to be an issue – it becomes a problem when those tasks are forced in without removing other tasks. The available time is the same but suddenly there are now more things to do.

The results of this include:

  • Deadlines being missed – the work still has to be done, it’s just there is now more of it than when the original time estimate was discussed. Without adjustment then it is inevitable that everything will now take more time.
  • Existing work taking longer than expected – if a critical bug comes in that needs to be dealt with immediately, that will cause a context switch that has an impact on individual or team productivity.
  • Multiple teams being impacted – the original work plan may have external dependencies from other teams e.g. a marketing campaign launch.
  • Requests for overtime – the team may be expected to work longer and later, despite this not actually correlating with productivity.

At my company, Server Density, we eventually implemented a solution to this we called “parachutes”.

  • Development cycles were a fixed length.
  • All the tasks to be completed were scoped in tickets each of a fixed length.
  • Every member of the team had a known level of productivity i.e. how many tasks they could complete in a cycle.
  • This gave us a total known capacity for the cycle, which allowed us to slot in the tickets that would be completed in the cycle.
  • We discussed which tickets needed to be done for each cycle, with stakeholders across the business sponsoring the tickets they prioritised, everyone debating the priorities.
  • As a result, each ticket had a priority relative to the other tickets in the cycle.
  • If an unexpected task came in, it was first triaged to scope and size it. Once agreed that it needed to be dealt with urgently, the lowest priority ticket in the current cycle was “kicked out” and this new ticket “parachuted” in to replace it.

This resulted in knowing what we were sacrificing to get the new ticket solved, with the stakeholders for the kicked-out ticket being involved in the discussion about whether to parachute it. The cycle remained a fixed length and meant that we had a good level of certainty about what was going to be completed and when.

Dropping a new task into an existing cycle always has a cost. If you don’t acknowledge that time is fixed then the cost ends up being hidden under overtime, surprise missed deadlines and frustration from supporting teams. It is better to explicitly incur that cost in a defined way by sacrificing some other known task than to keep it hidden behind a bad process.

Parachuting tasks is always frustrating, especially for the owners of the kicked out ticket. But the key is visibility and shared decision-making. Without this, teams continue to maintain a fiction that everything will still be completed on time with the same effort, until it isn’t. That’s significantly more frustrating.

Communication inside startups – what and how

Once you have more than 1 person in a company, I think communication is the biggest challenge. I don’t think it matters whether you’re working 100% remotely or have everyone in the same office, the challenge of communication remains similarly difficult.

The challenge and how to solve it changes depending on the size of the company but there are common questions that everyone should have a concise and precise answer for at all times:

  • What is the overall mission of the company? Why are we doing what we’re doing? How is the company tracking against that mission? What is going well and what isn’t?
  • What are the company values? How should we evaluate the ways of achieving our mission?
  • How do we work and collaborate together?
  • What is expected of me today, this week, this month and this year? How does that link to the overall mission? How am I tracking against those goals? What is going well and what isn’t?
  • What is the rest of the company doing?

Practicing complete transparency is the best way to smooth the flow of information to different people across the company, whether they are employees, executives, board members or investors.

There are very few classes of information which may not benefit from full sharing – disciplinary and HR matters, complaints and perhaps individual compensation packages (but having a consistent, open set of salary bands makes up for that).


So what do these different areas involve? What are the key things that need to be communicated?

What is the overall mission of the company?

Later in the life of a company it should be obvious what the product is, but it may be harder to explain the why: Why are we building this? Why have we built that feature? What is the big problem we are trying to solve over the next 10-20 years?

This is crucial to understand because every other aspect of the company flows from this single mission. Everyone needs to know the mission because otherwise they cannot understand why the work they are doing is important to achieving that mission.

The best mission statements are short but specific. Easy to remember and obvious, but hard to achieve and involve playing the long game.

One of my favourites is Tesla:

…the overarching purpose of Tesla Motors (and the reason I am funding the company) is to help expedite the move from a mine-and-burn hydrocarbon economy towards a solar electric economy, which I believe to be the primary, but not exclusive, sustainable solution.

Secret Tesla Motors Master Plan

And SpaceX:

The company was founded in 2002 to revolutionize space technology, with the ultimate goal of enabling people to live on other planets.

About SpaceX

These are both very clear about the problem and what the company is trying to do.

But the mission doesn’t need to be as big picture or ambitious as what Elon Musk is trying to achieve – it’s just as important to have a clear mission if your goal is to build a small but successfully sustainable company.

Without a clear, written statement, you cannot align the rest of the company goals, understand how to prioritise what to do next and will certainly find it difficult to hire and retain your team.

Even simple mission statements require constant communication. The whole company need to know not just what the company is working towards but how things are tracking against that goal. With real numbers and commentary on both positive and negative progress, the mission will be reinforced.

This means having regular reporting of KPIs to the whole company. You are probably doing this for investors and your board already so this format can be reused for the rest of the company.

Information about company performance either gets out through actively sharing the real numbers or it is distributed through rumour and gossip. There is no option where information doesn’t travel, so why would you not want to be in full control of the narrative and detail?

Doing this builds trust in good times so that you can call on the team when you enter more challenging times. With everyone aware of the same numbers, everyone has the same context for making decisions, understanding why decisions were made and making their contribution as valuable as possible.

What are the company values?

The company mission is always very high level – it describes “what” and “why” but doesn’t really discuss “how”.

I used to think that the idea of “company values” was a cliché but that is only true if they are vague and written in generic management speak. If the company values don’t prescribe specific behaviour then they aren’t particularly useful.

Netflix has perhaps the most famous description of its culture and values, from the culture deck CEO Reed Hastings published almost 10 years ago:

More recently, these have been written out as a set of high level principles which they then break down into specific values.

Like all great companies, we strive to hire the best and we value integrity, excellence, respect, inclusivity, and collaboration. What is special about Netflix, though, is how much we:

1. encourage independent decision-making by employees
2. share information openly, broadly, and deliberately
3. are extraordinarily candid with each other
4. keep only our highly effective people
5. avoid rules

Our core philosophy is people over process. 

Netflix Culture

Whether you use the term “values” or not isn’t the point – it is about having a written description of how you approach things. They describe how the team behaves, help you recruit new people who match those values and evaluate the existing team as to whether they are meeting those expectations.

Note the keyword: written.

The task of writing down the values is a good thought exercise in itself but then they can also be referenced and modified over time. If they’re not written down then there is too much of an opportunity for digression, forgetting key values or people just misremembering.

Another example is GitLab, who are unusually transparent in everything they do. Indeed, this is one of their core values.

GitLab’s six values are Collaboration, Results, Efficiency, Diversity, Iteration, and Transparency, and together they spell the CREDIT we give each other by assuming good intent.

GitLab Values

The important thing is making them actionable, for example under “Results” they explain:

Measure results not hours
We care about what you achieve; the code you shipped, the user you made happy, and the team member you helped. Someone who took the afternoon off shouldn’t feel like they did something wrong. You don’t have to defend how you spend your day. We trust team members to do the right thing instead of having rigid rules. Do not incite competition by proclaiming how many hours you worked yesterday. If you are working too long hours talk to your manager to discuss solutions.

GitLab Values: Results

Making the values actionable and measurable is what transforms them from high level clichés into a true set of operating principles for the company.

How do we work and collaborate together?

Do you message someone directly on Slack? Do you send an email with an announcement? Is it expected that you will engage in discussion in the comments on a Google Doc? Does documentation go in Confluence?

These are questions everyone has when they first join a company and so should be part of the standard operating system of the business. Everyone should be using the same tools and processes and there should be exactly one way of doing things. You know this is going wrong when you see such behaviour as:

  • Missing crucial information because it was sent as a Slack message late at night in your timezone and there were hundreds of other messages afterwards so you didn’t see it.
  • Bugs getting reported through anecdote, mixing multiple issues that then get fixed in an ad-hoc way without tracking the resolution, often resulting in duplicated work as other people try to reproduce an issue that has already been fixed.
  • People don’t check their email often because they don’t get many messages and instead work in real time chat all day, meaning they miss meeting notifications or an important customer message.
  • Documentation is written in Confluence by one team but another team is using Sharepoint to work on a Word doc.
  • People are mentioned in document comments but the answer is provided through another channel and the original comment is left open and unresolved.
  • Information and documentation exists in one place but it isn’t easily searchable so people end up asking individuals for work that has already been done.
  • Decisions get made without you being aware and you only find out through luck or because someone happens to mention it in passing.

Discipline is easier when the company is small and just getting started – that’s the best opportunity to set the rules and processes, and tell new hires how to do things. When there is a lack of direction, teams will end up using their own tools and systems because it’s better for them but to the detriment of the overall efficiency of the company.

Sometimes changes to working processes are needed, especially when a new tool is discovered that makes a difference to the working practices of the organisation. But adopting something new should be a deliberate evaluation and decision process.

If teams end up spinning off their own processes and tools, that is an indication of poor direction and/or failure of an existing approach. Dissatisfaction with the status quo should trigger a review and debate rather than a secret team/deployment of a new vendor product.

I personally dislike Slack (and any real-time chat) for anything other than general, social chat or deliberate collaboration on an ongoing project e.g. triaging a customer issue with a group of people in real time, doing incident response, or backchannel chat between participants during a sales meeting. These are all instances where a small number of people come together for a specific task and for a specific period of time, after which they disband. Real time chat is an awful tool for anything else because it implies everyone must be available and responsive immediately, at all times. Information is too easily lost.

My personal preference is defaulting to asynchronous communication via email, where I can set up my own workflow, control notifications and respond in my own time and in appropriate detail. Chat has its uses but I consider it (and notifications in general) highly damaging for any kind of focused knowledge work like writing or coding. This is not a unique view:

What we’ve learned is that group chat used sparingly in a few very specific situations makes a lot of sense. What makes a lot less sense is chat as the primary, default method of communication inside an organization. A slice, yes. The whole pie, no. All sorts of eventual bad happens when a company begins thinking one-line-at-a-time most of the time.

Is group chat making you sweat, Signal vs Noise

With 10 years of remote working experience with the majority of that spent with chat as a core communication tool, my preferred working style now is to assume that everything is asynchronous by default, with the ability to do urgent interrupts rarely.

Real-time sometimes, asynchronous most of the time.

At Basecamp our perfect-world rule of thumb is “real-time sometimes, asynchronous most of the time”.

Basically, right now should be the exception, not the rule. That creates space and attention for the things that really do have to be discussed right now, and allows everything else to be thoroughly discussed asynchronously and thoughtfully over time.

Is group chat making you sweat, Signal vs Noise

Real time chat like Slack has its place, but should be limited by both configuration and convention. For example:

1. Slack is to be used for informal communication only. Only 90 days of activity will be retained.

2. If you use Slack, please avoid private messages. Private messages discourage collaboration. You might actually be contacting the wrong person and they cannot easily redirect you to the right person. If the person is unavailable at the moment, it is less efficient because other people cannot jump in and help. Use a public channel and mention the person or group you want to reach. This ensures it is easy for other people to chime in, involve other people if needed, and learn from whatever is discussed.


6. Slack messages should be considered asynchronous communication and you should not expect an instantaneous response; you have no idea what the other person is doing.

Slack, GitLab Handbook

Limiting retention forces everyone into the mode of the chat being ephemeral and minimising private messaging helps encourage transparency, avoiding interrupting specific people. And if it really does need to be synchronous, a video/audio call (which can then be recorded, transcribed and stored in a searchable format) is still probably better.

What is expected of me?

At this point we are down to the day to day activities of individual team members. If the above items are done well, each person will understand what the company is doing, what guides their approach and the key tools and processes they should be using. That leaves the question: what should I actually be doing?

The worst kind of work is where you are simply following orders. Sometimes specific tasks will be assigned e.g. preparing a spreadsheet of KPIs or fixing a reported bug. However, most work should be driven by outcomes, leaving the individual to decide how they achieve it guided by the mission, values and tools.

There are plenty of systems that can be used to decide what should be worked on, with the most popular (at least in startups and Silicon Valley) being OKRs. I’m actually not a big fan of them and couldn’t get them to work well at my company, but I do subscribe to some of the key principles of OKRs, specifically:

“The ability to track results on a quantitative basis.” Key results are not general or subjective actions you plan to take. They should always include numbers to make it clear how much has been achieved. For example, if Mary’s objective is to improve her sales prospecting skills, one key result might be to spend two hours a week shadowing Jennifer, the team member who demonstrates the most prospecting success.

How to Make OKRs Actually Work at Your Startup

Whatever system you use, the key is that people know what they should be doing and how it links to the overall company mission. OKRs are one way to achieve this but need to be implemented with all the other things discussed above – they won’t just magically solve a communication problem if there is no understanding of the company mission, values and processes. The team should not only know what they’re supposed to be working on and why, but be able to explain it to others:

Following the HR rep’s suggestion, I start to ask questions. Actually, a single two-part question: What are you doing, and Why?

For the What, I rely on a variant of my old I Can’t Be Stupid gambit: If I don’t understand what you’re saying, either you don’t actually know what you’re talking about, or you’re withholding something. As for the Why, please don’t say you’re just following orders — I know you have a mind of your own; and don’t hide behind “marketing demanded it” as if you respect marketing. Tell me how your work serves the common purpose. Does it improve performance, reduce cost, increase reliability, forge a killer feature? If you can’t tell me, perhaps you should go back to your desk, gather your thoughts, and come up with some answers that make me feel smarter and safer.

50 Years In Tech. Part 12: Cupertino Culture Trouble

What is the rest of the company doing?

With all of the above implemented, everyone should have a good idea of where the company is going and what is being worked on, at least at a high level. Knowing what is actually happening on a day to day basis is easier in small companies but as the team grows and the product becomes more complex (possibly with multiple products), it becomes harder to know who everyone is and what they’re individually doing.

This essentially comes down to two things:

  1. Transparency: are the systems, tools and processes open enough so that anyone can look at any document or ticket if they wish (subject to certain restrictions around confidentiality – GitLab again have good guidelines on what is not default-public)?
  2. Real communication: is there regular discussion from the executive team explaining what is going on throughout the business – whether that is highlighting big (and small) wins, releases, problems or just regular reporting on activities.

There is also a question of relevance. In a company of 10 people it makes sense for everyone to know everything, and to be specifically told about all activities. If you have 10,000 people then the level of detail needs to change. Information should still be available if individuals wish to look it up, they just might not be told the complete detail on a regular basis.

Most companies do a poor job at this, especially the bigger ones. Coordination across teams within a single department is often frustrating enough that trying to get all teams fully aware is a big challenge. That doesn’t mean it can’t be done.

One good example where this is a regular problem is with the selection of technology and building internal systems. Large companies, organisations and especially the public sector suffer from duplication of systems which can only really be solved through proper value chain mapping.

Wardley Maps are a great way to understand the landscape, something which most companies are awful at.

This doesn’t just stop at knowing who is building what but actually goes into the entire company strategy. That’s an entirely separate topic, though. Read the book on Wardley Maps for the real detail.

So how do you achieve this?

Establishing a regular communication and reporting cadence is a standard part of running a public company – financials and progress get announced in an expected format on an expected timeline. If public companies can do this on a quarterly basis, there’s no reason why smaller companies can’t do it too, and likely much more regularly.

At my company, I used to send out monthly updates to all my investors via email. This tied in with my monthly board meetings and included details about all KPIs, an analysis of the results and what the plans were for the next month. I sent out the exact same report to all team members internally as well.

I also ran weekly whole company meetings where everyone had a minute to talk about something interesting they had done during the week, and as the CEO I commented on positive developments, good work I’d seen and provided an update on challenges, problems or anything else that was relevant.

If I was implementing this again I would think about the following:

  • Reporting KPIs on a weekly basis via email and highlighting the top 2 metrics in the weekly meeting (likely revenue numbers). A good real-world example of implementing a system of reporting is described in The Founder’s Guide to Discipline: Lessons from Front’s Mathilde Collin, particularly around regular emails and meetings to highlight progress.
  • Building an internal KPIs dashboard with real time data that everyone could access.
  • Recording and transcribing all meetings so they are easily searchable and reference-able in future, then emailing this out to the entire company (for the whole company meetings, and having a good index for all other meetings).
  • Having all individual goals and measures for the “What is expected of me” section public on some kind of internal profile directory. This helps to answer the question of “What is the rest of the company doing?” and means people can directly link to their goals when helping others prioritise work. If everyone knows what you’re working on and why, they can figure out how their request can help achieve the goals of both parties.
  • Regularly discuss all aspects of the company mission, values and processes to get input on what is working and what isn’t. Just because you decide on an approach when the company is founded does not mean that you have to stick to it.
  • Being more deliberate about having a single source of truth for all documentation, values and the overall company handbook from the beginning of the founding of the company.

Communication is not static. The best way to do something today might change tomorrow. You have to repeat things and ask obvious questions. It can be difficult to know how well communication is actually working but you can certainly tell when it’s poor. Things are often getting done despite the communication overhead. People are guessing what they should be doing and there is a low level buzz about how well the company is truly doing. Nobody has any guiding principles helping them decide what they should be doing and there is often a lot of sudden urgency around arbitrary deadlines like working in a feature factory.

This is a big challenge all companies face, and it is one that is never solved. Whether you have an extreme level of secrecy and control like Apple or at the other side of the spectrum like GitLab, I think monitoring communication and actively working on improving it should be a top priority for the entire executive team.

Further reading

Influence is the product manger’s best tool

The product organisation is unusual in that it achieves its goals through influencing other parts of the company rather than through a managerial relationship with individuals that sit with its direct reporting line.

A typical product group will have a leader reporting into the CEO who then has product managers reporting into them. The number and complexity of those products will determine how deep that org chart is but the people actually doing the implementation are usually part of another reporting group e.g. engineers report up to the VP, Engineering (or perhaps the CTO).

This is of course a generalisation, but it’s rare to have engineers reporting into the product manager, for example. The same applies to other aspects of the company – marketing reports though the VP, Marketing or CMO. Sales is led by the VP, Sales or CRO. And so on.

Despite this, the product manager is working very closely with all these areas of the organisation. They define large parts of what the work is (but not the how) but are unlikely to be directly assigning tasks or “managing” the team they are working with.

This means that good product managers must rely on their powers of influence and persuasion to make a case and explain why something should be implemented. They have no direct reporting authority or rank, so must take care to build up respect and authority by the quality of their work.

In practical terms, this means:

  • Understanding the customer better than anyone else. Sales might know best about what prospects are looking for before they become customers, and support might understand key pain points in product usage for existing customers, but it is the product manager who should have the best understanding of the customer as a whole, across their entire lifecycle using the product. Product manager activities should involve bring on sales calls, visiting customers on-site, participating in UX reviews, dipping into support tickets and chats, running feedback sessions with sales and support, and triaging customer feedback.
  • Understanding the market better than anyone else. Marketing probably know how to generate the widest reach in the most targeted way and finance truly understand how margins should be optimised, but the it is the job of the product manager to understand why one marketing idea might be better than another based on an understanding of how customers consider alternatives, buy and then use the product. And to give finance the input they need about competitive pricing and operational costs. Product manager activities should involve collaborating on the go-to-market plan, reviewing funnel statistics, engaging with analysts, understanding infrastructure costs, testing competitor products, positioning product pricing packages against features and customer personas, discussing deal win/loss reasons with sales and understanding churn reasons.
  • Communicating across the organisation. Communication is the biggest challenge in any company, and only gets more difficult as it grows. There are so many levels of granularity in delivering a successful product so it is crucial that everyone understands the long term vision and how that translates into each feature release. Product manager activities should involve discussing the strategic plan with the CEO/executive team and with the wider product manager group, testing out pitches with real customers (can you articulate how you got to where you are today, and where the product is going?), sending out regular product updates to customers (newsletters) and internal groups (changelogs), building training decks for sales/support, then delivering the training itself, writing up roadmaps, converting feedback into product specs, helping debate the prioritisation of key initiatives and explaining to sales/support why something is going to be done (or not done).

Product management truly is the job with the most cross-company influence but is least able to rely on rank and position to derive authority. It means it is one of the more difficult roles to start from scratch in, but has the most opportunity to build influence and authority through results.

Product managers are not responsible for “how”

Product managers are responsible for two aspects of their product:

  • The product itself: what should we build, why, and when? This involves working with sales (deal win/loss reasons), support (patterns in support requests), engineering (product development, testing and release) and operations (supporting the infrastructure behind the product).
  • The go-to-market approach: what should we do to raise awareness and generate new revenue? This involves working with sales (training on new releases, working on sales collateral, being on calls with prospects) and marketing (influencing product marketing campaigns, attending conferences, creating content).

There are nuances within these categories e.g. if the release is about improving a difficult area of the product interface, the go-to-market approach may involve only existing customers and churn reduction rather than new revenue. Or a release may focus on reducing specific alerts and incidents for the on-call team, rather than direct customer functionality. But much of the day-to-day of a product manager will revolve around these two aspects.

Amongst all the things that the product manager is responsible for coordinating it is important to note the key aspect they are not deciding: how.

Whether you agree or disagree that the product manager is like the CEO of the product, once you get past the early product prototypes and have the scope to hire dedicated product managers, their job does not involve deciding how things should be done.

At the point when your team is big enough to include product management, it will also include specialists in engineering, operations, sales and marketing. They will have taken over from the generalist founders and should be much better than the original team who hired them. That’s why you hired them in the first place!

The same applies to product managers. Their job is to get into the details of what should be done and when, coordinating all the resources and people to hit product goals (revenue, launch, NPS, churn, etc). Their job is to define the outcome everyone is aiming for, not to determine how it should be done.

Designers decide how to build the best user experience.

Engineering decides how to implement a feature.

Operations decides how code should be deployed.

Marketing decides how to spend their budget.

Sales decides how to engage potential customers.

Support decides how to deal with inbound customer requests.

The “how” of product is deciding how the product delivers on the business goals through the teams that the product manager then has to coordinate. This involves “what”, “why” and “when”. It includes providing ideas and input into helping the other teams figure out their “how”, but that decision rests with the team actually doing the work.