Setting team KRs: A guide for first-timers

In December 2020, the leadership team of Fishtown Analytics had an offsite. While their calendars show that they met over Zoom, legend has it that they in fact went up a mountain, and returned with two stone tablets. On these tablets were inscribed the company’s commandments for 2021 — these commandments became known in our company as the Objectives and Key Results (OKRs).

Upon delivering these OKRs to the people (the rest of the company), each nation (team) was tasked with translating these commandments into their own Team Key Results.

Okay, okay, I’ll stop with the bad joke. The reality is this: our leadership team did a fantastic job of setting company-wide OKRs for 2021. For many of us, translating these company OKRs into team KRs was something we had never done before.

At times setting team KRs became an overwhelming task for me – any time I googled what a good KR looked like, I ended up reading SEO fodder that wasn’t helpful. I would spend an hour staring at the blinking cursor in our Google Doc and had no idea what to write. Slowly, slowly, our team chipped away at the task. We would write a KR, iterate on it, and then write another one. We built the KR-writing muscle, and eventually produced a set of team KRs that I’m extremely proud of.

While working through this process I feel like I learned so much, and wanted to distill some of what I learned in case others find it useful when they are next up for writing team KRs.

Please note: I am not the foremost expert on KRs, and one day I’ll hopefully look back at this post and realize I got some things wrong. My view here is that experts, and people that find a task easy, aren’t always the best person to coach that task — they may not remember the struggle of completing a task for the first time, and sometimes their advice amounts to “I dunno, you just do it” (which is completely unhelpful and alienates the person struggling with this task further). Or they write entire books on the topic, and really I’d just prefer to read a 5 minute post.

In my opinion, any time in your career that you go from “I don’t know how to do this / I really struggle with this task” to “I think I know how to do this” — write about that inflection point! It may help unstick someone else going through that struggle. Anyway, enough with the general career advice, let’s get into KRs!

Context: The company-wide OKRs

For hopefully obvious reasons, I’m going to have to allude to what the company’s OKRs are for the year, rather than write them out explicitly. Know this: our OKRs were really well-written. They are opinionated, they give us a sense of direction, and they also declare what we won’t be doing for the year.

It’s probably worth saying this: if you don’t have good company-wide OKRs, it’s going to make it difficult to write good team KRs.

What makes a good team KR?

Here are the criteria that I felt defined a good KR:

The key result is a result, not a task (another framing: it’s an outcome, not an input)
The key result is measurable, both in the sense of we can clearly define what success looks like, and we have the data points available to us
The key result may include a component that measures magnitude, and one that controls quality
The key result does not pre-suppose the exact program of works we will do (and in fact leaves room for creative solutions)
The key result states assumptions about why this team KR affects the top-level KR
The key result target should be informed by analysis. Note: this is not always possible, sometimes you just need to do a “stake in the ground” number when it’s a net new task!

While we didn’t achieve this on all of our team KRs (we were learning!), this checklist really set apart the great KRs from the good ones.

Example 1: The Getting Started Tutorial

For those that don’t know — I wrote the Getting Started Tutorial for dbt, the software my company creates. We knew we wanted to refresh our Getting Started Tutorial, and also had a good instinct that it would help us achieve our company KR related to the number of teams using our software.

Let’s check out how this KR evolved:

Refresh the Getting Started Tutorial

At first we wrote most of our ideas like this, but in hindsight we can already see why this KR isn’t great — first and foremost, this is a task (or an input) that we will do, not a result (or outcome). Further, it’s not measurable — what does “refresh” mean / how do we know when it’s done? We can’t measure this! 😖

Have X users complete the Getting Started Tutorial

Why does this team KR affect our company KR (# of teams using dbt)?
The GS tutorial helps people get over their inertia of starting with our software

Why this target number?
<TODO: check how many people did the GS tutorial last year, 2x it>

Already, we’ve flipped this from “what work can we do?” to “what is our result?”. By flipping this we can start thinking about more tactics than just refreshing the content (though that will be a big one), for example, maybe we want to increase the visibility of the tutorial, or advertise the tutorial on NYC subways (jk).

Often, we would use a placeholder X so that we could align on the the “shape” of the KR, and in parallel would do the analysis to help us figure out the right number.

I also got in the habit of writing out the above two “why” questions and filling them in with my assumptions as I continued to develop the KR, more on this below.

Have X users complete the Getting Started Tutorial in less than 1 hour

(As above)

Ah, now we can see that we’ve got an aspect that controls the quality of this key result — a one hour mark. We’ll have to balance these two things when doing work in pursuit of this KR, and can open ourselves up to even more interesting, and cross-functional tactics (e.g. “what if the GS tutorial was embedded in the product?”)

Have 1000 users complete the Getting Started Tutorial in less than 1 hour
(Exact numbers have been replaced in this writeup)

What’s the Getting Started Tutorial?
The focus of the Getting Started Tutorial is to get people using the main features of dbt as quickly as possible, while providing upgrade paths to learn more.

The GS tutorial should be able to be completed within 1 hour. Currently, our Learn students report spending 2-3 hours completing this tutorial, while our experienced community members report being able to do it in 30 minutes.

Why does this team KR affect our company KR (# of teams using dbt)?
Getting started is often the biggest barrier to adopting dbt. By making it easier to build a sample dbt project, users will feel empowered to start working on their own project.

Why this target number?
We estimate that 500 people completed the GS tutorial last year. This estimate is based on video view counts – as of Jan 2021, the final video in the tutorial, “Document your models”, has received 552 views

Based on this estimate of completions, 1000 feels like a good stretch goal.

Oooooh yeah. A KR that ticks all the boxes, and is ready for feedback and iteration! (More on this later)

Example 2: Slack health

Our team also tries to keep a 12.5k Slack community healthy.

At first we wrote a lot of KRs in here along the lines of:

refresh the onboarding flow
surface channels back to the community on a weekly cadence
interact in X channels a week

We’re still hammering this one out, but our KR might look a bit more like this:

Ensure 20% of new users post in a non-default channel within their first 30 days of joining the workspace
Why does this team KR affect our company KR (weekly active community members)?
Having new users join won’t affect our weekly active members rate unless they also participate. It’s wild to see how low these “conversion” rates actually are!

Our theory is that posting in a “non-default” channel is a better marker of long-term retention – it shows that someone was directed to a channel of interest, and felt compelled to participate in the conversation

Why this target number?
Currently, only around 10% of new users post in a non-default channel within their first 30 days (time-capped to avoid the problem of shifting metrics). We’re looking to double that — this is an aggressive target from a baseline perspective, but also because it reverses an overall declining trend

This KR completely flips the script: we’ve got a metric that we can measure, and the above tasks would all be in pursuit of this result.

The process for writing a team key result

One of the keys to effective team key results is making sure the whole team contributes to, and feels invested in, the process — it shouldn’t just be the team lead setting them for the team, or one team member setting them on behalf of everyone else. While we want every team member to contribute, it’s likely that not every team member is going to contribute in the exact same way — for example, a newer team member might not have enough context to generate KR ideas, but might have previous experience seeing KRs so can easily give feedback on a peer’s ideas.

Our goal was to make it easy for everyone to contribute and move the task forward, without expecting that we’d all contribute in exactly the same way. I feel like we did a pretty good job on this front!

Our approach was to encourage all team members work independently when generating KR ideas, getting each KR to a point where they were “ready for feedback and iteration”. Then, as a team we’d iterate on the draft KRs, and choose the KRs that were the right fit. This “ready for feedback and iteration” framing was really helpful — it helped team members make progress without feeling as though we were stepping on each others’ toes), and took off the pressure to get things right on the first go. For me, it also helped me put my ego to the side — if feedback was the goal, it was a sign of success if I got feedback rather than getting it perfectly right!

Here’s the rough four steps we took to writing our team KRs.

1. Brainstorm draft KRs focusing more on the “shape” than the content

In the first example, I showed how we started with the shape of a KR as our first iteration. We did this for other KRs too, where we said “How about ‘X% of users introduce themselves’?”. It was less important to get the exact number right, and more important to discuss whether it felt like the right “shape”. (For any of my avid dbt Discourse readers, you’ll notice this is the same approach I use in data modeling, heh).

2. State your assumptions

For each KR that we drafted, we asked ourselves (at least) two questions:

Why does this team KR affect our company KR?

This question is where we justified why we thought this was a key result worth investing in. For this round of KRs, we relied on our intuition to link a particular KR to a higher-level KR — that feels totally appropriate for the stage of the company. In the future we might use data science to check whether this team KR has a causative relationship to the company KR. The good thing about asking this question is that it forces us to state our assumptions!

Why this target number?

In answering this question we would do a bit of preliminary data analysis (we’re talkin’ rough and ready queries, not robust analytics engineering) and include a link to it in our writeup (omitted in the above examples). This data analysis was useful for two things:

Checking that we could actually measure the KR we were setting. In some cases, we found that we were limited by the availability of our data — we either had to reconsider how to measure this KR (e.g. rely on video views for measuring tutorial completion instead of page views), or think about what infrastructure we might need to build to be able to measure our KR. This also gave us a sense of how much work was going to be involved for each KR when we do the proper analytics engineers to measure it reliably.
Getting a baseline for the number so we could ground our projection for 2021 in reality. There’s still some hand-waving here, as in “if N people did this last year, it feels reasonable to assume that 3*N people will do it next year”

3. Iterate and get feedback

This was the fun part! Since the first few steps in this process were done somewhat independently, we had more candidate KRs than we wanted to commit to as a team, and some that were pretty similar in intent to each other. So it was time to reconcile the differences, and make sure we were all on the same page about what success looks like for our team.

We used a mix of Google Doc comments and synchronous meetings to make progress. We’d do one round of comments, and then another, and once our threads got too long, we met synchronously to hash it out.

4. Lock it in

We had a synchronous meeting where our team all confirmed whether each KR was a thing we wanted to lock in. We performed a ✅ emoji-adding ritual as a sign of our commitment (that’s legally binding afaik!)

Remember: you got this!

Aside from how to actually set our team KRs, I felt like I came out of this process with a few extra meta lessons

Good KRs take time! (But they’re worth it)

I really underestimated how much time this process would take, which is maybe why I’m sharing this writeup in April instead of January 🙂. This time and effort takes two forms:

Active effort: the writing and debating took time. Some of this happened asynchronously via Google Doc comments, other parts happened synchronously in Zoom calls.
Passive effort: I think all of us on out team had some level of percolation happening on this topic, where we’d go for a walk / get a coffee / drive cross country and let the ideas percolate in the back of our minds.

Writing KRs should feel challenging (especially if it’s your first round!), and like you’re bending your brain in new ways, otherwise you aren’t getting into the strategic mindset.

Similarly, they should take time. The percolation of ideas is a necessary part of the process, and probably needs to be designed for (i.e. don’t try to set your KRs in the space of a week).

Strategic thinking is a learnable skill

Anecdata: A year ago I tried to think strategically for probably the first time in my career, and sat at my desk for days staring at a blinking cursor. I even remember crying in the company’s wellness room because I felt like I just could not do it (Wellness room? More like unwellness room, amirite? (My small attempt to normalize crying at work))

Heck, even in January of this year, I looked at our KRs doc and felt overwhelmed and confused. Today, (evidently) I feel like I’m ready to teach others about what I learned.

All that is not to say that I am a genius, but rather that if this is a task that overwhelms you and feels unachievable, don’t fret! You got this! I believe in you! (And if you’re someone that I know, I might even be able to catch up for coffee over Zoom and unblock you)

Things I still don’t have answers on

The company I work for is still a relatively young company. Part of being a young company (or maybe just any company) is that there are some projects that we take on that are intuitively a Good Idea, but are hard to map to a key result, because you’ve are going where no person in the company has gone before! For example:

Within our open source community, we know that we want people to form individual relationships outside of direct relationships with our team. Measuring this is difficult though, without getting all Big Brother about it. We can imagine projects that we think will increase these external-to-Fishtown relationships, but without being able to measure the success of these efforts (except anecdotally) it’s not possible to put this down as a KR, and therefore prioritize it within a KRs framework.
As a past example, last year we moved our docs from readme.io to docusaurus — we knew it was a good idea, but didn’t have an idea of what “success” looked like.

In general: how does one prioritize an intuition-based project over a project that directly maps to the pursuit of KRs? Can we? Should we? Intuition-based projects are often the really magical ones: you don’t really know what they are going to lead to, but it’s worth trying it out just to see! In these cases, I feel that trying to put a metric on top of these projects dilutes or distracts the intuition.

This is still something I’m grappling with, so would love extra thoughts here if someone has balanced the two well in the past. My deets are in the footer below 👇