Hiring a VP of Engineering? Use This Framework from Shopify’s VPE to Get it Right | First Round Review

Roles
Any
Section
Role
Type
Article
Date Added
May 1, 2021 7:08 AM
Language
English

Farhan Thawar is VP of Engineering at Shopify. He joined the company after the acquisition of Helpful.com, where he was co-founder and CTO. Previously, he was the VP of Engineering at Pivotal and Xtreme Labs.

When advising fast-growing startups, one of the most frequent questions I get asked is “How do I hire a VP of Engineering?” After more than 20 years, eight companies, and thousands of hires, I’m starting to suspect this may be the wrong question. A better one is, “What is a VP of Engineering?”

In 2018, a company I advise wanted to understand how their VP Engineering hire could be let go after only one quarter at the company. They had written up a thoughtful job description, sourced many candidates, and put the final ones through multiple rounds of interviewing prior to extending an offer — all for the new hire to last only three months in the role. What went wrong? It turned out that the company had a mistaken assumption about what they needed versus what they wanted in a VP Engineering. In this case, they hired a deep technology-focused leader when they needed an empathetic people leader to scale the organization. Luckily they figured that out quickly, and were able to pivot to another candidate who was a better fit.

But how do we help companies avoid this all-too-common churn? For me, it always comes back to defining what you’re looking for and what good looks like on a very granular level. Answering that will help you better understand how to find, hire, motivate, and grow folks into this role. You may find you don’t even need a VP of Engineering — many times, a team lead, manager, or director is what’s called for. (Even better than looking outside your four walls would be to promote someone internally to take this leadership role.)

First, consider what’s necessary at each level:

Team Lead: An engineer with the capacity to guide others but who’s still primarily a sole contributor. Not just a high performer with seniority, but someone who helps others with their craft.

Manager: An engineer with the capacity to lead others. Managers still contribute, but they also enable others to contribute. Managers focus on ensuring their team remains effective over time.

Director: A leader who manages multiple teams and managers. An empathetic person with a proclivity for systems thinking who has made mistakes and would make better choices in the future.

VP: A leader with a vision for how to build a highly effective engineering organization. Great ones know what work to hand off and are continuously motivating those beneath them in the style required.

Next, if a VP of Engineering is indeed what you’re after, I’ve found that candidates must be skilled in all three classic areas:

Process: What occurs when?

People: Who’s involved and how are they motivated?

Technology: What will we use to get there?

If you ask a technology leader how they would rank the above three skills from most important to least, you get a good sense of their skills and interests. But even more valuable is the chance to tease out whether their skills align with what you need right now, in your company, at its current stage.

The order of these will change over time for both your company and the person. In early-stage companies, for example, you’re usually looking for someone to pair with a very technical co-founder or CTO to help define the early development process. As the company scales, the requirements change. You might now be looking for a more people-focused leader as the team grows in size and technical craft and career development come into focus. A failure to adjust these profiles and fine-tune the hiring search is where most startups run aground in my experience.

A “right fit” VP of Engineering fit is partly a function of timing — their skills and interests must match the present need.

There is tons of content and advice out there around navigating this important hire, but few practical decision frameworks that help founders and technical leaders probe more deeply into what it is they’re looking for in a VP of Engineering. Over the years, the process-people-technology framing is one I’ve come to lean on and coach startups through. In this article, I’ll dive into each of these skill areas to discuss why they’re important, how to know whether they’re a priority for you at this stage, and how to recognize people who are a good fit for each.

There are many different and successful people, styles, and processes that help you build effective engineering teams. There are no silver bullets. But hopefully this framework — along with some questions to ask in interviews and red flags to watch out for — will help you dive a few levels deeper into figuring out what type of leader you need to grow your company.

image

Farhan Thawar, VP of Engineering at Shopify.

PROCESS: ASSESSING A CANDIDATE'S OPTIMIZATION SKILLS

A great engineering leader has opinions on a favored development process. This doesn’t necessarily mean they will force this process onto your company (well, maybe), but it’s good to have strong opinions and their own toolbox of process techniques to try out.

This need goes beyond just fine-tuning the development process, however. They also need tools for winning and keeping engineers’ trust. Back in 1968, The Harvard Business Review (HBR) published a seminal article on how to motivate employees. Many of the lessons still resonate today. The most notable finding is that employees can become greatly dissatisfied based on company policy and how it’s administered. Good leaders must reduce this dissatisfaction, and it’s an especially urgent need in engineering.

Most engineers care deeply about optimization and suboptimal processes can drive them crazy.

Another danger for any technologist is to get so excited by technology and forget about why they are building in the first place. The customer matters, whether internal or external. A great leader knows how to run projects with short feedback cycles, as well as projects with very long ones. This means the customer may see value early and often (as with a feature update) or over a long-term horizon (as with a platform rewrite).

Let’s dive into a few of these specific areas:

1. Experiments with the development process

I don’t believe there’s truly one correct way to write software. As such, every development team and environment can be different and still be successful. You only have to look at the debates around closed versus open offices or in-office versus remote work to see that there are many ways to win.

This means a strong leader must know whether to experiment with ideas like pair programming, daily standups, remote work, flex time, weekly retrospectives, frequent one-on-one’s, quarterly themes, and so on. Just because a technique worked in their last role, doesn’t mean it’s going to work at your company. Similar to how a product leader runs experiments with users in the product, an engineering leader in a growing team is running experiments in their domain: the development process.

More specifically, a strong engineering leader knows when and for how long to try process improvements for the development team. For example, at Shopify we started experimenting with six-week development cycles in 2019, as a way to break our work into more manageable chunks, demo high-fidelity work and make better prioritization decisions. After experimenting with this in one team, it quickly grew to become the cadence for all of engineering and commercial teams company wide.

Good process is like a traffic light. It may slow down the commute for a single driver, but it optimizes the system for everyone on the road.

Look for these qualities to find a strong experimenter:

Has shipped code with different development processes.

Has examples of things that have failed but they’ll try again.

Has strong opinions but is willing to change their mind.

Ask these questions to surface the best candidates:

Describe the development process at your last company? What would you try again and why? What would you not try again and why not?

What have you changed your mind about when it comes to the development process or engineering team practices?

Watch out for these red flags to avoid a mishire:

Strong opinions, strongly held. Not reflective on their experience. If things didn’t work before, they would do the same thing again.

Has delivered software only using one development process.

2. Reduces admin burden

It’s often overlooked how powerful a dissatisfier administrative tasks can be. Referring back to my favorite 1968 HBR article, the most powerful reason for employee dissatisfaction — company policy and administration — is almost as powerful a force as the top satisfier — achievement.

I keep this image as a favorite photo on my phone and refer to it often.

image

Chart showing factors affecting job attitudes

The best leaders can shield engineers from unnecessary admin and policy due to shitty systems like vacation tracking, getting their expenses paid on time, submitting resume referrals, or even helping to kill ineffective meetings (this one requires advanced skill).

A good leader will also find ways to be attuned to what’s bothering the team from a process standpoint. One way is via surveys (once the team gets large enough). You’ll be surprised what you can learn from a quarterly survey just asking simple questions like “What’s the worst part of your day?” or “What tool do you wish we used but don’t?”

Letting your engineers spend time working keeps them happy (technically, less dissatisfied). Nick Caldwell, VP Engineering at Twitter, suggests adding a question to your quarterly surveys on “time wasters” in your engineering organization and then having a crack team eliminate them.

Look for these qualities to find an unblocker:

Unblocks all issues (not just engineering ones).

Regularly identifies and destroys time wasters.

History of high morale engineering orgs.

Ask these questions to surface the best candidates:

Describe a shitty system at your last company. What did you do about it?

On larger teams, how do you get a sense of what’s working and not working?

Watch out for these red flags to avoid a mishire:

Has a tolerance for sub-optimal systems. Doesn’t have a track record of making things better.

Only focuses on engineering issues. Doesn’t broaden to fix organizational issues.

3. Focuses on continuously delivering value to the customer

Some of the best engineering leaders I’ve worked with want to interact with customers. This means they are always thinking about how to deliver value the customer can feel. Notice I didn’t say what, but how. This means they like to ship frequently to get feedback.

In tandem with a product and UX leader who wants to experiment with the product and how it works, an engineering leader should have a process cadence that allows the customer to give regular detailed feedback. A great process, without unnecessary back-office spinning (undue refactoring, long QA cycles, approvals, etc.) will let the customer see the product often and give a steady stream of comments on the order of days or weeks, not months.

One danger smart people fall into is to think they understand what the customer wants. It’s really important to realize that the customer owns the problem, but that you can own the solution. Keeping these two ownership circles distinct, allows two great processes to happen separately. Firstly, the customer can vividly articulate their needs without “solutioning” too early. And then, the technology team can dream up and test amazing prototype solutions to their problems. Win-win.

Look for these qualities to find a champion of the customer:

Knows which projects are appropriate for a fast feedback cadence.

Says things like “This will take three weeks, but if we make this change we can test in two days.”

Let’s the customer own the problem, but takes ownership of the solution.

Ask these questions to surface the best candidates:

How would you get feedback on a platform rewrite?

How do you handle customer feature requests?

Describe a time where you felt your team wasn't shipping frequently enough. What did you do about it?

Watch out for these red flags to avoid a mishire:

Works with very long feedback cycles and is unable to figure out how to get feedback more quickly.

Triages and implements customer feature requests, instead of understanding the customer problem.

PEOPLE: ASSESSING A CANDIDATE'S TEAM BUILDING CHOPS

I don’t think it’s an overstatement to say that “people” are the most important tool in the engineering leader’s toolbox. However, at different inflection points in a company’s growth, people concerns can become either more or less important as their primary focus.

The key for a leader here is to ensure all of engineering (and adjacent disciplines) can be pointed in the right direction. This pointing (through vision and mission) need not be — and often isn’t — dependent on their position in the org chart. Influence exists separate from authority. Here are the specific qualities I look for to spot engineering leaders who are strong here:

1. Attracts candidates and creates pipeline

Great people want to work for great leaders. In every instance that I’ve seen an amazing leader move to a different team or company, sharp folks follow in their wake. That’s why it’s a great sign if someone transplanted almost their entire team into a new organization — it tells you people want to follow them, and that they go to bat for their people to get them hired.

This force of personality effect is the real key to recruitment — not merely a system of recruiters, quotas, cold calling, or LinkedIn searches. I find that many of the best people aren’t actively looking anyway. They’re motivated by more than pay and want to prove themselves by tackling big, audacious challenges. One of the only ways to lure them out of a comfortable role is to offer that great challenge under a good leader.

Good leaders also intuitively understand the value of going on the road to attend recruiting events, give talks, and keep tabs on prospective candidates. You have to go find the top talent, and they know it. They also know that the work doesn't end once the offer letter is signed. This means they are constantly advocating for people on their team to be part of large opportunities, either under them or another leader. Good leaders also value diversity and a diversity of opinions. By collecting evidence over time, they also know when to part ways with someone and how to do it effectively.

The leader's vision in this area should come through in the interview, and it should come through in how they interview candidates. They should be able to make you want to be part of what they’re building. The best VP of Engineering candidates will do two things:

They will ask lots of probing questions and take notes (or have a good memory for the details discussed).

They will formulate and share a sample plan for their first quarter in role based on what they learned from in the step above.

It’s rare to meet candidates who do the above, but it’s refreshing when you do. It shows that they know how to probe the organization for information and can come up with things to try when in seat. Of course, you’ll never know if the candidate is all-talk or can actually execute in your company until they have started, but you can use past-performance as a lossy proxy to that.

Look for these qualities to find a strong recruiter:

Their team tends to follow them to next opportunity.

Has mentors or mentees outside the company and continues to do external 1:1’s.

Is effective at parting ways with low performers.

Ask these questions to surface the best candidates:

Have you ever worked with the same people or teams at different companies? How did that occur? Are there advantages and disadvantages of working with some of the same people over time?

Tell me about a superstar you’ve worked with. Would you work with or hire X again? Why or why not?

Watch out for these red flags to avoid a mishire:

Has never worked with a previous teammate in a new company.

Has never let anyone go.

2. Makes the team more valuable for having worked there

There are some organizations that look great on the resume because they’re a household name. There are others that look great to people in your industry because they know you solved a big challenge there. The two aren’t necessarily related, and the latter is far more valuable. You want the sort of leader who helps the team achieve such success that the company becomes a byword among engineers.

A company doesn’t have to be large or have a tremendous financial exit for the alumni to form a valuable network. The two strongest networks from my past are Trilogy and Xtreme Labs. Both were midsized companies which never passed the few hundred person mark. Both focused on folks early in their careers and left a strong imprint for future roles. Yet each has a very strong and engaged alumni community that makes its members stronger with private social networks and job referrals.

This isn't just about cultivating a strong alumni network or building a good engineering brand — it's about the opportunities you create and encourage while employees are still at the company.

As an example, great leaders I’ve seen invariably ask their engineers to interface directly with clients and engineers at other companies. They believe in having them join customer calls, hold hackathons, give talks, write blog posts, and take on leadership roles because it’s good for them and because it improves their work.

Look for these qualities to find an educator:

Known engineering brand at new employers

Ask these questions to surface the best candidates:

Do you keep in touch with your previous teams? Why or why not?

How do you keep track of the careers of the folks you’ve worked with previously?

Watch out for these red flags to avoid a mishire:

Has never participated or built an alumni community based on previous experience

The engineering brand of previous companies is flat to down

3. Gets energy from helping others succeed

Great engineering leaders delegate even when they know the results will be less than perfect. As a policy, they trust first, because they know that micromanagement erodes people’s confidence.

If your parents always correct your homework, you never learn to check it yourself. Good leaders let you try, fail, and then step in. They trust, but verify.

At previous companies, I went as far as to remove checks and balances between engineers and code that went into production. “Look, no one else is going to check your work,” I’d tell them. “If you think it’s ready, it’s going straight to production. You’re not accountable, you’re responsible.” I found that this no-brakes approach bred a totally different mindset among teams of engineers. They became owners, not participants, and produced higher quality code.

A good leader also shows up to the little things. A leader who’s so overwhelmed with obligations doesn’t have time to invest in their people, especially when it means they’re late for things like 1:1s. I see this as a personal time-management issue, not an organizational issue. You’re never too busy if you’re running an efficient operation. Great VPs are delegating and empowering their people and so have time to show up.

Look for these qualities to find a servant leader:

Takes all of the blame and none of the credit.

Available during a crisis.

Focused on giving folks the best opportunity of their lives.

Ask these questions to surface the best candidates:

Do you do regular 1:1’s with your team? Why or why not?

Describe a time when you felt a superstar should leave your team to go to another.

What’s a small example of how you try to show up for the people on your team? What’s an example where you realized you could maybe be doing a better job here?

Watch out for these red flags to avoid a mishire:

Doesn’t use 1:1’s as a way to surface issues, learn about their team and offer stretch assignments.

Has never felt that a team member could have a bigger opportunity outside of their team.

TECHNOLOGY: ASSESSING A CANDIDATE'S ABILITY TO DIVE DEEP

Your VP of Engineering doesn’t have to be the expert on every technology, although good ones will have a deep understanding of a few domains. Instead, they have to tinker, know what’s possible, and build a team that finds, implements, and improves the right technologies. Not all of these leaders will code regularly, but many will find a way to prototype ideas or interview highly-technical candidates with code.

Per Conway’s Law, your architecture generally reflects your organizational chart. If your VP of Engineering is the final say on ALL technology choices, they can become a bottleneck. Far better to have someone whose team chooses great technology, but who remains objective, challenges them, and improves their choices.

It’s worth noting that great leaders will have great technicality. They should be able to roll up their sleeves and dive deep into an issue even if it’s just to ask a question that’s one or two levels deeper or validate an unspoken assumption. I would do a deeper dive of leaders who left individual contributor work to “get away” from technology. In my view, answers like: “I wanted to get away from coding” or “I always wanted to be a manager” should be treated as red flags. Engineering leaders are pulled away from writing code kicking and screaming once they realize they could have greater leverage through people.

Technicality can go all the way up the org chart. At Shopify, it’s not uncommon to see Tobi (CEO) or JML (CTO) rolling up their sleeves to prototype a new technology, participate in hackathons or comment on a PR even in a company with thousands of engineers.

1. Isn't afraid to ask tough or deep questions

A good VP of Engineering can probe enough to know whether someone has mastered their area or not. Joel Spolsky has a great story on his interaction with Bill Gates. While a leader doesn’t need to know the full context, they should be able to go a few levels deeper. At Apple, for example, leaders are expected to go 3-levels deep into their organizations. The goal is not to micro-manage the team, but to be able to ask good probing questions. By staying close to the technology and being able to dig further, a leader can question some fundamental assumptions that may no longer be true. Again, trust but verify is the goal.

Good VPs also excel at structuring discussions to come to objective conclusions. For instance, formats like apocalypse meetings can be used to ensure everyone’s voice is heard — especially the quiet ones who don’t like to publicly debate but are often right on arcane points.

A leader’s job is to short-circuit the regular information flow to get to the real answer. I like to think of TJ Watson Jr’s letter to his company — IBM. Here, Watson tries to understand why a smaller and more nimble company was able to execute more effectively than IBM.

A good VP is healthily obsessed with getting better every day, and asks questions to get there.

Look for these qualities to find an inquirer:

Seen as union leader.

Knows when conditions change to make technological change.

Can dive deep into a few things per quarter.

Ask these questions to surface the best candidates:

Do you still code? Why or why not?

Why leave IC (individual contributor) work?

Give me an example of how you dig deeper into a project your team is tackling. Are there certain questions you like to ask, or approaches you’ve found to be effective here?

Watch out for these red flags to avoid a mishire:

Is uninterested in understanding technical concerns or even looking at code.

Went into management to “get away” from coding.

2. Stays unbiased toward their own experiences

Patton was famous for saying, “Lead me, follow me, or get out of my way.” Great VPs have a bit of Patton (minus the ego). They’re in it to accomplish the mission, and are open to any means to solve it. They don’t just bring in tools they’ve been successful with in the past — they ask what the current situation calls for, and adapt. If they used Java in the past, they’re open to Ruby. If they’ve worked in the office, they are open to remote work. If they’re accustomed to native mobile apps, they’re open to Flutter and React Native.

A recent example comes to mind: In 2013, I wrote an article for First Round on native mobile engineering based on my experience building hundreds of mobile apps at Xtreme Labs. But the fundamental assumptions around mobile engineering slowly changed and by 2019 were no longer correct. Once I started leading Mobile Engineering at Shopify, we decided to go all in on React Native — an approach I was violently against in my 2013 First Round article. So while it looks like I’m contradicting my own advice, I’m not falling prey to my own bias.

The key ability here is knowing how to objectively weigh tradeoffs, like the team’s existing familiarity versus the future technology curve — where the technology is heading. Or the need to hire for a specific skillset. Or the recruiting potential of a new technology because it might allow you to hire a future set of engineers.

Another critical attribute is the ability to determine what’s going on and who has the best context. It’s tough to not rely on the “information bubble” that a leader may find themselves in. A great leader will be able to roll up their sleeves and get their hands dirty in the actual work — instead of relying on a weekly status report.

Look for these qualities to find a leader of technologists:

Changes mind with new information.

Can try solutions to problems they’ve never encountered before.

Doesn’t use a proxy to performance.

Ask these questions to surface the best candidates:

What technology have you recommended in the past that you no longer recommend?

What do you ensure you don’t remain in an information bubble?

Watch out for these red flags to avoid a mishire:

Continues to do things the same way they’ve always done them.

Doesn’t have a small council or skip-level conversations and projects to get the real data from ICs.

3. Trusts the team but can play tie-breaker

Good VPs are air traffic controllers. They’re not planning the routes for all the planes, nor fueling them up, nor flying them. But when they converge on a critical intersection, the controller tells them how not to crash into each other and re-plans optimal routes. They mostly let the planes do as they please, but if two are about to collide or burn fuel needlessly, they have no reservations about breaking the tie and calling the shots.

A skill I particularly treasure is the ability to see when decisions aren’t zero-sum. There’s a classic exercise where two people are told to negotiate over an orange. Both are told they run drug companies creating life-saving vaccines. (Boy has this scenario become more realistic since this exercise was invented.) Both participants are given a budget. The game is designed so that people negotiate all day without making progress, before someone finally discovers that negotiator A needs the rind and negotiator B needs the juice. They can share.

The engineering corollary to this: Between A and B, can you just do both or pick one and then sequentially the other? Oftentimes this is faster than spending the time on analysis.

Look for these qualities to find a coach:

First question is always: “What do you think?”

Frequently called upon to be coach, sounding board, or devil’s advocate.

Ask these questions to surface the best candidates:

When two members of your team disagree, how do you help solve the disagreement?

What are some ways you get included in the “information flow” of teams?

Watch out for these red flags to avoid a mishire:

Either always decides disagreements or always delegates. Doesn’t take context into account.

Not in the information flow of projects, i.e. not trusted by the team to be helpful.

Thanks to Joshua Koppelman for first introducing me to this framework, and to Mallorie Broadie for asking me the right question in the first place. For questions, hit me up on Twitter.