Coding Challenge Best Practice Checklist

Criteria for great technical recruitment test design

v0.1.0
Matt Fisher
September 2019

---

It's becoming more common to use coding challenges as technical recruitment tests.

Talking to people in the industry, it has become clear that experiences vary wildly, and there are no standards or best practices to apply when issuing coding challenges. This is a problem.

Coding challenges can be a highly effective and efficient part of recruitment for all involved - but poorly designed tests waste time and lose talent.

At Candidate Code we'd like to see that change. So, ask yourself:

Each of these items is expanded on below. If you want to contribute to this list (and the discussion around it), please checkout the contribution guidelines in the GitHub repository and issue a PR.

---

Criteria In Detail

It generates strong hiring signals

• A coding challenge for hiring purposes should give you strongly actionable information to help make a hiring decision
• Think carefully about what skills it is most important for a candidate to demonstrate, and then design your challenge specifically to test those skills
• You will usually obtain the most relevant results if you organize your test into separate but complimentary tasks, each focusing on a particular technique or problem
• Prioritize what is most important to the role you're hiring for. Respect your candidates' time. Only ask them to work on things that are actually important to you.

The assessment criteria are well-defined

• Be explicit about what criteria you are using to assess a candidate
• Be explicit about what does not form part of the assessment; e.g. don't have candidates spend time on writing tests if you don't consider it an important part of the exercise
• Make sure this is communicated to the candidate as part of the challenge's instructions

An expected duration is communicated

• It is important for a candidate to know roughly how long they should be spending on a coding challenge
• Knowing this allows applicants to schedule out and manage an appropriate period of time - or walk away if they choose
• It also allows engineers to sanity check their progress. If they've spent a day on a two hour test, it might be time to reach out for help

Scoping is clear

• All scope should be captured in written instructions that can be referred to during the course of the test
• A candidate should be clear about what they're supposed to be doing; don't allow figuring that out to become part of the test
• Communicate specific exclusions on out-of-scope tasks if required

A time limit has been considered

• Always consider application of time limits carefully. The decision to apply a time limit should always be driven by what is best for your candidates
• In general, time limits are a good idea for shorter screening tests (e.g. 30 minutes); they can prevent an unsuitable candidate spending hours pursuing a role they are not qualified for
• Time limits may be inappropriate for longer challenges completed in non-continuous sessions
• Any time limit applied should allow successful candidates to complete the test with plenty of time to spare. Don't allow time pressure to compromise your ability to assess a candidate's skills

A starter project is provided where appropriate

• Where possible, give candidates a starter project containing the files required to install dependencies, run tests and execute their code
• Provide explicit instructions for setting up and running the project
• Supply as many tools as possible to give the candidate feedback on their own progress in line with your own assessment criteria. E.g. tests, linting configuration
• This may not be possible for some challenges which are language-agnostic. If this is the case, consider whether your test meets the other checklist guidelines carefully

The submission process is documented

• Tell candidates how to return their completed code to you. It should be obvious, but it is often overlooked
• This can be accomplished in a number of ways (email attachments, Google Drive uploads, public PRs, purpose-built services). Try to make it as low friction as possible
• If necessary, instruct candidates about which files and directories should be present in the returned submission (e.g. include the .git folder, but exclude node_modules)

It has been evaluated by in-house engineers

• Always evaluate your coding challenge on your in-house engineers. Try to mimic the process which your candidates go through as closely as possible
• Record how long it takes to complete in order to provide a better idea of how long it should take your candidates
• Use the opportunity to collect feedback and improve the instructions and scoping of your challenges