In this feature spotlight we’re going to take a look at creating tests with the Mix & Match Skills test builder.

Based on feedback, we brought in a new method for creating tests that simplifies the process while giving you more control and flexibility to be able to choose which specific skills and frameworks you want to test for. Watch the video above for more detail.

In this feature spotlight we’re going to take a look at the Insights sub-system within Evaluate.

Insights were released earlier this week, and there are two main business use cases for this.

1. More accurate evaluation of candidates. Instead of relying on the score to tell you if a candidate has the skills to do the job, you can look at how they compare to all other candidates who have taken that test.
2. It gives us a feedback mechanism to improve the quality of the tests that we offer by getting real-time feedback on whether tests and questions are at the right difficulty level for the target market

This is just the first release of our insights engine and we’ll be making updates to it in the coming months.

Cheers,
The evaluate team

In this feature spotlight we’re going to take a look at the auto-scoring code based questions. This was the top requested feature in our roadmap and allows you to test candidates real-world programming skills.

There were a few things that we specifically focused on

1. We wanted to have quite a few smaller code questions in the tests we create, so we wanted to make sure that code questions could be created quickly and easily enough that you (the user) could make them yourself with a minimum of support
2. We wanted to make sure that we supported a range of the most popular languages at launch to give us good coverage. At the moment we have support for Javascript, NodeJS, TypeScript, PHP, Python, Ruby, C, C++, C#, Java, Swift and Kotlin with quite a few more to come in the next few months
3. We want to make sure the experience for candidates was simple so that the platform wouldn’t get in the way of them expressing their understanding and skills

We hope this brings you value. We’re working hard to make Evaluate the best code screening platform possible, and making it affordable for small tech startups.

If there’s anything that you’d like us to add to our roadmap, please make sure to message us or send an email to developers@evaluate.io.

Cheers,
The evaluate team

Principals / Overview

This article will outline the guiding principles for the creation of the screening tests that are available in evaluate. All of our tests are built with certain requirements:

• Tests should assess a range of different experience levels
• Tests should be practical. We’re looking to assess if candidates have the real world skills required to be a productive member of your team. A minimum of 50% of questions should be coding or practical in nature
• Each test is specific to a certain skill set or language. Tests can be combined together if multiple skills need to be assessed
• Avoid trick questions. We’re not looking to trip people up
• Tests should take between 30 mins to 1 hr to complete. This gives enough time to assess a variety of topics, but won’t burden the candidate with an overly lengthy screening process
• Tests should assess as many different aspects of the skill set as possible, and shouldn’t focus on any particular methodology or frameworks
• Questions that have googleable answers should be avoided. We want to see people’s experience and talent coming through

Structure of a test

The following are the test creation guidelines. Tests should:

• Have between 20 to 30 questions
• Be structured in thirds:
  • First batch of questions is basics / fundamentals
  • Second batch is intermediate
  • Third batch is challenging and geared towards seniors
• Questions should be designed to take 1-2 minutes each. Questions that are expected to take longer than 5 mins should be avoided
• All questions where possible should be auto-scoring, as we want test results to be auto-generated without manual scoring
• Marks should be evenly distributed across all questions. We want to avoid overly weighting any aspects of the skill set
• Question types include single choice, multiple choice, short answer, long answer (avoid as it can’t be auto-scored) and code (only use auto-scoring code questions)

Sourcing tests

Whenever possible, we build these tests in-house. This helps to maintain consistency and quality standards. In cases where we don’t have the applicable skill set in-house, we source subject matter experts carefully using open source contributions, stackoverflow and recommendations.

Verifying tests

Before making tests available to the general public, we invite and get feedback from a minimum of 5 subject matter experts. Any questions that are controversial / confusing or that more than 3 of the reviewers got wrong are removed and replaced.

Closing notes

The above guidelines are geared towards making tests: 

• Generally applicable
• Assess as much of the skill set as possible and 
• Do so in as time efficient a manner as possible