TDLR: This is my opinion.

To candidates: Go where you’re treated the best.
Why would you spend X hours upfront on an unpaid project to be interviewed?
While at top tier companies, if you succeed the 1h phone interview, you’ll be invited on-site with travel costs paid.

To companies: DON’T SEND “full application to complete” take home challenges to filter out. It sends the signal: “we are OK to waste hours of your time, so you can have the privilege to interview”.
Instead, use a short interview screen. 30 min is more than enough and what a proper code review would have taken.
The quality of people will rise.

To be a known quantity, please share your experience in this survey . A list of good companies will be published.

Take-home projects are heavily in favour of company side: no time invested, free consulting.

Some argue take-home challenges reproduce the real work environment with developer tools that anyone can do at their own convenience. The very same camp argues that whiteboard coding (algorithms + data structure) does not reflect daily routine. However the take-home approach has a huge time investment and the risk to be ghosted (ignored without feedback) is high.

The reasons why take-home assignments became popular is time invested asymmetry. It takes much less time to review than to do an assignment.
It basically sends the message: “we prefer to waste 100 hours of candidate time than a few hours of company ‘s time”

However take-home challenges themselves as a filter is just a wrong strategy.

Their hiring funnel looks like this

Here are the steps in the hiring funnel:

• Applicant pool. The candidates who applied are in the applicant pool. The resumes and cover letter are usually read by a human in smaller companies who may call each candidate for the 1st interview.
• HR: Major HR interview is a friendly 30 minutes chat. After everyone will receive a coding interview to complete at your home on your time. Some passive developers are contacted on various channels ( Linkedin, StackOverflow … ) by the HR/recruiter who try to bring attention.
• Most recruiters have no way to filter technical people, usually almost everyone who interviewed with the recruiter will receive the coding challenge.
• take-home challenge: Once the homework read, the majority of developers believe the challenge is not worth their time and will stop there. Among the candidates who interviewed first with HR and received the take-home test, how many of them complete the task? Few recruiters were transparent with me about the percentage of people who submit their code. My guesstimate is fewer than a third submitted for an established product company . As Gayle mentioned if this is screening round, it should be kept short.
• interviews Often the code challenge is not even discussed, it was only used as a filter. On several occasions, I like that these interviews served as an open discussion for design choices and the take-home project serves as a template to add additional features during a live coding interview (googling is allowed)
• hiring committee (if there is any). Interview reports are read and the salary range is discussed. An offer might match.

Pain points in this funnel

You can already see two major pain points in the funnel :

• Almost everyone in the applicant pool was interviewed individually by HR. HR has to sell candidates to complete the homework. The HR is really busy to filter, interview, follow up with candidates. The focus is on quantity rather than quality.
• survivorship bias on who completed the challenge. The hiring managers may believe that only successful candidates were the ones who submitted and may brush off that those who didn’t submit not were up to the challenges. In their vision, the take-home challenge is a test of worthiness for the next interviews.
• Do you really think qualified candidates will do the test? People in demand will simply prioritise better alternatives.
• very few candidates submit the take-home test. Because full-time employees who apply elsewhere prefer low hanging fruits. Didn’t you notice the HR keeps sending reminders about the project status?

If the conversion rate is low, something is wrong in the funnel. Which leads to why don’t candidates just submit the homework?

Take-home challenge is not scalable

The take-home test approach is not scalable for both the interviewers and interviewees. As each project is unique, the interviewers have to reply to candidates who request clarity (if the authors ever reply at all). Reviewing the submitted project takes time to understand the logic. The reviewer may spend more than 1 hour to review correctly. This hour should be rather spent on tech screening.

At your work, a full code review of feature request may take more than one hour. Yet the scope is smaller than a full blown project.

Project scope & time underestimated

I don’t mind to complete an automated graded test online for 1-2h.

However, most take-home tests are the type “Build an app/website that X and Y from scratch. We expect tests, threading, SOLID architecture, deployment, production quality, offline mode, 100% test coverage … You have 4h (to unlimited time) to complete the project”.

Were the expectations and scope of the test stated properly? Often you have to navigate with ambiguity and the project scope is ridiculously too large.

I said I don’t mind to spend 2h. Oh now the company set the time limit to 2h ? and the scope is still the same. Of course, the projects are highly underestimated.

For tech screening, online tests are much faster to evaluate candidates.

Automated tests as tech screening

Since the code challenge have to be reviewed by humans. It is much better to filter with online automated test (coderpad.io ) . Main advantages are:
– partial grading. Expectations are known. Different reviewers have different way to grading.
– time efficiency for the company. No one has to correct behind the scenes. Candidate

No feedback, no review at all

With take-home assignments, rejection without feedback is common.
While if I do an in-person interview, I guess where I missed the answers. The HR always follows up with feedbacks.

You don’t know on what criteria you will be evaluated. Your application could be rejected on “unstated” trivia criteria:

– “we don’t care about problem solving, we only care about your code style.”
– use the wrong formatting tool. In the developer circle, I heard various stories: “I spent 8h on an Android tic tac toe project, they turned me down because they didn’t like the code formatting”
– use a popular library that is not known by the team
– use an architecture that the team is not familiar with
– deliver a high quality project, over-engineered for the reviewer
– …

Intellectual property

Imagine, you gave your best shoot and product of your work. You submit. No feedback, you are rejected.

However your code solved a business relevant to this company. You solve in a novel approach. They use your code in production.
You are still owner of your code, but you were not paid.

Only complete a challenge that a company has already solved in different ways long ago.

Why Top Tiers companies don’t use take-home tests?

Top tech companies FAANG (Facebook, Amazon, Apple, Netflix, Google) and other respectable ones in Europe ( Spotify, Zalando, Criteo … ), Asia (Alibaba …) DON’T give take-home assignments as a filter. There is not even a take-home coding challenge. There is no shortage of candidates willing to work for them who would take the unpaid assignment in a heartbeat.

After passing the HR interview (30 min) and first technical screening (45 min to 1h), the candidate is offered to travel on-site (flight+hotel+food). So why spend more than 4h+ only on the possibility to reach interviews? While top candidates invested 1h30 interview time and are compensated with travel and accommodation.

Take-home assignment is a huge opportunity COST. Qualified candidates will be turned off. For screening, a 1h interview is enough. Serious companies know that the better candidates won’t put up with crazy requirements like a lengthy take-home that most likely lands in black hope.

I once explained this situation to an HR lady: “We are a start-up, we don’t have the resources”. In admitting this, she hinted the compensation will be lower.

There are way more people applying there. The ratio of candidates to interviewers is higher. How do these companies find the time to interview candidates?

Look at their hiring funnel. The hiring funnel picture below is not at scale. First, notice the applicant pool is much larger and filtered for the HR screening. Second, the take-home challenge is replaced by a short technical screening.

Optimal hiring funnel

• much larger applicant pool. At large companies, it is not uncommon that candidate profiles are filtered by machines ( ATS = application tracking software) based on keywords. To stand out from a large number of people, you should ask an employee (who knows you) to refer you.
• HR: usual same friendly HR interview who will guide about the whole process. HR will prepare you for the materials and resources to study to ace the next technical/behavioral interviews.
• Technical screening (45min to 1h): introduction of candidate and interviewer followed by one whiteboard coding, sometimes two (if you do well). If you performed one working solution that is not the optimal ( time and space complexity), you will get a 2nd technical screening. Depending on the companies, the success pass ranges from 5 to 20%.
• interviews. 3 to 5 interviews based on algorithms, system design, and behavioral. A bit more difficult than screening. Generally, you are invited on-site, flight and hotel are offered by the company. In the Coronavirus period, the interviews are online.
• the hiring committee will read the interviewer report. Depending on the headcount and availability, they will make an offer.

Hire the best, not the desperate

The best job candidates have choices and will not ignore those jump through hoops processes.
The desperate have time on their hands to complete the challenges. I would advise to work on personal demonstrable projects is better use of their time.

Whiteboard practice

From now on, I prefer to stick to whiteboard coding style interviews because they are time-boxed and scalable. Once I understand a problem, I never forget and re-apply the same principles elsewhere. The Leetcode efforts are amortized. If you believe that Leetcode is learning by heart, you are doing wrong. You need to understand the problem solving approach under the hood.

On whiteboard coding , the metrics are clear: the solution passes all tests, thought processes, efficiency in time and space complexity.

It is fun to experiment with a project with the latest libraries ( still in dev, alpha, beta version, I am looking at Android Hilt, Jetpack Compose … ). But it is better to understand the CS fundamentals used to build these libraries or frameworks. The algorithms and data structure are timeless knowledge.

Usually, employees working at companies without whiteboard interview might be good technician of a particular framework and know by heart the API methods. But they lack basic data structures (Map, Set) and CS fundamentals. Being on the hiring side, I interviewed such candidates coming from these companies.

Their solutions on the whiteboard are far from the most optimised, almost no notion of Big O notation. To the question: “Propose a solution to find an element in an array in constant time”. The candidates would use the built-in binarySearch in Java on an unsorted array. Instead of pre-caching the values of the array in a Map or Set. My colleagues unanimously said: “For sure, NO hire.”

What are your experiences on the hiring funnel for any companies that send take-home projects? Please comment below.

If you are on the hiring side, please post your funnel numbers (how many received tests /submitted/were invited for interviews)

PS: from others experience, companies with take-home assignment are only “interviewing but not actually hiring”.

Views from other industries

I once spoke to a friend who is actor, David . Each casting is a free interview. But the recording won’t be used for commercial purposes.

No actors, dancers will record a commercial scene for free. What you told me is :
Imagine you go to as many restaurants and tell them:
“I will pay only the best after I sample each potential restaurants for free.”
Won’t you feel bad to about this to each restaurant? This is exactly what these companies are doing to you.

Dear fellow engineers, don’t get scammed, you are in demand. You have the power to say NO.

FREE WORK?

Ask other professionals for free sample. Try to ask a a locksmith to open your door for free? It takes a few minutes of their time.

Now developers are asked to spend hours on a single take home assignment. Do you think to it’s fair offer your time for free?

A few times per month, I receive a cold phone call from a recruiter. “We have this amazing opportunity. Would you like to hear more? … We ask for a small code challenge, it will take only 3 hours”. After a quick glance, the project is a full-fledged app with pixel perfect Sketch designs. The assignment asked for unit tests, integration tests, complete CI/CD pipeline, [docker & kubernetes]. It would take at least a week to complete. My response: “Listen , you are the ONE who contacted me. I never applied.”
And your response should be:

Hi [recruter],
To be perfectly honest, I'm really excited about [company] and got a great impression of the company from my interview with [you or engineer]. I estimate this assignment will take at least H hours of work. My contract rate is €$ X/hour. I can complete this assignment for [total] and return it to you completed by the end of next week. If that isn't acceptable, I do have an extensive portfolio of my work available on github (link). Please let me know how you would like to move forward.
Best,

You could probably charge for 165 USD/h for 20 like this guy did.

Even Free work from reputable Firms?

Reputable firms are classical banks (not online banks), retail stores or any companies with 9 digits annual revenues.

I submitted code to a firm. The reviewers were impressed but the firm wanted more features, it would take 2-3x times. I politely declined. The manager invited me for the rest of interviews. So the code reviewer was not even important enough to be part of the interview.

Another time, I sent the project. The company sent feedbacks, but the position was refilled by an internal employee.

Or the best, I inserted analytics in the project and readme.html ( not .md file), so that when the documentation or project are opened, I’d be informed. The company never opened and sent a blank template “no pass”.

Most likely, you will gain nothing from completing take home assignment except adding to your public portfolio. My repeated and peers experience is that simply being asked to do a “take home project” is a strong indication that companies are cheating and not actually hiring.

Survey

Please share your experience in this survey

The goal is not to shame and name the authors. But to get an overview in the hiring funnel and share the fair players in this coding homework game.