Companies that give Take-Home Tests: review your hiring funnel

TDLR: This is my opinion.

To candidates: Go where you’re treated the best.
Why would you spend X hours upfront on an unpaid project to be interviewed?
While at top tier companies, if you succeed the 1h phone interview, you’ll be invited on-site with travel costs paid.

To companies: DON’T SEND “full application to complete” take home challenges to filter out. It sends the signal: “we are OK to waste hours of your time, so you can have the privilege to interview”.
Instead, use a short interview screen. 30 min is more than enough and what a proper code review would have taken.
The quality of people will rise.

To be a known quantity, please share your experience in this survey . A list of good companies will be published.

Take-home projects are heavily in favour of company side: no time invested, free consulting.

Some argue take-home challenges reproduce the real work environment with developer tools that anyone can do at their own convenience. The very same camp argues that whiteboard coding (algorithms + data structure) does not reflect daily routine. However the take-home approach has a huge time investment and the risk to be ghosted (ignored without feedback) is high.

The reasons why take-home assignments became popular is time invested asymmetry. It takes much less time to review than to do an assignment.
It basically sends the message: “we prefer to waste 100 hours of candidate time than a few hours of company ‘s time”

However take-home challenges themselves as a filter is just a wrong strategy.

Their hiring funnel looks like this

Here are the steps in the hiring funnel:

  • Applicant pool. The candidates who applied are in the applicant pool. The resumes and cover letter are usually read by a human in smaller companies who may call each candidate for the 1st interview.
  • HR: Major HR interview is a friendly 30 minutes chat. After everyone will receive a coding interview to complete at your home on your time. Some passive developers are contacted on various channels ( Linkedin, StackOverflow … ) by the HR/recruiter who try to bring attention.
  • Most recruiters have no way to filter technical people, usually almost everyone who interviewed with the recruiter will receive the coding challenge.
  • take-home challenge: Once the homework read, the majority of developers believe the challenge is not worth their time and will stop there. Among the candidates who interviewed first with HR and received the take-home test, how many of them complete the task? Few recruiters were transparent with me about the percentage of people who submit their code. My guesstimate is fewer than a third submitted for an established product company . As Gayle mentioned if this is screening round, it should be kept short.
  • interviews Often the code challenge is not even discussed, it was only used as a filter. On several occasions, I like that these interviews served as an open discussion for design choices and the take-home project serves as a template to add additional features during a live coding interview (googling is allowed)
  • hiring committee (if there is any). Interview reports are read and the salary range is discussed. An offer might match.

Pain points in this funnel

You can already see two major pain points in the funnel :

  • Almost everyone in the applicant pool was interviewed individually by HR. HR has to sell candidates to complete the homework. The HR is really busy to filter, interview, follow up with candidates. The focus is on quantity rather than quality.
  • survivorship bias on who completed the challenge. The hiring managers may believe that only successful candidates were the ones who submitted and may brush off that those who didn’t submit not were up to the challenges. In their vision, the take-home challenge is a test of worthiness for the next interviews.
  • Do you really think qualified candidates will do the test? People in demand will simply prioritise better alternatives.
  • very few candidates submit the take-home test. Because full-time employees who apply elsewhere prefer low hanging fruits. Didn’t you notice the HR keeps sending reminders about the project status?

If the conversion rate is low, something is wrong in the funnel. Which leads to why don’t candidates just submit the homework?

Take-home challenge is not scalable

The take-home test approach is not scalable for both the interviewers and interviewees. As each project is unique, the interviewers have to reply to candidates who request clarity (if the authors ever reply at all). Reviewing the submitted project takes time to understand the logic. The reviewer may spend more than 1 hour to review correctly. This hour should be rather spent on tech screening.

At your work, a full code review of feature request may take more than one hour. Yet the scope is smaller than a full blown project.

Project scope & time underestimated

I don’t mind to complete an automated graded test online for 1-2h.

However, most take-home tests are the type “Build an app/website that X and Y from scratch. We expect tests, threading, SOLID architecture, deployment, production quality, offline mode, 100% test coverage … You have 4h (to unlimited time) to complete the project”.

Were the expectations and scope of the test stated properly? Often you have to navigate with ambiguity and the project scope is ridiculously too large.

I said I don’t mind to spend 2h. Oh now the company set the time limit to 2h ? and the scope is still the same. Of course, the projects are highly underestimated.

For tech screening, online tests are much faster to evaluate candidates.

Automated tests as tech screening

Since the code challenge have to be reviewed by humans. It is much better to filter with online automated test (coderpad.io ) . Main advantages are:
partial grading. Expectations are known. Different reviewers have different way to grading.
time efficiency for the company. No one has to correct behind the scenes. Candidate

No feedback, no review at all

With take-home assignments, rejection without feedback is common.
While if I do an in-person interview, I guess where I missed the answers. The HR always follows up with feedbacks.

You don’t know on what criteria you will be evaluated. Your application could be rejected on “unstated” trivia criteria:

– “we don’t care about problem solving, we only care about your code style.”
– use the wrong formatting tool. In the developer circle, I heard various stories: “I spent 8h on an Android tic tac toe project, they turned me down because they didn’t like the code formatting”
– use a popular library that is not known by the team
– use an architecture that the team is not familiar with
– deliver a high quality project, over-engineered for the reviewer
– …

Intellectual property

Imagine, you gave your best shoot and product of your work. You submit. No feedback, you are rejected.

However your code solved a business relevant to this company. You solve in a novel approach. They use your code in production.
You are still owner of your code, but you were not paid.

Only complete a challenge that a company has already solved in different ways long ago.

Why Top Tiers companies don’t use take-home tests?

Top tech companies FAANG (Facebook, Amazon, Apple, Netflix, Google) and other respectable ones in Europe ( Spotify, Zalando, Criteo … ), Asia (Alibaba …) DON’T give take-home assignments as a filter. There is not even a take-home coding challenge. There is no shortage of candidates willing to work for them who would take the unpaid assignment in a heartbeat.

After passing the HR interview (30 min) and first technical screening (45 min to 1h), the candidate is offered to travel on-site (flight+hotel+food). So why spend more than 4h+ only on the possibility to reach interviews? While top candidates invested 1h30 interview time and are compensated with travel and accommodation.

Take-home assignment is a huge opportunity COST. Qualified candidates will be turned off. For screening, a 1h interview is enough. Serious companies know that the better candidates won’t put up with crazy requirements like a lengthy take-home that most likely lands in black hope.

I once explained this situation to an HR lady: “We are a start-up, we don’t have the resources”. In admitting this, she hinted the compensation will be lower.

There are way more people applying there. The ratio of candidates to interviewers is higher. How do these companies find the time to interview candidates?

Look at their hiring funnel. The hiring funnel picture below is not at scale. First, notice the applicant pool is much larger and filtered for the HR screening. Second, the take-home challenge is replaced by a short technical screening.

Optimal hiring funnel

  • much larger applicant pool. At large companies, it is not uncommon that candidate profiles are filtered by machines ( ATS = application tracking software) based on keywords. To stand out from a large number of people, you should ask an employee (who knows you) to refer you.
  • HR: usual same friendly HR interview who will guide about the whole process. HR will prepare you for the materials and resources to study to ace the next technical/behavioral interviews.
  • Technical screening (45min to 1h): introduction of candidate and interviewer followed by one whiteboard coding, sometimes two (if you do well). If you performed one working solution that is not the optimal ( time and space complexity), you will get a 2nd technical screening. Depending on the companies, the success pass ranges from 5 to 20%.
  • interviews. 3 to 5 interviews based on algorithms, system design, and behavioral. A bit more difficult than screening. Generally, you are invited on-site, flight and hotel are offered by the company. In the Coronavirus period, the interviews are online.
  • the hiring committee will read the interviewer report. Depending on the headcount and availability, they will make an offer.

Hire the best, not the desperate

The best job candidates have choices and will not ignore those jump through hoops processes.
The desperate have time on their hands to complete the challenges. I would advise to work on personal demonstrable projects is better use of their time.

Whiteboard practice

From now on, I prefer to stick to whiteboard coding style interviews because they are time-boxed and scalable. Once I understand a problem, I never forget and re-apply the same principles elsewhere. The Leetcode efforts are amortized. If you believe that Leetcode is learning by heart, you are doing wrong. You need to understand the problem solving approach under the hood.

On whiteboard coding , the metrics are clear: the solution passes all tests, thought processes, efficiency in time and space complexity.

It is fun to experiment with a project with the latest libraries ( still in dev, alpha, beta version, I am looking at Android Hilt, Jetpack Compose … ). But it is better to understand the CS fundamentals used to build these libraries or frameworks. The algorithms and data structure are timeless knowledge.

Usually, employees working at companies without whiteboard interview might be good technician of a particular framework and know by heart the API methods. But they lack basic data structures (Map, Set) and CS fundamentals. Being on the hiring side, I interviewed such candidates coming from these companies.

Their solutions on the whiteboard are far from the most optimised, almost no notion of Big O notation. To the question: “Propose a solution to find an element in an array in constant time”. The candidates would use the built-in binarySearch in Java on an unsorted array. Instead of pre-caching the values of the array in a Map or Set. My colleagues unanimously said: “For sure, NO hire.”

What are your experiences on the hiring funnel for any companies that send take-home projects? Please comment below.

If you are on the hiring side, please post your funnel numbers (how many received tests /submitted/were invited for interviews)

PS: from others experience, companies with take-home assignment are only “interviewing but not actually hiring”.

Views from other industries

I once spoke to a friend who is actor, David . Each casting is a free interview. But the recording won’t be used for commercial purposes.

No actors, dancers will record a commercial scene for free. What you told me is :
Imagine you go to as many restaurants and tell them:
I will pay only the best after I sample each potential restaurants for free.”
Won’t you feel bad to about this to each restaurant? This is exactly what these companies are doing to you.

Dear fellow engineers, don’t get scammed, you are in demand. You have the power to say NO.

FREE WORK?

Ask other professionals for free sample. Try to ask a a locksmith to open your door for free? It takes a few minutes of their time.

Now developers are asked to spend hours on a single take home assignment. Do you think to it’s fair offer your time for free?

A few times per month, I receive a cold phone call from a recruiter. “We have this amazing opportunity. Would you like to hear more? … We ask for a small code challenge, it will take only 3 hours”. After a quick glance, the project is a full-fledged app with pixel perfect Sketch designs. The assignment asked for unit tests, integration tests, complete CI/CD pipeline, [docker & kubernetes]. It would take at least a week to complete. My response: “Listen , you are the ONE who contacted me. I never applied.”
And your response should be:

Hi [recruter],
To be perfectly honest, I'm really excited about [company] and got a great impression of the company from my interview with [you or engineer]. I estimate this assignment will take at least H hours of work. My contract rate is €$ X/hour. I can complete this assignment for [total] and return it to you completed by the end of next week. If that isn't acceptable, I do have an extensive portfolio of my work available on github (link). Please let me know how you would like to move forward.
Best,

You could probably charge for 165 USD/h for 20 like this guy did.

Even Free work from reputable Firms?

Reputable firms are classical banks (not online banks), retail stores or any companies with 9 digits annual revenues.

I submitted code to a firm. The reviewers were impressed but the firm wanted more features, it would take 2-3x times. I politely declined. The manager invited me for the rest of interviews. So the code reviewer was not even important enough to be part of the interview.

Another time, I sent the project. The company sent feedbacks, but the position was refilled by an internal employee.

Or the best, I inserted analytics in the project and readme.html ( not .md file), so that when the documentation or project are opened, I’d be informed. The company never opened and sent a blank template “no pass”.

Most likely, you will gain nothing from completing take home assignment except adding to your public portfolio. My repeated and peers experience is that simply being asked to do a “take home project” is a strong indication that companies are cheating and not actually hiring.

Survey

Please share your experience in this survey

The goal is not to shame and name the authors. But to get an overview in the hiring funnel and share the fair players in this coding homework game.

 

IntelliJ bigger Font size

At any meetup, conferences, the speaker shows the code on the large. People in the back struggle to read what is on the screen.

Usually, the advice is Ctrl+plus , or Ctrl + mouse wheel up on Mac or PC. But this applies only to the current file (not a global setting).

Recently (yesterday) I had a hangout call, where I had to show code on my IDE.

The Smart Way to increase Font size

In order to keep the same font size global, go to IntelliJ IDEA > Preferences

Preferences > Editor > Font > Size . Enter the new font size

Don’t forget to press “Apply”or “OK”

After the presentation you can revert to the previous font size.

 

Tutorial: change Color Scheme on IntelliJ IDE

You found a nice theme packaged in a .jar file (eg. from http://color-themes.com). Now you wonder how to change your IntelliJ based IDE to the new colors.

A long time ago, I wrote how to change the Theme in IntelliJ based IDE . The word “Theme” is misleading for Scheme (read below) and Theme (Preferences > Appearance & Behavior > Appearance > Theme).

In MacOS, the Preferences bar can be accessed at the top left corner under the application name or shortcut “cmd + ,” . Please post in the comment, how to in Windows or Linux.

In order to change the Scheme, go Preferences > Editor > Color Scheme > General.

On the cog icon circled in red, select “Import Scheme” and select the .jar file.

select the color scheme .jar file

And voilà , the look & feel of Android Studio Dark Kotlin

Now if someone has an idea how to import these themes from JetBrain, I am curious.

 

AWS Summit Paris 2019

AWS ( Amazon Web Services ) dominates the Cloud industry with the breadth of services ( over 100+ ) and is way ahead Google Cloud and Microsoft’s Azure.

If an AWS Summit is in your town and go ahead, it is well worth it and free. The keynotes , sessions are really enriching. It’s great way to network with the local players. The AWS partners range from cloud consulting to service to tooling providers. You can expose your cloud problems to an AWS Solutions Architect.

I went to the keynotes just for Adrian Cockcroft, ex-Netflix , now VP Cloud at AWS .

Alexa

The sessions were enriching especially the “Alexa design” by Benoit Nachawati. The arrival of Dynamic Entities helps to personalize Alexa Skill experiences by adapting the interaction model at runtime without edits, builds, or re-certification. Basically imagine that Alexa can only accept a finite Set of words , say a drink : coffee, latte, tea … in the utterance. The user says “water” , and it is not a defined value, your Alexa Skill can remember this word for later use.

Startup Central

An area was dedicated to startups . I went to see the architecture of 8 finalist startups. Serverless was used by all of the finalists, maybe a condition from AWS to promote this technology.

 

Droidcon Greece experience

Last month Thessaloniki welcomed the first Droidcon in Greece. It was also my first time in Greece, and I was fortunate to be invited as a speaker. Aside from the high quality of the conference itself, the organizers (special thanks to Eliza Camberogiannis ) did a great job of spoiling us. It started at the airport, where a taxi was waiting for me. On Droidcon Eve, the other speakers and I were given a bus tour of Thessaloniki.

Droidcon speakers in Thessaloniki, under the Arch of Galerius

Droidcon speakers in Thessaloniki, under the Arch of Galerius

We stopped at a Bougatsa shop around the Byzantine Walls and tried both the salty and sweet versions of the pastry; my vote went to the sweet Bougatsa with cinnamon, pictured below :

Sweet Bougatsa

Sweet Bougatsa

Before lunch, we visited a local market and tried some mezze (ham, cheese and the biggest olives I’ve ever seen.) Then we were treated to a seafood lunch where, again, we experienced that extremely good Greek hospitality and cuisine.

The talks I liked the most

Droidcon took place at Noesis ( ΝΟΗΣΙΣ ), a science park on the outskirts of Thessaloniki. All of the talks were interesting, but these takeaways were the most memorable:

All the talks were interesting , some were more memorable.

Damien Mabin, Google developer advocate and former game developer , gave advices for game monetization :

    • Even if an user spends st 1$ in your game it’s still more than what you would get from him watching ads in your game/app. As soon as the user spent money on your app, remove the ads, he gave you enough money. The whales (big spenders) still generate more revenue the average user who spends less than 5$. Duet cleverly uses this strategy.
  • Damien also noted that time-constraint challenges incentivize players to spend more time in the game; these challenges generate a nice surge in traffic from the same users. (This point applies to non-game apps as well.)

talk_google_damien_mabin_side

Developer and evangelist Svetlana Isakova of Jetbrains (editors of Android Studio) demoed Kotlin, Jetbrain’s homegrown programming language. Nicknamed “the Swift of Android,” Kotlin is considered to be less verbose but more readable than Java, and it supports immutability, nullable types and lambdas. The easiest way to start using it is by automatic conversion of Java file to Kotlin file, done with an Android Studio/IntelliJ plugin. Java can interoperate with Kotlin methods and vice versa. In one of my pet projects, I’ve found Kotlin to be very concise for POJO classes with setter and getter methods.

talk_kotlin

Svetlana Isakova

Josh Skeen, an instructor at Big Nerd Ranch, led a workshop on RxJava — something I was really looking forward to. Rx stands for Reactive, and you may have heard of Retrofit + RxJava in combination. Josh explained the fundamental concepts of Rxjava, a library for composing asynchronous and event-based programs by using observable sequences. We tried to solve his RxJava Koans in order to reach RXJava enlightenment.

talk_rxjava_josh

Savvas Dalkitsis from Shazam discussed how aspect-oriented programming can isolate code related to analytics and ads from your business logic. It reminded of our code base at Zalando, which uses a similar technique (tracking) for analytics.

Special kudos to the most productive speakers : two French men. Damien Cavaillès presented three talks and Damien Mabin did two talks.

As for my own talk (my second Droidcon presentation — I got to speak at Droidcon Berlin earlier this year), I focused on “App Fails and Retrospectives.” The topic didn’t quite fit the SDK track, which was where I appeared, in that it mostly offered failure-related anecdotes based on real-world Zalando experiences. Judging by the audience’s reaction, our “best” failure was the time our app was about to become the Editor’s Choice on Google Play, and on the same evening I had to revert to the previous app version because of an untested crash that was going to affect millions of users.

I came up with the idea of “epic fails” from an active Zalando group chat called “#guild-fuckup,” one of the most active of Zalando Tech’s +100 guilds. I found inspiration there the day before the Droidcon Berlin deadline, submitted the talk … and was accepted without sponsorship! I couldn’t offer expertise on trendy topics: Internet of Things, wearables, Android Auto, but failure and mistake-making was something I thought every dev or dev team could relate to, big company or small startup.

All Good Things come to an end

After the very last talk, we took a “family” photo and finished the event with a Greek dinner (of course).

Droidcon organizers and speakers

Droidcon organizers and speakers

Before leaving Greece, I bought a Terkenlis chocolate cake (tsourekia) for my team in Berlin. A Greek colleague recognized the cake and thanked me for bringing the best back from Thessaloniki. I thought it was the least I could do for all the hospitality I enjoyed there! It was definitely worth the trip.

Originally posted at https://jobs.zalando.com/tech/blog/my-droidcon-greece-experience

 

How to change UI theme on IntelliJ and Android Studio

This post is deprecated, please read updated this tutorial how to change the color scheme from .jar file.

If you are tired of the default look or darcula theme of IntelliJ (tested on 2017.x and 2018.x versions) or Android Studio , you can switch to a better looking theme . This is a very short process

idea-obsidian

IntelliJ with Obsidian.jar theme and darcula

1 Select your theme

color-themes.com has plenty of themes ( Sublime Text 2 , Monokai,) . Download the .jar file . Roboticket.jar is my current theme

 Import the theme

Import to IntelliJ or Android Studio by going to File -> import settings -> and select the  .jar file you downloaded.
Select All . press OK. Then Restart your IDE.

idea_color_scheme

Note : if you use darcula theme previously , you will notice the new theme is only applied to  the Text Editor. The light theme clearly contrasts

idea-darcula-roboticket

3 Appearance theme

On Mac , go to Preferences -> Appearance  . You can also use the shortcut Preferences -> search theme on the top left search bar
Next to Theme , Select Default . Or Darcula if you are using a dark theme .
Restart your IDE.

idea-appearance-dark

4 The final look

 idea-final

Roboticket.jar

 

Raygional 3.0 with notification

A user contacted me this year. Due to his visual impairment, he uses Android Talkback. He found Raygional useful for switching the system locale. He asked me if I could build with notification, so it can be easily accessed on the system menu.

Battery by MacroPinch is an app I find really useful. The app is really useful to indicate the battery charge level. I really like the notification. My inspiration for Raygional comes from Battery.  I reversed engineer Battery and I found for each level , there is 3 icons ( one for GingerBread, one for Honeycomb, one for others ) , so there are at least 300 icons just for the status notifications. The programming of Battery is not so difficult, the main attraction is the graphic . BTW I’m looking for graphic designers, anyone ?

Because Google Keyboard does a great job to switch the keyboard settings for message typing, I did not use my Raygional as often. With my day job, if I did not set timeboxed goals. It took FOUR months , that’s way too long. During the last few weeks , I started to get interested to update the app. Today I released Raygional 3.0, it feels nice to use the notification.

raygional30

The app is only compatible from Android 4.0 because the UI uses the latest updates . To make compatible from Android 2.1 , I have to import the Holo theme and ActionBar UI pattern , I’ll have a look at the latest Action Bar Compatibility which the Google ‘s answer to ActionBarSherlock .

 

Thanks Peter

 

Github library : Android-TextFontStyle

The designer studio where I work has a tendency to use special font in the text  because it’s eye-catching.

In others previous projects, I saw the font is applied programmatically. It’s difficult to maintain each TextView because you have to keep track in the java code.

For design purpose, I prefer to work in xml.
Style in Android works like Microsoft Word style : define a style and apply it to a TextView. Any change  to the style will be taken in account the TextView.

I hope more developers use typography as the app really stands out. Maybe Steve Jobs was right about the importance of typography.

And we designed it all into the Mac. It was the first computer with beautiful typography. If I had never dropped in on that single course in college, the Mac would have never had multiple typefaces or proportionally spaced fonts. And since Windows just copied the Mac, it’s likely that no personal computer would have them.

The library is available here.  https://github.com/raychenon/Android-TextFontStyle

screenshot