Boj

How might we improve the career and internship search experience for students?

[User Research] [Mobile Application]

My role

I worked with three other UX researchers on this project.

We were each in charge of more than one aspect of this project. I was in charge of the survey design and analysis, concept testing, as well as the usability testing.

We worked closley with the developers and CEO of Boj App on this project. 

RESPONSIBILITIES /
User research, interviews, concept tester, report writer, survey creator and analyst

TIMELINE /
3 months (Jan-Apr 2019)

TEAM /
Rochelle Campbell - UX Researcher
Shih-Ting Huang - UX Researcher
Corbet Griffith - UX Researcher
Carlo Sugatan - UX Researcher

Boj is a startup company that matches job and internship seekers with companies that match their culture and values.

Boj is going through a major redevelopment of their product in order to succeed in the job-service industry.

Problem Scope --

Boj faces many user drop-off rates in their application. They want to understand the reasoning behind this through a dive deep into their product and usability. 

We worked with Boj to help them transition to their new and reimagined product by working with potential job seekers to understand their motivations for using  job matching services. 

Boj has a new idea... 

Boj has an idea to implement interactive games to encourage users to complete their profile or as a way to make the job or internship process less stressful.

The challenge

• How can we help Boj transition to their reimagined product with success?
• How can we increase the user base of both job seekers and companies for Boj?
• Is their new plan with adding gamification effective?

Impact --

Boj streamlines the job search for both college students and companies in order to match one another based on cultural fit, education, and skills. This allows business find the best talent effectively while also ensuring that students are working with companies that match their values.

But, how did we get here?

We used various reserach methodologies in order to identify ways to improve Boj. 

We focused on the needs assessments. We did this through interviews, a comparative evaluation, a survey, and a concept test to better understand users’ behaviors, wants, and needs when using a job-matching service.

Interviews --

We interviewed college students and recent graduates about their values in looking for a company to apply to and about their pain points in job searching.

• We saw that our participants valued location, company culture, and salary the most. The most cited pain point is the lack of communiation with recruiters. 
  
  
• Our participants are eager to use a service to have streamline the job searching experience, but haven't found the right one yet.  

We were able to create two personas of one student and one recent graduate to help us inform and guide the rest of the study design. 

Comparative Evaluation --

We investigated different competitors within the job seeking industry to get a better idea of industry’s landscape.

• We found that most job service applications provide free services with only Indeed and LinkedIn providing premium versions.
  
  
• Surprisingly, we saw that even if the job-service application offered more perks (application tracker, salary research, one-click applying), it also offered sophisticated functionalities (sign-in using Google account, importing resume information) as well as more usable interfaces (steps required to create a profile).
  
  
• We found that most job-service applications offer both a mobile and web application.

Survey --

We deployed a survey to quantitatively measure people's behaviors and choices in their job search journey. 

• Most use their computers when looking for a job or internship and prefer a web application.
  
  
• Users are still unlikely to pay for a premium service even if it did offer extra benefits.
  
  

• Implementing interactive games in a job-matching service application had negative reviews (statistically significant at p<0.05).
  

Concept Test --

We deployed a concept test where we asked users how they feel about different gamification features. This allowed us to dig deeper as to why gamification may not work for Boj. 

• Most participants expressed the most interest in assessments.
  
  
• Most participants expressed disinterest in implementing badges in a job-matching service.
  
  
• Participants are less likely to spend time completing games (1-5 minutes) but are willing to spend more time with assessments and personality quizzes.
  
  
• Many participants expressed concern over biases and discriminatory results of implementing gamification in a job-matching service.
  
  

 
The gamification features we included on the concept test were:

• Skill assessments
• Badges
• Neuroscience-based games
• Personality questions

Heuristic Evaluation --

We conducted a heuristic evaluation using Jakob Nielsen's ten heuristics to conduct the comprehensive usability of evaluation. 

We evaluated the app individually with a ranking system to prioritize the major issues based on the severity. Afterwards, we conducted an interpretation session to identify and consolidate the the most-pressing issues and key findings, and to brainstorm solutions.

• Users are not presented with error messages
  
  
• The only documentation for help, or “tips” is difficult to navigate
  
  
• Users have to re-enter some selections if they made a mistake or need to add additional selectionsMost participants expressed the most interest in assessments.
  

Usability Test --

We conducted a usability test to investigate the salient usability issues around the current Boj job-matching application. 

 Boj currently faces high fall-off rates during this process so we wanted to investigate these reasons so they can improve on these issues in the redevelopment process. We created four tasks centered around profile creation and exploration. Upon completion, we asked participants’ feedback about the app. 

• Users spend too much time scrolling through options when creating their profile
  
  
• Users are overwhelmed with selecting multiple options (choosing location, industry, roles)
  
  
• Modifying user profiles is confusing
  

Overall Recommendations --

After a deep investigation of the Boj mobile app, many interesting and unexpected insights were revealed.

After synthesizing the results from each methodology, we summarize the high-level recommendations as:

• Keep Boj free to use
  
  
• Have a web application available to be used aside from a mobile application
  
  
• Improve error prevention, and reduce profile questions.
  
  
• Improve usability of profile editing & make profiles more customizable
  
  
• If gamification is implemented, it should be used to have candidates understand their skills rather than a measure of compatibility with a company or role
  

Listen to the users!
Many users did not favor implementing gamification features onto a job-search application. Boj had this idea and was supported by others who are not the target population. By understanding the wants and needs of users, we were able to show how Boj should reconsider adding gamification features, or to use it to have users understand their skills, rather than something to be used by employers, in which case, has the potential to be used against them.

Use different methodologies to reveal different insights
It was useful to go through different research/UX methodologies to reveal different issues and learn about the users. Doing a survey wasn't useful for gauging the potential of gamification featuers, in which a concept test was proven more useful. 
 

Next steps
We only interviewed one part of the users base, which was job-seekers. However, companies would also be using Boj to upload their job searches and to decide on how to pick a candidate. Further reserach should increase representation of the product. In addition, further research should look into the benefits and negative consequences of gamification on the product, maybe using participatory design.