AI Magazine - Spring 2018

the users to validate the top 10 skills per resume ranked by relevancy scores. To measure recall, we ask the users to add up to five skills that were missing from the presented list.

The results show that the current skill tagging framework attains 90 percent precision and 73 percent recall, which is better than the 82 percent precision and 70 percent recall of the previous version (Zhao et al. 2015), as noted in Table 4. Moreover, a strong correlation between the relevancy score and the user approval rate is observed. Table 5 shows that the higher the relevancy score, the higher the chance of approval by users.

Technical Design Overview Our projects require a successful synergy between data scientists and data engineers to move from prototype to production. The SKILL system is one of the first projects at CB that demonstrated successful collaboration between the two organizational teams. Projects with a strong data science component initiate with the data scientists conducting R&D research spikes to build prototypes to verify the feasibility of business ideas. The data engineers are involved in technical design discussions as soon as it becomes evident that the prototype can move to production. The synergy between the two teams is critical because it is important to understand the limitations of open-source tools used by data scientists in production environments. These limitations sometimes influence the tools used by the data scientists, but in general, we do not place hard restrictions during the prototyping phase.

For the SKILL system, the goal was to provide a skill tagging service (The SKILL service) as a microser-

Table 3. List of Tagged Skills of the Resume Sample Presented in Table 2.

Raw Term Normalized Term Relevancy Score Type software development Software Development . 95 Hard Skill machine learning Machine Learning . 95 Hard Skill automated testing Test Automation . 93 Hard Skill Android Android (Operating System) . 92 Hard Skill java Java (Programming Language) . 90 Hard Skill python Python (Programming Language) . 88 Hard Skill mobile app Mobile App . 87 Hard Skill unix Unix . 85 Hard Skill api Application Programming Interface . 85 Hard Skill biomolecular engineering Biomolecular Engineering . 84 Hard Skill chemical engineering Chemical Engineering . 84 Hard Skill mandarin chinese Mandarin Chinese Language . 82 Hard Skill

Figure 2. Sample Documents Representing the Two Senses of the NLP Skill Term.

Each colored box represents a set of related skill entities extracted from resumes or job postings.

AI CHARACTER_RECOGNITION
SENTIMENT_ANALYSIS
WEB-ONTOLOGY NEURAL_NETS
MACHINE_LEARNING TEXT_MINING
LANGUAGE_MODEL STOP_WORDS

5_STEPS_CALES_PROCESS
PUBLIC_SPEAKING PRODUCT
SERVICE APPLICANT_TRACKING
SYSTEMS ACCOUNT_MANAGEMENT
LEADERSHIP
NLP Natural
Language
Processing
NLP Neural
Linguistic
Program