Frequently asked questions
What’s the level of statistical confidence for Iris?
We determined that the goals of Iris are accuracy and getting a complete picture of a skill blueprint. Determining confidence gets into the technicalities Item Response Theory models (which are the industry standard methods) for calculating item difficulty, certainty and guessability. If we apply Item Response Theory modeling to our assessment questions, we find that with our questions (even at relatively low levels of characterization) have high levels of correlation between our unique parameterization models and the industry-standard IRTs. Recalling that our models allow item characterization with little or even no responses, this is solid evidence that Iris has high levels of accuracy.
How is Skill IQ (Iris quotient) different than a traditional intelligence quotient?
Skill IQ is not related to intelligence quotient. Skill IQ measures a user’s technical abilities relative to the population of users that take the assessment, and does not measure a person’s broad intelligence.
Who creates the questions in an assessment?
Questions are created by subject matter experts (SMEs) selected by Pluralsight. These SMEs are elite technologists who speak at conferences, author Pluralsight courses and offer years of professional experience in their fields. On average, we accept less than ten percent of applicants looking to join our SME pool. Before any questions are written, SMEs create an outline of the objectives and sub topics that make up the breadth of a skill. SMEs then write questions to ensure coverage of the outline.
Are questions reviewed for accuracy?
Yes, questions undergo a review process where other SMEs check for accuracy. Questions are then reviewed by the Pluralsight curriculum team for grammar and clarity.
Are questions reviewed for relevance?
Yes, questions are designed to cover objectives. Objectives are periodically reviewed by the Pluralsight curriculum team and our SME pool for accuracy and relevance.
Does the content of questions come directly from Pluralsight courses?
No, questions are written independently from Pluralsight courses.
What is the distribution of question difficulty in an assessment?
Questions are designed to cover a range of difficulties with a goal of having a normal distribution of question difficulties; there will be more questions of moderate difficulty but fewer at the easy and hard difficulty levels.
Are questions weighted toward certain proficiencies within a skill?
No, questions are not specific to certain proficiencies unless specified in the assessment title. For example, there are multiple assessments in Photoshop to cover the different applications of the product.
How many questions are in the question pool for assessments?
There are anywhere from ~150 to ~300 questions in the pool for an assessment.
Why does a user receive the same or different questions during a retake?
A retake starts a brand new assessment session. Because assessments are adaptive, a user may get the same or different questions than their previous session depending on the question pool and how they answer. It’s possible to get all new questions or repeated questions during a retake.
How can a user report an incorrect or out of date question?
Technology changes fast and this can impact the accuracy or relevance of questions. Users are encouraged to provide feedback on questions when they believe a question or answer is incorrect or irrelevant. Feedback can be provided during and after an assessment by going to the “review questions” screen and clicking “Give feedback.”
What happens when a user submits feedback about a question?
Feedback is directed to the Pluralsight curriculum team which will pull in the appropriate subject matter experts to make the necessary edits and updates. The current feature does not have a way to confirm with the user that the feedback submitted has been addressed.
What is the outcome of the assessment?
Each result includes an IQ, a skill level, a percentile rating for the technology skills and a date of verification.
What do novice, proficient and expert mean?
A Skill IQ is a continuous score from 0 to 300 that is based on your percentile. Your percentile (between 0 and 100) tells you the percent of all people with this skill that you should expect to know more than. The skill rating range (0-300) is broken into 3 buckets that contain the 1st-20th percentiles (Novice), the 21st - 80th (Proficient), and the 81st and higher (Expert).
Is Skill IQ linearly distributed?
No, Skill IQ is normally distributed across the population of users that take the assessment.
What does a Skill IQ represent?
When you see a Skill IQ from Pluralsight, it means that you have third-party verification of your skills. When you see a Skill IQ on a Linkedin profile, you can trust that the person has done more than just complete a course. You'll know their exact skill level.
How is a Skill IQ different from certificates or badges in the market today?
Skill IQ is the fastest, easiest and most accurate measure of technical skill today. Fueled by questions from technology experts and built with high statistical significance, you can trust the results. Other options give badges for course completion or points for activities. A Skill IQ is a numeric assessment of skill.
How is a Skill IQ different from vendor certifications?
A Skill IQ can live side-by-side with vendor certifications. Skill IQ is less time intensive and less expensive than certifications. It makes skill assessment on different technologies far more accessible and more current. As technologies evolve, users can reassess themselves to keep their Skill IQ a current representation of their skill.
How do users get help to improve their skills?
Users can improve their skills by joining Pluralsight and starting the path for the skill they are most interested in pursuing.
Can I use Skill IQs for hiring, firing and promotion decisions?
Pluralsight’s skill assessments are not designed to be used as a candidate screening tool in the hiring process. While the results of Skill IQ are useful to motivate users to learn and benchmark abilities within an organization, employers should be aware that Pluralsight does not support a standard set of features that are commonly perceived requirements for screening tools. As such, Pluralsight does not recommend using Skill IQ as the singular measure of hard skills in the hiring process.
If a user has been issued a business Pluralsight license, and has activated that license, a business may choose to use their Skill IQ and Role IQ levels (Novice, Proficient, or Expert) in decision making. If a user does not have a paid and activated Pluralsight license, they may sign up for a free, personal account and choose to share their Skill IQ and Role IQ with their current or prospective employer.
For other employment-related decisions, Skill IQ can be used as one of many data points and considered in context of other information the employer has available on the employee.
How does Iris calculate Role IQ levels?
Iris calculates Role IQ levels by collecting skill data from learners who complete each role and uses it to model the thresholds of proficiency required for each skill in a role at each level. These thresholds are then used in a predictive machine learning model to assign each learner to the correct level when they complete all the required assessments.
