Why I Created an Institutional Review Board at Pluralsight

I recently spent the last few months getting a federally registered Institutional Review Board (IRB) together at Pluralsight. If you haven’t conducted scientific research with human subjects in a federal, academic, or clinical setting before, you’ve probably never heard of this entity. If you have, then you’re probably familiar with their importance. 

For those who are unfamiliar with IRBs, an IRB is an institute’s independent research ethics committee, tasked with providing ethics training, review, and oversight on all human subjects research that is scientific and generalizable to the broader population. The creation of an IRB is highly standardized: they are maintained by the US Department of Health and Human Services, and every IRB is registered at the US Office for Human Research Protections. 

To receive an external stamp of validation that they are conducting ethical human subjects research, scientists who conduct research with human subjects in federal, academic, or clinical settings must receive IRB approval for every research study they conduct before they collect any data. 

While IRBs are required in academic and clinical institutions, they aren’t required in industry settings. And some industry scientists love this. After all, can’t we just keep ourselves accountable? Why go through the hassle of filling out the paperwork to get every study reviewed by an ethics board? Why deal with delays in collecting data as you wait for approval? Why spend months formally creating and registering IRB policies, guidelines, and procedures? Why go through the hassle of finding and convincing five other people that have the technical and scientific knowledge of human subjects research and ethics to serve on your board, where they will have to commit to assessing research studies, providing substantive feedback about potential ethical violations or concerns, and stay up to date on ethical standards and issues?* It’s not like we’re evil! We’re just scientists!

---

*Note. To assess research plans with the strengths of diverse viewpoints and provide a counter to potential bias, IRBs consist of at least five individuals and should contain a mix of field-relevant scientists and community members, at least one of which does not have any ties to the organization. IRBs must also contain a mix of gender identities.

Here’s the thing: in this world, science has the power to significantly impact society’s values, beliefs, norms, and behaviors. It’s why we learn about ethics early on in our training, why we learn to consult with colleagues, and to be careful, methodical, comprehensive, and detail-oriented. Because our mistakes can irreparably hurt people. We know this because, quite frankly, there have been many many instancesin which this has already happened. Take, for example, the Tuskegee syphilis studies, the Guatemala sexually transmitted infections studies, and the Minnesota Starvation Experiment - all research endorsed by the government and the scientific community at the time, while causing horrific damage to people’s lives. In fact, these events are the reason we even developed IRBs in the first place. 

Even though most research will never be anything like these terrible examples, it’s still important to design even seemingly innocuous projects to uphold the ethical responsibility and duty of care that researchers should have towards participants. For example, some study methodologies in psychology have studied how people respond to failures by misleading participants, giving them a false score in a game. A proper study protocol would include making sure that participants are “debriefed” afterward–given the correct information so that they are not taking away a false negative impression of their own skills.

So while it’s easy to dismiss the most egregious ethical violations in science as “bad things done by bad people,” the reality is that most ethical violations emerge as a result of unchecked biases and extenuating circumstances, not some sort of evil scientist plot. This makes all of us prone to ethical violations because we all have biases that we bring into our science.

I have said it many times before and I will say it yet again: there’s no such thing as bias-free science. This is, in part, because the questions we ask and the ways we choose to ask and answer them are inherently biased by who we are as scientists and the cultural contexts we live in. Put simply, science is biased because we, the scientists, are biased. 

Now technically, everything humans do is biased. And although we all like to conveniently forget about this fact (in a rather biased manner, I should add), the fact remains that each of us brings our own biases, contexts, beliefs, and experiences to the metaphorical table. This isn’t a bad thing. After all, it’s part of what makes us so delightfully diverse as humans. It is, however, an extremely important thing to acknowledge, explore its impact on, and attempt to mitigate in science. And this is easier said than done. Because as is typical of bias, it can be nearly impossible to see our own biases.

This is where having an independent group of people reviewing your research can be helpful. While scientists adhere to a set of ethical guidelines (in the United States, this is called the Common Rule) regardless of an ethics board, ethics boards provide another set (or five) of eyes to ensure that scientists aren’t accidentally breaking or bending a part of the guidelines. 

For example, the first principle of the Common Rule is to respect participants’ autonomy. This means that participants give voluntary consent to all research procedures and are given adequate information about the purpose and procedures of the research study. Our Pluralsight IRB ensures that we at the Developer Success Lab follow this by making sure that our consent procedures cover everything about our study that is important to our participants. This is helpful because sometimes, something we think is an unimportant procedure might actually be worth mentioning to others.

The second principle is to do good and do no harm. This means that scientists should maximize the benefits of participation for subjects and society, while minimizing the risks to participants. That is, studies should do more good than harm. Our IRB can be helpful here because scientists tend to love their research like they’re our lil’ brain babies, which means that we’re biased to think our studies are worth doing, even if they may not be worth the risks. 

The third principle is to distribute the risks and benefits of research equally across populations. This means that scientists don’t only work for the good of one group within a population of interest over another. For example, our IRB can make sure that we aren’t always only collecting data from Pluralsight developers because this places an unequal burden on them of being our research subjects, as well as gives them an unequal benefit if we are testing a benefit or intervention - something that you can do as an organizational researcher for a company, but not as a scientist creating generalizable knowledge for the field. 

Finally, the Common Rule requires that scientists keep data secure and confidential. This means that raw data is only used for the purposes of the proposed research study, is de-identified before published, and is kept confidential and secure. Our IRB ensures that we only collect relevant identifying information, and that we keep our data secure, confidential, and separate from the rest of the organization. This means that we don’t use the raw data to identify customers or share information about them to others.

Ethics boards, of course, aren’t immune to failure. After all, they are composed of people who bring their own biases and cultural contexts to each review. Although efforts should be made to create a diverse board to reduce bias (ours is), this doesn’t always happen. Despite this, because ethics boards don’t have a vested interest in a study being run, they represent a less biased perspective that can help scientists mitigate biases and ethical violations overall. This is particularly important for scientists wanting to move beyond general survey research and deeper into human experiences, intervention science, and experimental methods. The question most industry scientists and their organizations thus face is: “Are these benefits worth the extra work?” For us, the answer is yes! 

Setting up an IRB took some detective work, but it was ultimately surprisingly easy! Here are the steps we took that we encourage other research teams to consider:

A board requires at least 5 members. We recommend identifying a culturally and professionally diverse group of scientists and community members both within and outside of your organization. Make sure that your IRB chair is someone with significant experience evaluating research and ethics. In the event that your IRB chair has a conflict of interest with any piece of research, also identify an acting or alternate chair.

Don’t expect everyone on the board to remember every nuance of the Common Rule, the types of reviews required for different studies, and the different roles and duties of each person on the board. There’s a lot to it! Write everything up for your board members and scientists so that everyone is clear about the process.

Create an IRB application and submission system for scientists and a review system for the submissions.

Create training modules for scientists and board members to refresh their knowledge of ethics and the Common Rule. If applicable, include training on the international Good Clinical Practice (GCP) guidelines.

To ensure everyone is on the same page, meet with your panel to discuss the handbook, policies, training, and procedures.

You can register your IRB at the Office for Human Research Protections website. The process is quick and completely online. Once you’ve been approved, you’re good to go!