Unlocking Program Health: Using Data to Strengthen Credentialing Programs
Insights drawn from webinar “Unlocking Program Health: Using Data to Strengthen Credentialing Programs,” presented 3.19.2026
Video: https://vimeo.com/1175314670?fl=pl&fe=sh
Credentialing organizations collect an enormous amount of data:
- Application data
- Exam data.
- Candidate performance data
- Recertification trends
But data alone doesn’t improve a certification program. What matters is how you use it.
In our recent webinar, “Unlocking Program Health: Using Data to Strengthen Credentialing Programs,” Prolydian President Michael Mayer moderated a discussion with psychometric experts Ian Hembry and David Chen from Quadterion on how credentialing organizations can use data to evaluate and strengthen their programs.
Their message was simple: Data doesn’t have to be complicated, but it must be intentional.
When Data Is Invisible, Problems Stay Hidden
Many credentialing programs operate across multiple disconnected systems. That fragmentation can make it difficult to see what’s really happening inside a program.
As Mayer shared about a previous client during the discussion: “They really didn't have visibility into their data. So, they quite honestly didn't even know what questions to ask about their certification program.”
Once data becomes visible and centralized, organizations may discover insights they didn’t expect:
- Candidates abandoning applications mid-process
- Confusion around certification pathways
- Operational inefficiencies increasing program costs
When programs gain visibility into their data, they can ask better questions and make better decisions.
Data Should Start with the Right Questions
One of the most important points raised in the webinar: data collection should always begin with purpose.
Organizations sometimes gather large volumes of information without clearly defining what they want to learn.
Hembry explained, “The ability to ask informed questions is crucial… whatever data you're collecting should always tie back to some sort of an objective or an outcome.”
In practice, that means every data initiative should connect directly to two things:
1. The organization’s mission
2. The specific decision or outcome the organization wants to evaluate
Without that connection, data risks becoming noise instead of insight.
Data Is a Team Sport
Effective data use isn’t the responsibility of a single department.
Credentialing programs benefit from bringing together multiple perspectives, including:
- Strategic leadership
- Operational staff
- IT or engineering support
- Psychometric and research expertise
- Subject matter experts
These perspectives help ensure that the organization asks the right questions and interprets the results correctly.
Hembry clarified, “Executive people are good at strategy…[whereas] people like David and I are good at coming up with research questions.”
Strong collaboration ensures data efforts remain aligned with both operational realities and strategic priorities.
What Does a Healthy Credentialing Program Look Like?
The panel highlighted several indicators organizations can monitor to evaluate program health.
- Pass Rates: Changes in pass rates can signal shifts in candidate readiness, eligibility requirements, or exam difficulty
- Exam Item Performance: Metrics such as item difficulty and reliability help ensure assessments measure competencies accurately
- Recertification Trends: Monitoring how many certificants maintain their credentials can reveal whether recertification requirements are realistic
- Content Relevance: Certification exams must evolve as professional practice evolves
David Chen noted: “Exam development is an ongoing process… items should always be checked to make sure that they aren’t changing over time.”
Data Doesn’t Have to Be Complex
Another key takeaway from the discussion: you don’t need advanced algorithms to start using data effectively.
Simple tools and visualizations can provide meaningful insight.
For example, plotting item difficulty can immediately show whether exam questions are too easy or too difficult, helping subject matter experts review and refine the assessment.
Chen explained, “Sometimes the best way to approach these tools are just with the most basic means instead of overly complex terms.”
Often, the most valuable insights come from clear, simple analysis.
The Hardest Part: Listening to the Data
Sometimes the most important lesson is accepting what the data reveals—even when it challenges assumptions.
Hembry shared this reminder: “You need to be willing to listen to the data… sometimes data speaks back, and it's not what you wanted to see.”
That humility is essential for organizations committed to continuous improvement.
The Big Takeaways
From the conversation, three ideas stood out:
1. Data-driven decision-making is collaborative. It requires input from leadership, operations, technical experts, and subject matter experts.
2. Healthy programs align data with mission. The most valuable evidence supports the certification program's purpose and credibility.
3. Start small by focusing on the questions that matter most and building from there.
Credentialing programs operate in complex and evolving professional environments. By asking the right questions and using data intentionally, organizations can strengthen program integrity, improve candidate experience, and ensure their credentials continue to deliver meaningful value.
If you're interested in learning more about how data can support the health of credentialing programs, feel free to reach out.
Comments