Back
The Scenario
Our client developed a laser treatment system targeting acne vulgaris, a common dermatological condition affecting millions worldwide. The laser treatment was considered by the FDA as a Class II device, requiring clinical trials to demonstrate safety and efficacy. Study investigators administered the laser treatment and were therefore not blinded to subject identity, needed for unbiased assessment of the treatment. An independent review of the evidence was thus required to determine treatment efficacy.
During the studies, subject photos were taken as evidence before and after treatment, including controls. The photos captured changes in acne lesion counts that were used as the primary outcome measure. For demonstration of treatment efficacy, our client had to rely on the independent reviewers' subjective assessment using photos.
The Challenge
Assessing acne in digital photos demands expertise and experience due to its inherent limitations. Photos yield two-dimensional representations with fixed lighting, which means that evaluators need to translate what they see back into clinical evidence, including lesion mimics that can falsify or obscure results.
Four experts were needed to provide sufficient strength to the evidence review. Optimizing inter-rater reliability and documenting their expertise relevant to the task at hand posed additional challenges. With hundreds of photos requiring assessment, a streamlined approach was needed to capture expert assessments accurately and consistently while minimizing bias.
The Solution
In collaboration with our client, we devised a comprehensive solution addressing the outlined challenges:
- Rapid enlistment of four dermatology researchers with proven experience in acne photo assessment.
- Creation and implementation of training and consensus-building modules to ensure evaluator agreement and expertise documentation.
- Procurement and configuration of identical computer systems dedicated to the project.
- Preparation of a de-identified and randomized photo collection for lesion counting.
- Adoption of software facilitating easy learning for evaluators, standardized lesion count data capture, and formatting for statistical analysis.
- Provision of technology training, support, and encouragement to expert evaluators.
The Results
The solution facilitated a fair and comprehensive evaluation of the client’s laser treatment efficacy. Through the optimized expert review process, the treatment system demonstrated its effectiveness convincingly, paving the way for regulatory approval and subsequent market entry.
Conclusion
By addressing the challenges inherent in evaluating acne treatment efficacy through expert photo review, our collaborative efforts ensured a robust and unbiased assessment, bolstering confidence in the client's innovative solution and its potential to positively impact acne management.
Click here to open a PDF version of this case study.
