Recent attention has focused on the post-market retraining of AI-based software as a medical device (SaMD) following the introduction of a new approval process in Japan, known as the “Improvement Design within Approval for Timely Evaluation Notice (IDATEN).” This process facilitates the post-market performance improvement of SaMD. However, repeated use of test data raises concerns about over-adaptation to the test dataset, potentially introducing performance bias. In this study, performance bias was evaluated using a deep model designed to support hotspot detection in bone scintigraphy by simulating the selection and integration of multiple post-market models with repeated use of identical test data. Performance bias was observed in both ensemble learning approaches employing bagging- and boosting-inspired sequential aggregation with pretrained models. Additionally, bias reduction was demonstrated using Thresholdout_AUC, based on differential privacy principles. These findings are expected to be useful for SaMD development, leading to continuous performance improvements.