Retrieval practice is an evidence-based approach to teaching; here, we evaluate the use of PeerWise for embedding retrieval practice into summative assessment. PeerWise allows anonymous authoring, sharing, answering, rating, and feedback on peer-authored multiple choice questions. PeerWise was embedded as a summative assessment in a large first-year introductory biochemistry module. Engagement with five aspects of the tool was evaluated against student performance in coursework, exam, and overall module outcome. Results indicated a weak-to-moderate positive but significant correlation between engagement with PeerWise and assessment performance. Student feedback showed PeerWise had a polarizing effect; the majority recognized the benefits as a learning and revision tool, but a minority strongly disliked it, complaining of a lack of academic moderation and irrelevant questions unrelated to the module. PeerWise can be considered a helpful learning tool for some students and a means of embedding retrieval practice into summative assessment.