Hi, I'm Sam

Applied NLP & LLM ApplicationsClinical NLP · Knowledge Grounding · EvaluationI build and evaluate language-model applications that connect external knowledge sources to practical workflows. My work focuses on knowledge grounding, terminology-based verification, and evaluating NLP systems in clinical and legal settings._{Master’s in Linguistics : Computational Specialisation, University of Bergen (Jan 2024)}

About

I work on NLP and LLM applications in settings where information quality and traceability matter. My background in linguistics informs how I approach these problems, with attention to domain terminology, the context in which outputs are used, and how system performance is evaluated.My projects span legal document retrieval, medical terminology verification, clinical text harmonisation, and ontology-informed text classification. Across them, I examine how external knowledge and systematic evaluation can make language-model applications more useful and their limitations easier to identify.

Research Focus

My work focuses on the reliability of NLP systems used in clinical and legal settings. Across my projects, two questions recur: whether system outputs are sufficiently grounded to support expert decisions, and how robust the evidence of system performance is to evaluation choices.Current themes• Clinical NLP
• Ontology-grounded language models
• Verification of LLM outputs
• Evaluation design and robustness
• Retrieval-augmented generation in regulated domains

Selected Research & Engineering Projects

Production Legal Document RetrievalRetrieval-augmented application for Norwegian legal documents, developed for use at a legal-AI startup.Lawyers needed reliable retrieval across Norwegian case law and statutes, with an auditable basis for assessing relevance. I worked with three lawyers and two engineers to define evaluation criteria around the practical relevance of retrieved material.After we adapted the evaluation process with lawyers, acceptance on the evaluated sample increased from 75% to 98%. The system used the Claude API, MongoDB Atlas Vector Search, and Azure App Service, with version-controlled prompts and documented rationales to make retrieval decisions easier to inspect.The project was supported by Innovation Norway.Links: GitHub
------------------------

Clinical Text HarmonizationBiomedical NLP pipeline for entity extraction, terminology mapping, and explicit handling of unsupported clinical concepts.Clinical text must be converted into structured representations without assuming that every extracted mention maps cleanly to a standardized concept. This project investigates errors across that process, from biomedical entity extraction to terminology mapping and domain-aware routing, while preserving mentions that cannot be mapped reliably.Using the BC5CDR corpus, I compared dictionary-based extraction with fine-tuned PubMedBERT and evaluated a confidence-thresholded hybrid. With its threshold selected on development data and frozen before testing, the hybrid achieved 0.761 precision, 0.772 recall, and 0.766 F1 on the held-out test set, compared with 0.661 F1 for the dictionary baseline. The pipeline also evaluates terminology mapping separately, distinguishing supported mappings from extracted mentions that remain unmapped.The prototype uses a limited vocabulary and evaluates terminology mapping separately from entity extraction. The reported results do not establish end-to-end clinical correctness or comprehensive terminology coverage.Links: GitHub------------------------

MedTermCheckPrototype for verifying LLM-extracted medical terminology against clinical terminology resources.LLMs can generate medical codes that do not exist or that fail to match the source text. MedTermCheck checks extracted entities against ICD-10-CM and SNOMED CT resources, examining code matches, concept lookup and source-text grounding.Its verification pipeline combines ontology evidence, textual grounding and the model's self-reported confidence into an evidence score. Results are labelled Grounded, Partial or Ungrounded to make the basis for each assessment easier to inspect.The evaluation uses 20 annotated test cases. The evidence score reflects support for a terminology match, not diagnostic correctness. The current SNOMED CT integration relies on a limited embedded fallback dictionary.Links: GitHub · Live demo------------------------

Cohen Benchmark: Evaluating Automatic Medical Subject HeadingsResearch on MeSH assignment methods and the sensitivity of systematic-review screening results to evaluation design.This study investigates whether expert-assigned and mechanically assigned MeSH terms are interchangeable as features for systematic-review screening, and how the answer depends on evaluation protocol.Using three drug-class topics from the Cohen et al. benchmark, I compare bag-of-words logistic regression with BiomedBERT across different text representations. The results show a topic-dependent pattern: the expert-versus-automatic MeSH performance gap is robust for Statins but not clearly detectable for Opioids or ADHD at the available sample sizes. BiomedBERT substantially reduces the measured gap for Statins without conclusively eliminating it.The work examines how annotation methods, sample composition and evaluation choices shape conclusions about model performance. The manuscript is under review.Links: arXiv preprint------------------------

Medical Text Classification (Master’s Thesis)Ontology-informed text classification for systematic-review screening of dementia intervention studies.My thesis investigated whether domain knowledge from the Neurologic Examination Ontology (NEO), a 1,611-concept subset derived from SNOMED CT, could improve classification of abstracts for a Cochrane review of dementia interventions.Using 150 abstracts, I compared logistic-regression configurations with and without ontology-derived features across 11 configurations. The original best configuration achieved 93% accuracy on a 30-example test set. A later five-fold stratified cross-validation re-analysis yielded accuracy of approximately 80–86%. Ontology enrichment changed the features used by the classifier but did not consistently outperform the text-based baseline.This work established the foundation for my subsequent interest in terminology-grounded NLP and the evaluation of structured biomedical knowledge.Links: GitHub · Thesis
------------------------

GDPR Article 9 Compliance CheckerRule-based auditing of healthcare AI documentation against GDPR Article 9 requirements.This project explores how healthcare AI documentation can be checked against explicit data-protection requirements using auditable rules.The tool evaluates documentation against 42 Article 9 checks, extracts supporting evidence and generates a compliance-gap report. Each finding can be traced to the rule that triggered it, making the basis for a flag easier to inspect.The checker is an English-language gap-analysis tool, not a legal compliance certification system. Its keyword-based approach can miss relevant context and does not assess GDPR provisions beyond Article 9.Links: GitHub · Live demo

Publications

PreprintsOkoe-Mensah, S. M. (2026). Evaluation design conditions the expert-vs-auto MeSH gap: a controlled comparison of bag-of-words and BiomedBERT on the Cohen benchmark. arXiv:2607.21685 [cs.CL].Paper (arXiv) · Archive (Zenodo)

Background

BackgroundI studied Chinese and Linguistics at the University of Ghana before completing a Master’s in Linguistics - Computational Specialisation at the University of Bergen in January 2024. My thesis examined whether ontology-derived features could improve medical text classification. During my master’s, I also supported Python and NLP teaching, with a reported course completion rate of 91%.Before moving into computational linguistics, I spent three years on the Volta River in Ghana, coordinating operations between Ghanaian teams and Chinese engineering personnel on an underwater timber harvesting project. The experience involved technical coordination and communication across languages and professional disciplines.I now teach Python and NLP at the University of Ghana, alongside independent work on clinical NLP, terminology grounding and the evaluation of language-model applications.

• View GitHub

About Me

Outside of research, I spend most evenings at the piano working through jazz standards, in the kitchen trying to get a new recipe right, or reading science fiction. Having lived and worked in Ghana, China, and Norway, I’ve grown accustomed to finding my way through unfamiliar environments and learning from people with different perspectives.

Contact

If you are working on clinical NLP or on verification and evaluation for LLMs in regulated settings,
I would be glad to hear from you.[email protected] | LinkedIn | GitHub | Hugging Face

Built with Carrd. Content © Samuel Okoe-Mensah 2026.