Larger models yield better results? Streamlined severity classification of ADHD-related concerns using BERT-based knowledge distillation

This work focuses on the efficiency of the knowledge distillation approach in generating a lightweight yet powerful BERT-based model for natural language processing (NLP) applications. After the model creation, we applied the resulting model, LastBERT, to a real-world task-classifying severity levels of Attention Deficit Hyperactivity Disorder (ADHD)-related concerns from social media text data. Referring to LastBERT, a customized student BERT model, we significantly lowered model parameters…

via https://pubmed.ncbi.nlm.nih.gov/39913350/?utm_source=Other&utm_medium=rss&utm_campaign=None&utm_content=1lqZ3NPYysePVKsoyz66mDSgu4veDGJwnUBS47TBQPoOuNZY5J&fc=None&ff=20250220011008&v=2.18.0.post9+e462414