Abstract: Recent advancements in scaling large language models (LLMs) have enhanced various natural language processing (NLP) tasks. However, open-source moderately sized models, such as BERT, are ...