I'm Abhay Kumar, Senior LLM Research Engineer at BluOrion, with a deep passion for training neural networks, particularly language models and large language models (LLMs). With over 8 years of experience in Data Science and NLP.
- Senior LLM Research Engineer at BluOrion
- Co-Author of Komodo LLM: An open-source, language-specific LLM for Indonesian.
- 8+ years of experience in Data Science and NLP, with a focus on Language Modeling.
- ZClip: An Adaptive Spike Mitigation for LLM Pre-Training.
- Variance Control: Variance Control via Weight Rescaling in LLM Pre-training.
- Komodo LLM: A foundational large language model tailored for a specific language.
- miniLLama: A straightforward and compact implementation of the LLAMA Model, inspired by Andrej Karpathy's minGPT.
- GPT2-TF: Implementation of GPT-2 in TensorFlow 2, recognized as the first repository for GPT-2 in TensorFlow 2.