Projects - Yosef Worku Alemneh

July 2025 - Present

Splade-Index is a Python package that implements a fast inverted index for SPLADE sparse retrieval models. It can index, query, and rank millions of documents.
It works with any SparseEncoder model supported by sentence-transformers, including naver/splade-v3 and rasyosef/splade-tiny, a 4.4M-parameter sparse encoder trained by me that achieves 30.8 MRR@10 on MSMARCO Dev, outperforming BM25 by 65%.
Features include just-in-time compilation with Numba, memory mapping for low RAM usage, and PyTorch with CUDA for 10x faster search on NVIDIA GPUs. View on GitHub.
Save and load indices locally, or upload them to the Hugging Face Hub so others can download and use them. Memory mapping loads a 1-million-document index with just 2GB of RAM without a noticeable change in search latency.
Use NVIDIA GPUs and PyTorch for up to 10x faster search than the CPU-based Numba backend when working with indices containing more than 1 million documents.
A Splade-Index usage example has been added to the official Sentence-Transformers GitHub repository.

July 2025 - Present

Trained SPLADE sparse retrieval models based on BERT-Tiny (4M parameters), BERT-Mini (11M), and BERT-Small (28.8M) by distilling the ms-marco-MiniLM-L6-v2 cross-encoder on MSMARCO.
The tiny SPLADE models beat BM25 by 65.6-89.4% on MSMARCO. Despite being 6-15x smaller than naver/splade-v3, they retain 80-88% of its performance while producing embedding vectors with up to 45% fewer active dimensions.
rasyosef/splade-mini also outperforms the 6x larger naver/splade_v2_max on MSMARCO, and the models are small enough to run without a GPU on datasets of a few thousand documents.
rasyosef/splade-tiny has been downloaded over 150,000 times on Hugging Face and cited in a paper by Principal Engineers at Amazon Web Services.
Download the models or view the code on GitHub. The distillation dataset is also available on Hugging Face.
rasyosef/splade-tiny and rasyosef/splade-mini have been added to the list of recommended SPLADE models in the Sentence-Transformers library documentation.

July 2024 - August 2024

Fine-tuned Microsoft's Phi-2 base LLM to follow instructions through Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO), using the TRL library and a single A100 40GB GPU for both stages.
Supervised Fine-Tuning: Used 128,000 instruction-response pairs from the OpenHermes-2.5 dataset.
Direct Preference Optimization: Used 20,000 preference pairs from the UltraFeedback Binarized dataset.
Evaluation: Evaluated Phi-2 Instruct on IFEval, GSM8K, MMLU, TruthfulQA, and Winogrande using EleutherAI's lm-evaluation-harness. It scored 13 points higher than the base model on IFEval and performed better on every other benchmark except MMLU.
Open LLM Leaderboard: Ranked 3rd among chat models with fewer than 3B parameters and beat Google's Gemma-1.1-2B-IT by 6.14 points on IFEval. View the models.

May 2024 - January 2025

Pretrained BERT and RoBERTa encoder models (4M-110M parameters) on a 290M-token Amharic corpus with a 512-token context window.
Built tokenizers from scratch on the same corpus, with vocabulary sizes of 28,672 for BERT and 32,000 for RoBERTa.
Fine-tuned the models for Amharic sentiment classification and named entity recognition. roberta-base-amharic (110M) achieved F1 scores of 0.88 and 0.78, while roberta-medium-amharic (42M) reached 0.84 and 0.75.
roberta-base-amharic exceeded afro-xlmr-large (560M) by 0.02 F1 on both tasks, and roberta-medium-amharic (42M) surpassed xlm-roberta-base (279M).
The models have been cited by four peer-reviewed publications, including two ACL papers and two SIGIR papers. View the models.

October 2024 - December 2024

Pretrained 400M- and 180M-parameter versions of Meta's Llama 3.2 1B from scratch on approximately 300 million (274M) Amharic tokens. The 400M Base model trained for 23 hours on one A100 40GB GPU and reached 41.3 validation perplexity; the 180M Base model trained for 26 hours and reached 42.5. View the model collection.
All models use a 32k vocabulary Llama 3.2 tokenizer, trained from scratch on the same pretraining dataset, and have a 1024-token context.
The 400M model was instruction-tuned on approximately 20k Amharic poems, stories, jokes, and Wikipedia articles.
The models generate fluent Amharic text, while the Instruct versions support conversations, question answering, sentiment and text classification, and creative writing.
Models: Llama 3.2 Amharic 400M Base, Llama 3.2 Amharic 400M Instruct, Llama 3.2 Amharic 180M Base, and Llama 3.2 Amharic 180M Instruct. Try the chat demo.