I Fine-Tuned Microsoft Phi-2 to Read Teacher Evaluations. It Started at 34% Accuracy.

The first time I ran the raw model on a batch of student comments about their teachers, it called 93% of them “neutral.” A kid wrote “Mr. Harrison made me actually look forward to calculus, and I hate math” — neutral. Another wrote “Ms. Rivera spent the whole semester reading off slides she didn’t make and never answered a single question” — also neutral.

Turns out a 2.7-billion-parameter language model, fresh off its general-purpose training, has no idea what teacher feedback sounds like. It defaults to the middle of the road every time. I needed to teach it.

The goal was straightforward on paper: take Microsoft Phi-2 — one of the strongest small language models you can run on a single GPU — and fine-tune it (train it further on a specific dataset) to classify student evaluations as positive, neutral, or negative. On paper. In practice I burned a weekend on data formatting alone.

What Worked (and What Didn’t)

Let me lead with the numbers because they tell the story. After fine-tuning:

• Overall accuracy: 34.9% → 87.2%
• Positive sentiment accuracy: 7.3% → 97.0%
• Negative sentiment accuracy: 11.0% → 84.7%
• Training loss: dropped 38% (from 1.41 to 0.87)

The jump on positive sentiment is the one that surprised me. The base model was essentially guessing — 7.3% is worse than random across three categories. After fine-tuning it caught 97% of the genuinely positive comments. Negative sentiment was harder; sarcasm in student evaluations is real, and the model still misses some of it. I’ll take 84.7% over 11% any day.

Setting Up

You’ll need a handful of Python libraries. Everything here works with standard pip install — no exotic dependencies:

pip install accelerate peft einops datasets bitsandbytes trl transformers datasets

The Data: Why I Filtered Out Short Comments

The dataset was a tab-separated file of teacher evaluations — student comments paired with sentiment labels. I loaded it with pandas and immediately hit my first wrong turn: short comments.

A comment like “Good teacher” tells you exactly nothing useful. It’s positive, sure, but there’s no signal for the model to learn from — no patterns in word choice, no structure, no context. I filtered out anything under 200 characters. That single decision cleaned up the training data more than any hyperparameter tweak I tried later.

import pandas as pd
from sklearn.model_selection import train_test_split

# Load data
filename = "./data/teacher_performance/ReadyToTrain_data_2col_with_subjectivity_final.tsv"
training_data_df = pd.read_csv(
    filename,
    sep='\t',
    encoding="utf-8",
    encoding_errors="replace"
)

# Filter comments longer than 200 characters
training_data_df = training_data_df[
    training_data_df['StudentComments'].str.len() > 200
]

I also needed balanced splits. If you dump all the data into training without stratifying by sentiment, your model sees way more of one class and skews hard toward it. I split each sentiment category separately using scikit-learn’s train_test_split with a fixed random seed so the results are reproducible:

# Split data for each sentiment
X_train = []
X_test = []

for sentiment in ['positive', 'neutral', 'negative']:
    train, test = train_test_split(
        training_data_df[training_data_df['Sentiment'] == sentiment],
        random_state=42
    )
    X_train.append(train)
    X_test.append(test)

Getting Phi-2 to Fit on One GPU

Phi-2 has 2.7 billion parameters. At full 16-bit precision (the standard floating-point format most models ship in), that’s about 5.4 GB just for the weights — before you add memory for activations, gradients, and optimizer states during training. A consumer GPU with 24 GB of VRAM would buckle under a full fine-tune.

The fix is 4-bit quantization — compressing each weight from 16 bits down to 4 bits using a format called Normal Float 4 (NF4), which is designed specifically for the bell-curve distribution most neural network weights follow. This cuts VRAM usage by roughly 60% with almost no quality loss. bitsandbytes handles this through Hugging Face’s BitsAndBytesConfig:

from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig
)

# Quantization configuration
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=False,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16
)

# Load model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained(
    "microsoft/phi-2",
    trust_remote_code=True,
    device_map="auto",
    quantization_config=bnb_config
)

device_map="auto" tells the loader to spread layers across available GPUs if you have more than one. On a single card it just puts everything there. trust_remote_code=True is required for Phi-2 because it uses a custom model architecture that hasn’t been merged into the main Transformers library yet.

Training Without Melting the GPU

Even with 4-bit quantization, updating every weight in a 2.7B-parameter model during training is expensive — both in memory and in risk of the model forgetting what it already knew (catastrophic forgetting, where new training overwrites general knowledge the model needs).

LoRA — Low-Rank Adaptation — sidesteps both problems. Instead of touching the original weights, LoRA freezes them in place and injects pairs of small trainable matrices into specific attention layers. Only those tiny matrices get updated during training. The original model stays frozen. At the end, you merge the small matrices back into the base weights, and the model runs at full speed with zero overhead.

I targeted four projection layers inside the attention mechanism: q_proj, k_proj, v_proj (the query, key, and value projections that let the model decide which parts of the input to pay attention to) and dense (the output projection). The rank r=16 controls how many dimensions those adapter matrices have — higher means more capacity to learn but more parameters to train. 16 is a solid default for a task like this.

from peft import LoraConfig

lora_config = LoraConfig(
    r=16,
    lora_alpha=16,
    target_modules=[
        "q_proj",
        "k_proj",
        "v_proj",
        "dense"
    ],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)

For the training run itself, I pushed to 50 epochs (full passes through the training data) because the learning rate was low and the adapter matrices are small — they need more passes to converge. A batch size of 4 with gradient accumulation across 8 steps means the effective batch is 32 samples before each weight update. fp16=True keeps the forward/backward passes in half-precision to save memory.

training_arguments = TrainingArguments(
    output_dir="./runs/Sentiment-Analysis-Phi2-fine-tuned",
    num_train_epochs=50,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=8,
    learning_rate=2e-4,
    weight_decay=0.001,
    fp16=True
)

Running the Training

Hugging Face’s SFTTrainer (Supervised Fine-Tuning Trainer) wraps the model, dataset, LoRA config, and tokenizer into a single training loop. You call .train() and it handles the rest — forward passes, loss calculation, backpropagation, optimizer steps, and checkpointing:

sft_trainer = SFTTrainer(
    model=base_model,
    train_dataset=train_data,
    eval_dataset=eval_data,
    peft_config=lora_config,
    tokenizer=tokenizer,
    args=training_arguments
)

sft_trainer.train()

The training loss curve told the real story. It started at 1.41 and bottomed out at 0.87 after 50 epochs — a 38% reduction. Loss measures how badly the model’s predictions miss the correct answer; lower is better, and a steady downward curve without spiking back up means the model was learning without overfitting.

from peft import AutoPeftModelForCausalLM
import torch

new_model = AutoPeftModelForCausalLM.from_pretrained(
    adapter_dir,                  # directory SFTTrainer wrote to
    torch_dtype=torch.bfloat16,
    device_map={"": 0},
)
merged = new_model.merge_and_unload()
merged.save_pretrained("merged_model", safe_serialization=True)
tokenizer.save_pretrained("merged_model")

Where It Still Stumbles

84.7% on negative sentiment is good but not great. The remaining 15% are mostly borderline cases — students being politely critical, sarcastic, or mixing a compliment with a complaint in the same sentence. A human reader catches the tone shift; the model sometimes registers only the positive words and misses the knife twist at the end.

If I were iterating on this further, I’d experiment with a higher LoRA rank (r=32 or r=64) to give the adapter more capacity to capture those subtleties, or hand-label a few hundred of the misclassified examples and do a second fine-tuning pass on just the hard cases. I’d also try adding a subjectivity score column — the original dataset had one, and I suspect comments rated highly subjective are exactly the ones the model struggles with.

The full training notebook is on GitHub. Questions or ideas for the next iteration: [email protected].