📝 Text Classification Report

• Word Segmentation: All texts are preprocessed using VnCoreNLP to correctly group Vietnamese compound words (e.g., converting "học sinh" to "học_sinh").
• Subword Tokenization: Apply Byte Pair Encoding (BPE) to tokenize text into subwords, generating integer sequences optimized for the model architectures.
• Padding/Truncation: Input sequences are truncated or padded to a strict maximum length of 128 tokens.

# Import necessary libraries
from transformers import AutoTokenizer
from pyvi import ViTokenizer

# 1. Word Segmentation (Grouping compound words)
raw_text = "Tàu du lịch cao tốc Cần Thơ"
segmented_text = ViTokenizer.tokenize(raw_text)
# Output: 'Tàu du_lịch cao_tốc Cần_Thơ'

# 2. Load Pretrained PhoBERT Tokenizer
tokenizer = AutoTokenizer.from_pretrained("vinai/phobert-base")

# 3. Subword Tokenization, Padding & Truncation
encoded_input = tokenizer(
    segmented_text,
    padding='max_length',
    truncation=True,
    max_length=128,
    return_tensors='pt'
)

RNN: LSTM (Trained from scratch)

Contains an Embedding layer (dim=300), an LSTM layer (hidden=256), Dropout (0.3), and a FC layer. Trained with Adam (LR=1e-3) for 5 Epochs with Batch Size 16.

🔗 Understanding LSTM

Transformer: PhoBERT (Fine-tuned)

Based on RoBERTa architecture. To improve efficiency and prevent overfitting, we froze the transformer encoder and only trained the classification head. Trained with AdamW (LR=1e-3) for just 2 Epochs with Batch Size 8.

🔗 PhoBERT Paper

# Load the pre-trained PhoBERT model
model = AutoModelForSequenceClassification.from_pretrained("vinai/phobert-base", num_labels=10)

# Freeze all layers in the base RoBERTa encoder to save compute and prevent overfitting
for param in model.roberta.parameters():
    param.requires_grad = False

# Only the weights of the classification head (classifier layer) will be updated
for param in model.classifier.parameters():
    param.requires_grad = True

🔄 Data Pipeline & Tensor Shapes

Below is the specialized data pipeline detailing the Tensor dimensions (Batch Size is denoted as B) and the semantic meaning of the data at each step.

Note: Evaluated on the test split (approx. 20% of the active dataset).

Performance Comparison (Test Set)

Confusion Matrices

* Click on any matrix to view in full screen.

LSTM (Trained from scratch)

PhoBERT (Fine-tuned)

Common Confusions

By analyzing the Confusion Matrices above, we noticed both models struggle with multi-topic or ambiguous articles where semantic overlap exists. Here are the most frequent misclassifications:

Real-world Error Examples