June 22, 2024 · NLP · 10 min read
Extracting and Summarizing Academic Theses with NLP
How I built a system to extract thesis information from theseus.fi and summarize concepts using Hugging Face transformers, T5 models, and BERT.
In this project, I developed an automated system to extract and summarize academic thesis information from theseus.fi, Finland's national repository for theses and publications.
Project Architecture
The system consisted of several components:
- Web scraping module for data extraction from theseus.fi
- NLP pipeline using Hugging Face transformers
- React Native mobile app for user interface
- Flask API for backend processing
NLP Implementation
I utilized several state-of-the-art models:
- T5 model for text summarization
- BERT for text classification and entity recognition
- Custom fine-tuned models for academic text processing
Impact
The system successfully processed over 10,000 thesis abstracts with high-quality summaries.