Total 3 posts
After splitting a document, not every chunk is worth keeping. Ads, nav bars, disclaimers, and duplicates are noise that pollutes retrieval. This walks rule-based, structural, semantic, and LLM filtering strategies, with a Java pipeline you can adapt.
The first decision in RAG isn't picking a vector DB, it's what you store after chunking the PDF. Full-text ingestion and structured extraction are two completely different paths; data from a medical scenario makes this very clear.
Using the open-source project Horizon plus GitHub Actions' free tier, automatically scrape Hacker News, Reddit, and major official blogs every day; after AI scoring and filtering, generate a daily brief pushed to Feishu. Completely free.
