AI Document Categorization for BetterRegulation.com

BetterRegulation.com publishes hundreds of legislative updates monthly across UK, Ireland, and EU jurisdictions. Before automation, every new statute or guidance document required an editor to read the full text - from 2 to 350 pages - and manually populate approximately 15 fields covering document type, jurisdiction, organization, legislation area, and enactment year. A second editor then verified the entries. The process consumed roughly one full-time equivalent and limited how fast new content could reach subscribers.

Challenge

The platform needed to:

• Reduce time spent on initial document reading and manual field entry.
• Auto-populate roughly 15 text, taxonomy, date, and reference fields per document.
• Map extracted values to existing Drupal taxonomy terms by entity ID, not free text.
• Handle PDFs from 2 to 350 pages with complex legal formatting.
• Keep editors in control with review before publication.
• Scale document volume without proportional headcount growth.

Solutions

Generate with AI in the editorial workflow

Editors upload a PDF to the Original Document field and click Generate with AI. Processing completes in 10 seconds to 2 minutes depending on document size. Auto-filled fields appear on the same Drupal form for review and adjustment before save. The workflow augments editors rather than replacing them: AI handles extraction, humans verify accuracy.

PDF extraction with Unstructured.io

Legal PDFs often include multi-column layouts, headers, footers, and embedded tables that break naive parsers. After testing several extraction methods, Unstructured.io delivered cleaner text with fewer artifacts, reducing token usage and improving categorization accuracy on production documents.

Structured prompts and taxonomy injection

Each prompt includes the full list of available taxonomy terms for every reference field. The language model returns structured JSON with matched term entity IDs, which AI Automators writes directly into Drupal entity reference fields. This prevents orphaned terms and keeps relationships consistent as taxonomies evolve.

AI Automators on Drupal 11

The AI Automators contrib module orchestrates the multi-step pipeline: PDF text extraction, GPT model calls, JSON validation, and field population. Prompts are configurable from the admin UI without code changes per field. Watchdog logging captures failures with retry logic for transient errors. Documents exceeding token limits route to a manual review queue with clear editor messaging.

Result

Overall document ingestion, categorization, review, and publication time dropped by 50%. Initial reading that took 20 minutes to several hours per document now takes 10 seconds to 2 minutes of AI processing plus roughly 5 minutes of focused editor review. Processing succeeds in more than 95% of runs, with fewer than 5% of fields requiring correction. The platform can handle roughly twice the document volume without adding editorial staff, freeing capacity equivalent to one full-time role for verification and strategic work.

Examples of implemented features:

• Generate with AI button on document edit forms with synchronous on-demand processing.
• Auto-population of roughly 15 fields including title, jurisdiction, document type, and organization.
• Semantic taxonomy matching that returns Drupal entity IDs for reference fields.
• Graceful handling of oversized documents via manual review queue.
• Two-layer editorial QA preserved: editor review plus second verification before publish.