0Pricing
AI Engineering Academy · درس

استخراج البيانات من النصوص غير المنظّمة

ابنوا pipeline لاستخراج المعلومات يقرأ نصوصًا خامًا مثل رسائل البريد الإلكتروني والإيصالات والمقالات، ويعيد حقولًا منظّمة بأنواع وقيم افتراضية وقواعد تحقق.

استخراج البيانات من النصوص غير المنظّمة درس مجاني في AI Engineering Academy على CoddyKit. هذا هو الدرس 3 من أصل 4. يمكنك قراءة الدرس كاملاً أدناه مجاناً — ثم تمرن عليه مباشرة في المتصفح باستخدام محرر أكواد مدمج ومدرس ذكاء اصطناعي متاح 24/7. هذا الدرس جزء من مسار التعلم في AI Engineering Academy، وتقدمك يتزامن عبر الويب وتطبيق CoddyKit. تتضمن دورة AI Engineering Academy 4 دروس في المجموع.

بعض أجزاء هذا الدرس لم تُترجم بعد وتظهر باللغة الإنجليزية.

The Information Extraction Problem

Organizations are drowning in unstructured text: emails, support tickets, contracts, invoices, news articles, medical notes, and social media posts. Valuable structured data is buried in this text, but extracting it manually is slow, expensive, and error-prone. LLMs with structured outputs change this: they can read any text and populate a predefined schema with the relevant fields, at scale, with reasonable accuracy.

Information extraction (IE) is the process of automatically identifying and pulling structured facts from unstructured text. LLM-based IE dramatically outperforms earlier rule-based or classical NLP approaches because LLMs understand context, synonymy, and implicit information without needing hand-crafted regex patterns for every variation.

Common Extraction Use Cases

Information extraction powers many valuable business applications:

  • Invoice processing: Extract vendor, line items, amounts, and due dates from PDF invoices for accounts payable automation
  • Contract analysis: Extract parties, effective dates, payment terms, and termination clauses from legal documents
  • Resume parsing: Extract skills, experience, education, and contact info from CVs for ATS systems
  • Support ticket routing: Extract category, severity, affected product, and customer tier to route tickets automatically
  • News monitoring: Extract entities, events, and sentiments from news articles for competitive intelligence

Building an Email Extraction Pipeline

Let us build a practical extraction pipeline that reads customer emails and extracts actionable structured data. The pipeline uses a Pydantic schema to define exactly what we want from each email, then processes emails in batch.

import openai
from pydantic import BaseModel
from typing import List, Optional
from enum import Enum

client = openai.OpenAI()

class Priority(str, Enum):
    urgent = 'urgent'
    high = 'high'
    normal = 'normal'
    low = 'low'

class EmailExtraction(BaseModel):
    subject_summary: str
    sender_intent: str
    product_mentioned: Optional[str]
    issue_category: str  # billing / technical / general / feedback
    priority: Priority
    action_required: bool
    action_description: Optional[str]
    customer_sentiment: str  # positive / negative / neutral / frustrated

def extract_from_email(email_body: str) -> EmailExtraction:
    result = client.beta.chat.completions.parse(
        model='gpt-4o-mini',
        messages=[
            {'role': 'system', 'content': 'You are an expert at analyzing customer emails and extracting structured information for a support team.'},
            {'role': 'user', 'content': f'Analyze this customer email:\n\n{email_body}'}
        ],
        response_format=EmailExtraction
    )
    return result.choices[0].message.parsed

Named Entity Recognition with LLMs

Named Entity Recognition (NER) is a classic IE task: identifying and classifying named entities (people, organizations, locations, dates, monetary amounts) in text. LLMs dramatically simplify NER because you just describe the entities you want and they extract them without needing a specially trained NER model.

import openai
from pydantic import BaseModel
from typing import List, Optional

client = openai.OpenAI()

class NamedEntity(BaseModel):
    text: str       # The exact text as it appears
    entity_type: str  # PERSON / ORG / LOCATION / DATE / MONEY / PRODUCT
    normalized: Optional[str]  # Standardized form where applicable

class NERResult(BaseModel):
    entities: List[NamedEntity]

text = '''
Apple Inc. CEO Tim Cook announced yesterday that the company will invest $1.2 billion
in a new manufacturing facility in Austin, Texas, expected to open in Q3 2026.
'''

result = client.beta.chat.completions.parse(
    model='gpt-4o-mini',
    messages=[
        {'role': 'system', 'content': 'Extract all named entities from the text. Classify each as PERSON, ORG, LOCATION, DATE, MONEY, or PRODUCT.'},
        {'role': 'user', 'content': text}
    ],
    response_format=NERResult
)
for entity in result.choices[0].message.parsed.entities:
    print(f'[{entity.entity_type}] {entity.text}')

Extracting from Documents at Scale

For production extraction pipelines processing thousands of documents, you need async processing and rate limit handling. A typical pattern uses asyncio with a semaphore to process documents in parallel while respecting the API's rate limits.

import asyncio
import openai
from pydantic import BaseModel
from typing import List, Optional

async_client = openai.AsyncOpenAI()

class InvoiceExtraction(BaseModel):
    vendor: str
    total_amount: Optional[float]
    currency: str
    invoice_date: Optional[str]

async def extract_invoice(doc_text: str, semaphore: asyncio.Semaphore) -> InvoiceExtraction:
    async with semaphore:  # Limit concurrent requests
        result = await async_client.beta.chat.completions.parse(
            model='gpt-4o-mini',
            messages=[
                {'role': 'system', 'content': 'Extract invoice data.'},
                {'role': 'user', 'content': doc_text}
            ],
            response_format=InvoiceExtraction
        )
        return result.choices[0].message.parsed

async def process_invoices(documents: List[str]):
    sem = asyncio.Semaphore(5)  # Max 5 concurrent requests
    tasks = [extract_invoice(doc, sem) for doc in documents]
    return await asyncio.gather(*tasks, return_exceptions=True)

# results = asyncio.run(process_invoices(invoice_texts))
print('Async pipeline defined - handles rate limits via semaphore')

Handling Implicit and Inferred Information

LLMs can extract not just explicitly stated information but also inferred or implicit information. If a review says 'I have been using this daily for a month and it still works perfectly', the model can infer durability as a positive attribute even though the word 'durability' never appears. This is a major advantage over regex-based extraction which can only find what is explicitly present.

However, this power comes with risk: the model may over-infer and populate fields with guesses rather than facts. For high-stakes extraction (legal, financial, medical), add a confidence field to your schema and instruct the model to rate its certainty, flagging low-confidence extractions for human review.

Extraction Prompt Design

The quality of your extraction depends heavily on prompt design. Key principles for extraction prompts:

  • Define ambiguous fields: If 'date' could mean invoice date, due date, or received date, specify exactly which one you want
  • Provide examples for unusual formats: 'For price, return the numeric value only, e.g., 29.99 not $29.99'
  • Handle normalization: 'Normalize country names to ISO 3166-1 alpha-2 codes'
  • Specify extraction source: 'Extract only from the subject line, not the email body'

Think of the extraction prompt as a precise specification for a human data entry operator — every ambiguity you leave in the prompt is a judgment call the model will make inconsistently.

Multi-Pass Extraction for Complex Documents

Some documents are too complex to extract in a single pass because the full schema is large, different sections require different expertise, or the document structure is highly variable. Multi-pass extraction breaks the task into sequential steps: first classify the document type, then extract the appropriate schema for that type.

import openai
from pydantic import BaseModel
from typing import Optional

client = openai.OpenAI()

class DocumentType(BaseModel):
    doc_type: str  # invoice / contract / resume / report
    confidence: float

def classify_document(text: str) -> str:
    result = client.beta.chat.completions.parse(
        model='gpt-4o-mini',
        messages=[
            {'role': 'system', 'content': 'Classify the document type.'},
            {'role': 'user', 'content': text[:500]}  # Use only the beginning for classification
        ],
        response_format=DocumentType
    )
    return result.choices[0].message.parsed.doc_type

# Then route to the appropriate extraction schema
EXTRACTION_SCHEMAS = {
    'invoice': 'InvoiceSchema',  # Replace with actual Pydantic classes
    'contract': 'ContractSchema',
    'resume': 'ResumeSchema',
}

print('Multi-pass: classify first, then extract with the right schema')

Post-Extraction Enrichment

Extracted data often needs enrichment after initial extraction: converting extracted company names to canonical forms by querying a company database, looking up the extracted zip code to fill in city and state, or converting extracted dates to a standard format. This enrichment step should happen in your application code after extraction, not during the LLM call itself.

Keeping extraction and enrichment separate makes the pipeline easier to test and maintain. You can unit-test the enrichment logic independently and swap out the extraction model without changing your enrichment code.

Measuring Extraction Accuracy

For production extraction pipelines, measure accuracy systematically against a labeled test set. Key metrics are:

  • Field accuracy: Percentage of fields extracted correctly per document
  • Exact match: Field value matches the ground truth exactly
  • Normalized match: Field value matches after normalization (e.g., '$1,234.00' == '1234.0')
  • False positive rate: How often the model extracts a field that should be null
  • False negative rate: How often the model returns null for a field that is present

Run this evaluation whenever you change models, update prompts, or add new document sources to your pipeline. Even small accuracy drops can cascade into significant business impact when processing thousands of documents.

Extraction Logging for Continuous Improvement

Every extraction result in production is a data point that can improve your pipeline. Log every input document, extracted output, and any validation errors to a database. Periodically sample from production logs to identify common failure patterns: certain document formats the model struggles with, fields that are frequently null when they should not be, or unusual values that indicate prompt drift.

This logged data also becomes your future training dataset if you ever want to fine-tune a model specifically for your extraction task, giving you a higher-accuracy, lower-cost alternative to general-purpose LLMs for structured extraction.

Quick Check

Test your understanding of AI Engineering concepts from this lesson.

Lesson Recap

In this lesson you learned: LLMs with Pydantic schemas extract structured data from any unstructured text source reliably, async processing with semaphores enables batch extraction of thousands of documents while respecting rate limits, and multi-pass extraction classifies documents first then applies the appropriate schema for each type. Next up we build validation and auto-retry logic to handle cases where extracted data fails business rules.

الأسئلة الشائعة

هل درس «استخراج البيانات من النصوص غير المنظّمة» مجاني؟

نعم — نص درس «استخراج البيانات من النصوص غير المنظّمة» كامل متاح مجاناً هنا على الويب. لتمرينه بشكل تفاعلي (محرر أكواد مدمج ومدرس ذكاء اصطناعي متاح 24/7) وفتح باقي دورة AI Engineering Academy، انتقل إلى CoddyKit PRO. تتضمن دورة AI Engineering Academy 4 دروس في المجموع.

ماذا ستتعلم في «استخراج البيانات من النصوص غير المنظّمة»؟

ابنوا pipeline لاستخراج المعلومات يقرأ نصوصًا خامًا مثل رسائل البريد الإلكتروني والإيصالات والمقالات، ويعيد حقولًا منظّمة بأنواع وقيم افتراضية وقواعد تحقق. تتمرن على AI Engineering Academy مع أكواد عملية تشغلها مباشرة في المتصفح، ومدرس ذكاء اصطناعي متاح 24/7 يجيب على أسئلتك أثناء عملك.

هل أحتاج إلى خبرة سابقة لأبدأ AI Engineering Academy؟

لا تُشترط خبرة سابقة. AI Engineering Academy على CoddyKit منظم للمبتدئين حتى المتقدمين، لذا يمكنك البدء من هنا أو من البداية والتقدم بسرعتك الخاصة. هذا هو الدرس 3 من أصل 4.

كم من الوقت يستغرق درس «استخراج البيانات من النصوص غير المنظّمة»؟

معظم دروس CoddyKit تستغرق حوالي 5–10 دقائق. كل منها موجز وتفاعلي، لذا تحرز تقدماً مستمراً وتستأنف من حيث توقفت عبر الويب والتطبيق.

هل يمكنني كتابة وتشغيل أكواد في درس AI Engineering Academy هذا؟

نعم. كل درس في AI Engineering Academy يتضمن محرر أكواد مدمج، لذا تكتب وتشغل أكواداً حقيقية مباشرة في متصفحك وتحصل على تعليقات فورية من الذكاء الاصطناعي — بدون إعداد محلي.

جميع الدروس في هذه الدورة

  1. وضع JSON وresponse_format
  2. المخرجات المنظّمة باستخدام Pydantic
  3. استخراج البيانات من النصوص غير المنظّمة
  4. التحقق من المخرجات الخاطئة وإعادة المحاولة
← العودة إلى AI Engineering Academy