---
name: document-intelligence
description: "Extract, parse, analyze and classify documents using Snowflake Cortex AI. Routes to AI_EXTRACT (structured fields), AI_PARSE_DOCUMENT (full text), AI_COMPLETE (visual analysis). Triggers: extract data from files, process documents, my files, my documents, PDF, image, invoice, contract, receipt, form, OCR, parse document, read document, get text, extract fields, document processing, file extraction, stage files, classify document."
parent_skill: cortex-ai-functions
---

# Document Intelligence

Entry point for all document processing tasks using Snowflake Cortex AI.

## ⚠️ CRITICAL: always use AI_* function names WITHOUT the `SNOWFLAKE.CORTEX` namespace prefix

## ⚠️ CRITICAL: Always Display Pricing Before Execution

**BEFORE executing ANY AI function (AI_EXTRACT, AI_PARSE_DOCUMENT, AI_COMPLETE), you MUST:**

1. Calculate estimated cost based on file size/page count
2. Display the estimate to the user

**Quick Pricing Reference:**

| Function | Pricing | Example |
|----------|---------|---------|
| AI_EXTRACT | 5 credits / 1M tokens | 1,000 pages ≈ 3-5 credits |
| AI_EXTRACT (with scale_factor) | 5 credits / 1M tokens × scale_factor | 1,000 pages at scale_factor 2.0 ≈ 6-10 credits |
| AI_PARSE_DOCUMENT (OCR) | 0.5 credits / 1,000 pages | 1,000 pages ≈ 0.5 credits |
| AI_PARSE_DOCUMENT (LAYOUT) | 3.33 credits / 1,000 pages | 1,000 pages ≈ 3.33 credits |
| AI_CLASSIFY | Credits based on input tokens (same as AI_COMPLETE) | Text: minimal; PDF: +parse cost |
| AI_COMPLETE (claude-3-5-sonnet) | 1.50 input / 7.50 output per 1M tokens | 100 images ≈ 0.60 credits |

**Detailed pricing with formulas:** See [AI_EXTRACT](../references/ai-extract.md), [AI_PARSE_DOCUMENT](../references/ai-parse-doc.md), [AI_COMPLETE](../references/ai-complete.md)

**Never skip displaying pricing.** This is mandatory before any AI function execution.

---

## Reference Files

**Important:** All paths below are relative to this file's directory (`document-intelligence/`).

| Reference | Location | Use For |
|-----------|----------|---------|
| Extraction | `references/extraction.md` | Structured field/table extraction (AI_EXTRACT) |
| Parsing | `references/parsing.md` | Full text parsing (AI_PARSE_DOCUMENT) |
| Visual Analysis | `references/visual-analysis.md` | Charts, blueprints, diagrams (AI_COMPLETE) |
| Classification | `references/document_classification.md` | Categorize/sort/triage documents (AI_CLASSIFY) |
| Pipeline | `references/pipeline.md` | Post-processing, storage, automation |

## When to Use

- User wants to extract data from documents (PDFs, images, Office files)
- User mentions AI_EXTRACT or AI_PARSE_DOCUMENT
- User wants to process invoices, contracts, forms, reports
- User needs to analyze charts, blueprints, diagrams
- User wants to classify, categorize, sort, or triage documents
- User mentions AI_CLASSIFY with documents/files

---

## Workflow

1. **Determine extraction goal** - Ask user what they want to extract [WAIT]
2. **Get file location** - Snowflake stage, local file, or cloud storage [WAIT]
3. **Validate file type** - Infer from extension, check compatibility
4. **Execute flow** - Load appropriate sub-skill
5. **Post-processing** - Offer storage and pipeline options

---

## Step 1: Determine Extraction Goal [WAIT]

Ask the user what they want to extract:

```
What would you like to do with your documents?

1. Structured extraction - Specific fields or tables (e.g., invoice number, line items)
2. Full text parsing - Complete document text with or without layout preserved
3. Visual analysis - Charts, graphs, diagrams, blueprints
4. Classification - Sort/categorize/triage documents by type

Or describe what you need in your own words.
```

**Route based on response:**

> **⚠️ Priority Rule 1 - Visual Documents:** When the source content is a visual document type (blueprint, drawing, engineering drawing, engineering diagram, complex diagram, technical drawing, diagram, schematic, chart, graph), **ALWAYS route to Visual Analysis** — even if the user also mentions extracting specific fields or data. Visual documents require AI_COMPLETE with a vision model, not AI_EXTRACT.

> **⚠️ Priority Rule 2 - Structured Extraction:** When the user mentions ANY of these keywords, **ALWAYS route to AI_EXTRACT** (not AI_PARSE_DOCUMENT):
> - "extract fields", "field names", "specific fields", "named fields"
> - "structured output", "structured data", "structured extraction"
> - "JSON output", "return as JSON", "get as object"
> - Specific field names like "invoice_number", "date", "amount", "total", "line_items"
> - "extract [something specific]" (e.g., "extract the date", "extract amounts")

| User Mentions | Flow | Load |
|---------------|------|------|
| **Structured extraction keywords:** extract fields, field names, structured output, structured data, JSON output, specific fields | **Extraction** | `references/extraction.md` |
| Specific fields, names, dates, amounts, IDs, invoice_number | Extraction | `references/extraction.md` |
| Tables with columns, line items, rows of data | Extraction | `references/extraction.md` |
| "Extract [specific thing]" (date, amount, name, etc.) | Extraction | `references/extraction.md` |
| Full text, all content, complete document, entire text | Parsing | `references/parsing.md` |
| OCR, scanned document, read text, get text | Parsing | `references/parsing.md` |
| Charts, graphs, plots | Visual Analysis | `references/visual-analysis.md` |
| Diagrams, complex diagrams, engineering diagrams, blueprints, schematics, drawings, engineering drawings, technical drawings | Visual Analysis | `references/visual-analysis.md` |

**Key distinction:**
- **AI_EXTRACT** = User wants specific fields/values as structured JSON (e.g., `{"invoice_number": "123", "date": "2024-01-01"}`)
- **AI_PARSE_DOCUMENT** = User wants full document text (no field extraction)

**If multiple types needed:** Execute flows sequentially, completing each before starting the next.

---

## Step 2: Get File Location [WAIT]

Ask where files are stored:

```
Where is your file located?
1. Snowflake stage (e.g., @my_db.my_schema.my_stage)
2. Local file on my computer
3. Cloud storage (S3, Azure, GCS, Google Drive, etc.)
```

**Snowflake stage:** Get full path and proceed.

**Local file:** You must ask for the full stage path. Do not create any stages or run any SQL until the user provides this information.

Ask: "Which database, schema, and stage name should I use? (e.g., MY_DB.MY_SCHEMA.MY_STAGE)"

Use the exact stage name the user provides. After user responds, create stage with server-side encryption:
```sql
CREATE STAGE IF NOT EXISTS db.schema.user_provided_stage_name
  DIRECTORY = (ENABLE = TRUE)
  ENCRYPTION = (TYPE = 'SNOWFLAKE_SSE');
```
Upload using: `snow stage copy "<local_path>" @db.schema.user_provided_stage_name`

**Cloud storage:** Load `openflow` skill to set up connector.

---

## Step 3: Validate File Type

Infer file type from extension. Do NOT ask user.

**Supported formats:**

| Extension | AI_EXTRACT | AI_PARSE_DOCUMENT | AI_COMPLETE |
|-----------|------------|-------------------|-------------|
| .pdf | Yes | Yes | Yes (convert to image) |
| .png, .jpg, .jpeg, .tiff, .webp | Yes | Yes | Yes |
| .docx, .pptx | Yes | Yes | No |
| .html, .txt | Yes | Yes | No |
| .csv, .md, .eml | Yes | No | No |
| .xlsx, .xls, .doc, .ppt | No | No | No |

**Unsupported formats:** Suggest exporting to PDF or loading directly into Snowflake.

---

## Step 4: Execute Flow

Load the appropriate sub-skill to create sql based on Step 1:
## ⚠️ CRITICAL: always use the AI_* function names WITHOUT the `SNOWFLAKE.CORTEX` namespace prefix

- Structured fields/tables → `references/extraction.md`
- Full text/OCR → `references/parsing.md`
- Charts/diagrams/blueprints → `references/visual-analysis.md`
- Classify/categorize/sort documents → `references/document_classification.md`

---

## Step 5: Post-Processing [WAIT]

Each workflow skill (extraction.md, parsing.md, visual-analysis.md, document_classification.md) handles its own post-processing menu and "Store Results" SQL in its Step 7.

If the user chooses to set up a **pipeline**, load `references/pipeline.md`. The routing table at the top of pipeline.md maps each workflow to its correct template.

**Multi-step pipelines:** If the session involved multiple chained flows (e.g., classify → extract, or classify → extract → summarize), use `references/pipeline.md` **Template E**. This creates a root task for the first operation and child tasks (via `AFTER`) for each subsequent step, so new files are automatically processed through the full chain.

---

## Key Constraints

| Function | Max File Size | Max Pages | Output |
|----------|---------------|-----------|--------|
| AI_EXTRACT | 100 MB | 125* | JSON |
| AI_PARSE_DOCUMENT | 50 MB | 500 | Text |
| AI_COMPLETE | 10 MB (image) | N/A | Text |

*AI_EXTRACT max pages decreases with scale_factor: `floor(125 / scale_factor)`. E.g., scale_factor 2.0 → 62 pages, scale_factor 4.0 → 31 pages.

---

## Follow-Up Requests

When user asks a follow-up question or wants to extract something different, re-evaluate which flow is best. Do not assume the same flow applies.

| User Wants | Best Flow | Why |
|------------|-----------|-----|
| Named fields (invoice_number, date, amount) | **Extraction** | Returns structured JSON with specific values |
| "Extract fields", "field names", "structured output" | **Extraction** | Explicit request for structured data |
| Table data with known columns, line items | **Extraction** | Returns array of structured objects |
| "Extract the [specific thing]" | **Extraction** | Specific value extraction = AI_EXTRACT |
| Full document text, all content | Parsing | Returns full text |
| OCR from scanned documents, read text | Parsing | Full text extraction without structure |
| Chart/graph interpretation | Visual Analysis | Requires vision model |
| Blueprint/diagram/drawing analysis | Visual Analysis | Requires vision model |

**Remember:** If user says "extract" + specific field/value → **AI_EXTRACT**. If user says "get all text" or "full content" → AI_PARSE_DOCUMENT.

**IMPORTANT:** If Step 5 (Post-Processing) has not been completed, always present the post-processing options before ending the session, even if the user says they don't need additional extractions.

---

## Stopping Points

| After Step | Wait For |
|------------|----------|
| Step 1 | Extraction goal selection |
| Step 2 | File location |
| Step 5 | Post-processing choice |

## Output

Routes user to appropriate sub-skill, then assists with post-processing options.
