Implements Phase 1.5 (Comprehensive MIME Type Detection) and adds built-in image processing support to IntelligentImportAugmentation.
**New Features:**
- ImageHandler: Extracts image metadata (dimensions, format, color space) using sharp
- EXIF extraction: Camera data, GPS, timestamps using exifr library
- Support for JPEG, PNG, WebP, GIF, TIFF, BMP, SVG, HEIC, AVIF formats
- MimeTypeDetector: Unified MIME type detection with magic byte support
- FormatDetector: Enhanced with image format detection via MIME + magic bytes
**Architecture Fixes:**
- Fixed brain.import() augmentation pipeline integration (src/brainy.ts:3140-3154)
- Added parameter spreading for ImportSource objects to enable augmentation access
- Fixed metadata propagation through ImportCoordinator to final results
- Added augmentation data check in ImportCoordinator.extract()
**Integration:**
- ImageHandler registered as built-in handler alongside CSV, Excel, PDF
- Images import as 'media' entities with 'image' subtype
- Full metadata preserved in knowledge graph entities
- Configuration options: enableImage, extractEXIF, imageDefaults
**Test Coverage:**
- 15 integration tests (image-import.test.ts) - 100% passing
- 27 unit tests (image-handler.test.ts) - 100% passing
- Format detection tests for all supported image types
- Error handling and resilience tests
**Breaking Changes:** None - backward compatible
Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
Add real-time progress reporting throughout the entire import pipeline
with a standardized API that works across all supported formats.
Workshop Team Feature Request:
- Eliminates "0% complete" hangs during AI extraction
- Shows continuous progress with entities/sec, throughput, ETA
- Reports contextual messages ("Processing page 5 of 23")
- Standardized progress API for CSV, PDF, Excel, JSON, Markdown, YAML, DOCX
Core Changes:
- Add FormatHandlerProgressHooks interface for extensible progress
- Wire up all 3 binary format handlers (CSV, PDF, Excel) with 7+ progress points
- Wire up all 4 text format importers (JSON, Markdown, YAML, DOCX)
- Add ImportProgress interface with stage, message, counts, throughput, ETA
- ImportCoordinator normalizes all format progress to standard interface
CLI Improvements:
- Import command now uses brain.import() directly with full progress
- Add --include-vfs flag to find command (v4.4.0 compatibility)
- Add --confidence and --weight options to add command
Documentation:
- docs/guides/standard-import-progress.md - Universal API guide
- docs/guides/import-progress-implementation.md - Developer guide
- docs/guides/import-progress-examples.md - Practical examples
- JSDoc on brain.import() with universal handler examples
Result: ONE progress handler works for ALL 7 formats with zero format-specific code!
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add IntelligentImportAugmentation with support for:
- CSV: auto-detection of encoding, delimiters, and field types
- Excel: multi-sheet extraction with metadata preservation
- PDF: text extraction, table detection, and metadata extraction
Features:
- Automatic format detection from file extension or content
- Intelligent type inference (string, number, boolean, date)
- Seamless integration with neural entity extraction
- Production-ready with 69 comprehensive tests
Dependencies added:
- xlsx@^0.18.5 for Excel parsing
- pdfjs-dist@^4.0.379 for PDF parsing
- csv-parse@^6.1.0 for CSV parsing
- chardet@^2.0.0 for encoding detection
Documentation:
- Updated README with import examples
- Updated API_REFERENCE with comprehensive import() docs
- Updated import-anything.md guide
- Added working example: import-excel-pdf-csv.ts
- Updated augmentations README
