soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

feat: add intelligent import for CSV, Excel, and PDF files

Browse source 
Add IntelligentImportAugmentation with support for:
- CSV: auto-detection of encoding, delimiters, and field types
- Excel: multi-sheet extraction with metadata preservation
- PDF: text extraction, table detection, and metadata extraction

Features:
- Automatic format detection from file extension or content
- Intelligent type inference (string, number, boolean, date)
- Seamless integration with neural entity extraction
- Production-ready with 69 comprehensive tests

Dependencies added:
- xlsx@^0.18.5 for Excel parsing
- pdfjs-dist@^4.0.379 for PDF parsing
- csv-parse@^6.1.0 for CSV parsing
- chardet@^2.0.0 for encoding detection

Documentation:
- Updated README with import examples
- Updated API_REFERENCE with comprehensive import() docs
- Updated import-anything.md guide
- Added working example: import-excel-pdf-csv.ts
- Updated augmentations README
This commit is contained in:
David Snelling 2025-10-01 16:51:03 -07:00
parent aaf8e0f411
commit 814cbb48ee
33 changed files with 4664 additions and 28 deletions
Download patch file Download diff file Expand all files Collapse all files

5
tests/fixtures/import/simple.csv vendored Normal file
View file

@ -0,0 +1,5 @@
name,age,email,active
John Doe,30,john@example.com,true
Jane Smith,25,jane@example.com,false
Bob Johnson,45,bob@example.com,true
Alice Williams,35,alice@example.com,true
1 name age email active
2 John Doe 30 john@example.com true
3 Jane Smith 25 jane@example.com false
4 Bob Johnson 45 bob@example.com true
5 Alice Williams 35 alice@example.com true