soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

feat: add intelligent import for CSV, Excel, and PDF files

Browse source 
Add IntelligentImportAugmentation with support for:
- CSV: auto-detection of encoding, delimiters, and field types
- Excel: multi-sheet extraction with metadata preservation
- PDF: text extraction, table detection, and metadata extraction

Features:
- Automatic format detection from file extension or content
- Intelligent type inference (string, number, boolean, date)
- Seamless integration with neural entity extraction
- Production-ready with 69 comprehensive tests

Dependencies added:
- xlsx@^0.18.5 for Excel parsing
- pdfjs-dist@^4.0.379 for PDF parsing
- csv-parse@^6.1.0 for CSV parsing
- chardet@^2.0.0 for encoding detection

Documentation:
- Updated README with import examples
- Updated API_REFERENCE with comprehensive import() docs
- Updated import-anything.md guide
- Added working example: import-excel-pdf-csv.ts
- Updated augmentations README
This commit is contained in:
David Snelling 2025-10-01 16:51:03 -07:00
parent aaf8e0f411
commit 814cbb48ee
33 changed files with 4664 additions and 28 deletions
Download patch file Download diff file Expand all files Collapse all files

5
tests/fixtures/import/tab-delimited.csv vendored Normal file
View file

@ -0,0 +1,5 @@
city country population area
Tokyo Japan 13960000 2191
New York USA 8336000 783.8
London UK 9002000 1572
Paris France 2161000 105.4
1 city country population area
2 Tokyo Japan 13960000 2191
3 New York USA 8336000 783.8
4 London UK 9002000 1572
5 Paris France 2161000 105.4