**feat(brainy-models): enhance workflows, update README, and improve model compatibility**

- **Scripts**:
  - Refactored logging within `release-workflow.js` for improved readability and maintainability.
  - Added a fallback mechanism in `download-full-models.js` to inject the "format" field into `model.json` for TensorFlow.js compatibility.

- **Documentation**:
  - Updated `README.md` with a redesigned structure:
    - Enhanced readability using emojis and a cleaner presentation.
    - Expanded `Overview`, `Features`, and `Quick Start` sections.
    - Refined "Use Cases" and added better explanations for bundled model benefits.

- **Models**:
  - Adjusted `model.json` to include the "format" field for compatibility with TensorFlow.js.
  - Updated `metadata.json` with a recent download timestamp.

**Purpose
This commit is contained in:
David Snelling 2025-08-01 17:33:13 -07:00
parent 92e2f017d5
commit a8a95a84f4
5 changed files with 71 additions and 35 deletions

View file

@ -1,12 +1,21 @@
# @soulcraft/brainy-models <div align="center">
<img src="../brainy.png" alt="Brainy Logo" width="200"/>
<br/><br/>
Pre-bundled TensorFlow models for maximum reliability with Brainy vector database. [![License](https://img.shields.io/badge/license-MIT-green.svg)](../LICENSE)
[![Node.js](https://img.shields.io/badge/node-%3E%3D18.0.0-brightgreen.svg)](https://nodejs.org/)
[![TypeScript](https://img.shields.io/badge/TypeScript-5.4.5-blue.svg)](https://www.typescriptlang.org/)
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](../CONTRIBUTING.md)
## Overview **Pre-bundled TensorFlow models for maximum reliability with Brainy vector database**
</div>
## ✨ Overview
This package provides offline access to the Universal Sentence Encoder model, eliminating network dependencies and ensuring consistent performance. It's designed as an optional companion to the main `@soulcraft/brainy` package for applications requiring maximum reliability. This package provides offline access to the Universal Sentence Encoder model, eliminating network dependencies and ensuring consistent performance. It's designed as an optional companion to the main `@soulcraft/brainy` package for applications requiring maximum reliability.
## Features ### 🚀 Key Features
- 🔒 **Maximum Reliability**: Fully offline model loading with zero network dependencies - 🔒 **Maximum Reliability**: Fully offline model loading with zero network dependencies
- 📦 **Pre-bundled Models**: Complete Universal Sentence Encoder model (~25MB) included - 📦 **Pre-bundled Models**: Complete Universal Sentence Encoder model (~25MB) included
@ -15,7 +24,7 @@ This package provides offline access to the Universal Sentence Encoder model, el
- 🛠️ **Easy Integration**: Drop-in replacement for online model loading - 🛠️ **Easy Integration**: Drop-in replacement for online model loading
- 📊 **Comprehensive Metrics**: Detailed model information and performance statistics - 📊 **Comprehensive Metrics**: Detailed model information and performance statistics
## Installation ## 🔧 Installation
```bash ```bash
npm install @soulcraft/brainy-models npm install @soulcraft/brainy-models
@ -26,7 +35,7 @@ npm install @soulcraft/brainy-models
- Node.js >= 18.0.0 - Node.js >= 18.0.0
- `@soulcraft/brainy` >= 0.33.0 - `@soulcraft/brainy` >= 0.33.0
## Quick Start ## 🏁 Quick Start
### Basic Usage ### Basic Usage
@ -88,7 +97,7 @@ await encoder.load()
const embeddings = await encoder.embedToArrays(['Sample text']) const embeddings = await encoder.embedToArrays(['Sample text'])
``` ```
## API Reference ## 📚 API Reference
### BundledUniversalSentenceEncoder ### BundledUniversalSentenceEncoder
@ -221,7 +230,7 @@ const models = utils.listAvailableModels()
console.log('Available models:', models) console.log('Available models:', models)
``` ```
## Model Variants ## 🎯 Model Variants
The package includes multiple model variants optimized for different use cases: The package includes multiple model variants optimized for different use cases:
@ -243,7 +252,7 @@ The package includes multiple model variants optimized for different use cases:
- **Memory**: Low - **Memory**: Low
- **Speed**: Medium - **Speed**: Medium
## Scripts ## ⚙️ Scripts
The package includes several utility scripts: The package includes several utility scripts:
@ -271,7 +280,7 @@ Verify model functionality:
npm test npm test
``` ```
## Development ## 🔨 Development
### Building the Package ### Building the Package
@ -291,7 +300,7 @@ npm test
npm run pack npm run pack
``` ```
## Comparison with Online Loading ## ⚖️ Comparison with Online Loading
| Feature | Online Loading | Bundled Models | | Feature | Online Loading | Bundled Models |
|---------|----------------|----------------| |---------|----------------|----------------|
@ -302,7 +311,7 @@ npm run pack
| **Network required** | Yes (first time) | No | | **Network required** | Yes (first time) | No |
| **Offline support** | Limited | Complete | | **Offline support** | Limited | Complete |
## Use Cases ## 💡 Use Cases
### When to Use Bundled Models ### When to Use Bundled Models
@ -319,7 +328,7 @@ npm run pack
- ✅ Environments with reliable internet connectivity - ✅ Environments with reliable internet connectivity
- ✅ Applications that rarely use embeddings - ✅ Applications that rarely use embeddings
## Troubleshooting ## 🔧 Troubleshooting
### Model Not Found Error ### Model Not Found Error
@ -351,15 +360,15 @@ For optimal performance:
2. **Speed-critical**: Use original float32 models 2. **Speed-critical**: Use original float32 models
3. **Balanced**: Use float16 compressed models 3. **Balanced**: Use float16 compressed models
## License ## 📄 License
MIT MIT
## Contributing ## 🤝 Contributing
Contributions are welcome! Please see the main [Brainy repository](https://github.com/soulcraft-research/brainy) for contribution guidelines. Contributions are welcome! Please see the main [Brainy repository](https://github.com/soulcraft-research/brainy) for contribution guidelines.
## Support ## 💬 Support
For issues and questions: For issues and questions:
- [GitHub Issues](https://github.com/soulcraft-research/brainy/issues) - [GitHub Issues](https://github.com/soulcraft-research/brainy/issues)

View file

@ -3,7 +3,7 @@
"version": "1.0.0", "version": "1.0.0",
"description": "Complete Universal Sentence Encoder model bundled for offline use", "description": "Complete Universal Sentence Encoder model bundled for offline use",
"dimensions": 512, "dimensions": 512,
"downloadDate": "2025-08-01T23:20:17.338Z", "downloadDate": "2025-08-02T00:28:52.947Z",
"source": "tensorflow-models/universal-sentence-encoder", "source": "tensorflow-models/universal-sentence-encoder",
"approach": "full-bundle", "approach": "full-bundle",
"modelUrl": "https://storage.googleapis.com/tfjs-models/savedmodel/universal_sentence_encoder", "modelUrl": "https://storage.googleapis.com/tfjs-models/savedmodel/universal_sentence_encoder",

View file

@ -7178,5 +7178,6 @@
} }
] ]
} }
] ],
"format": "tfjs-graph-model"
} }

View file

@ -138,6 +138,13 @@ async function downloadFullModel() {
// Read the model.json to get the weights manifest // Read the model.json to get the weights manifest
const modelJson = JSON.parse(fs.readFileSync(modelJsonPath, 'utf8')) const modelJson = JSON.parse(fs.readFileSync(modelJsonPath, 'utf8'))
// Add the required "format" field for TensorFlow.js compatibility
if (!modelJson.format) {
modelJson.format = 'tfjs-graph-model'
fs.writeFileSync(modelJsonPath, JSON.stringify(modelJson, null, 2))
console.log('✅ Added "format" field to model.json for TensorFlow.js compatibility')
}
// Download all weight files // Download all weight files
if (modelJson.weightsManifest) { if (modelJson.weightsManifest) {
for (const manifest of modelJson.weightsManifest) { for (const manifest of modelJson.weightsManifest) {

View file

@ -54,7 +54,9 @@ function executeStep(command, description, cwd = packageDir) {
return true return true
} catch (error) { } catch (error) {
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
console.error(`❌ Error during ${description.toLowerCase()}: ${error.message}`) console.error(
`❌ Error during ${description.toLowerCase()}: ${error.message}`
)
return false return false
} }
} }
@ -62,7 +64,9 @@ function executeStep(command, description, cwd = packageDir) {
// Main workflow // Main workflow
async function runReleaseWorkflow() { async function runReleaseWorkflow() {
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
console.log(`\n=== Starting @soulcraft/brainy-models-package Release Workflow (${versionType}) ===\n`) console.log(
`\n=== Starting @soulcraft/brainy-models-package Release Workflow (${versionType}) ===\n`
)
// Step 1: Build the project // Step 1: Build the project
if (!executeStep('npm run build', 'Building brainy-models-package')) { if (!executeStep('npm run build', 'Building brainy-models-package')) {
@ -73,45 +77,56 @@ async function runReleaseWorkflow() {
// Step 2: Run tests to ensure everything is working // Step 2: Run tests to ensure everything is working
if (!executeStep('npm test', 'Running tests')) { if (!executeStep('npm test', 'Running tests')) {
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
console.warn('⚠️ Tests failed. This might indicate issues with the release.') console.warn(
'⚠️ Tests failed. This might indicate issues with the release.'
)
// Ask the user if they want to continue despite test failures // Ask the user if they want to continue despite test failures
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
console.log('\n⚠ Do you want to continue with the release process despite test failures? (y/N)') console.log(
'\n⚠ Do you want to continue with the release process despite test failures? (y/N)'
)
const rl = readline.createInterface({ const rl = readline.createInterface({
input: process.stdin, input: process.stdin,
output: process.stdout output: process.stdout
}) })
const response = await new Promise(resolve => { const response = await new Promise((resolve) => {
rl.question('', answer => { rl.question('', (answer) => {
rl.close() rl.close()
resolve(answer.toLowerCase()) resolve(answer.toLowerCase())
}) })
}) })
if (response !== 'y' && response !== 'yes') { if (response !== 'y' && response !== 'yes') {
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
console.error('Release process aborted due to test failures.') console.error('Release process aborted due to test failures.')
// eslint-disable-next-line no-process-exit // eslint-disable-next-line no-process-exit
process.exit(1) process.exit(1)
} }
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
console.log('Continuing with release process despite test failures...') console.log('Continuing with release process despite test failures...')
} }
// Step 3: Update version and generate changelog // Step 3: Update version and generate changelog
if (!executeStep(`npm run release:${versionType}`, `Updating version (${versionType}) and generating changelog`)) { if (
!executeStep(
`npm run release:${versionType}`,
`Updating version (${versionType}) and generating changelog`
)
) {
// eslint-disable-next-line no-process-exit // eslint-disable-next-line no-process-exit
process.exit(1) process.exit(1)
} }
// Step 4: Create GitHub release // Step 4: Create GitHub release
if (!executeStep('npm run github-release', 'Creating GitHub release')) { if (!executeStep('npm run _github-release', 'Creating GitHub release')) {
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
console.log('Warning: GitHub release creation failed, but continuing with deployment...') console.log(
'Warning: GitHub release creation failed, but continuing with deployment...'
)
} }
// Step 5: Publish to NPM // Step 5: Publish to NPM
@ -126,7 +141,9 @@ async function runReleaseWorkflow() {
const newVersion = packageJson.version const newVersion = packageJson.version
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
console.log(`\n🎉 @soulcraft/brainy-models-package v${newVersion} release completed successfully! 🎉\n`) console.log(
`\n🎉 @soulcraft/brainy-models-package v${newVersion} release completed successfully! 🎉\n`
)
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
console.log('Summary of actions:') console.log('Summary of actions:')
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
@ -140,11 +157,13 @@ async function runReleaseWorkflow() {
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
console.log(`- Package published to NPM as @soulcraft/brainy-models-package`) console.log(`- Package published to NPM as @soulcraft/brainy-models-package`)
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
console.log('\nThank you for using the brainy-models-package release workflow!\n') console.log(
'\nThank you for using the brainy-models-package release workflow!\n'
)
} }
// Run the workflow // Run the workflow
runReleaseWorkflow().catch(error => { runReleaseWorkflow().catch((error) => {
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
console.error('Unexpected error during release workflow:', error) console.error('Unexpected error during release workflow:', error)
// eslint-disable-next-line no-process-exit // eslint-disable-next-line no-process-exit