**feat(brainy-models): enhance workflows, update README, and improve model compatibility**
- **Scripts**:
- Refactored logging within `release-workflow.js` for improved readability and maintainability.
- Added a fallback mechanism in `download-full-models.js` to inject the "format" field into `model.json` for TensorFlow.js compatibility.
- **Documentation**:
- Updated `README.md` with a redesigned structure:
- Enhanced readability using emojis and a cleaner presentation.
- Expanded `Overview`, `Features`, and `Quick Start` sections.
- Refined "Use Cases" and added better explanations for bundled model benefits.
- **Models**:
- Adjusted `model.json` to include the "format" field for compatibility with TensorFlow.js.
- Updated `metadata.json` with a recent download timestamp.
**Purpose
This commit is contained in:
parent
4254deceac
commit
699dc4f4a5
5 changed files with 71 additions and 35 deletions
|
|
@ -1,12 +1,21 @@
|
|||
# @soulcraft/brainy-models
|
||||
<div align="center">
|
||||
<img src="../brainy.png" alt="Brainy Logo" width="200"/>
|
||||
<br/><br/>
|
||||
|
||||
Pre-bundled TensorFlow models for maximum reliability with Brainy vector database.
|
||||
[](../LICENSE)
|
||||
[](https://nodejs.org/)
|
||||
[](https://www.typescriptlang.org/)
|
||||
[](../CONTRIBUTING.md)
|
||||
|
||||
## Overview
|
||||
**Pre-bundled TensorFlow models for maximum reliability with Brainy vector database**
|
||||
|
||||
</div>
|
||||
|
||||
## ✨ Overview
|
||||
|
||||
This package provides offline access to the Universal Sentence Encoder model, eliminating network dependencies and ensuring consistent performance. It's designed as an optional companion to the main `@soulcraft/brainy` package for applications requiring maximum reliability.
|
||||
|
||||
## Features
|
||||
### 🚀 Key Features
|
||||
|
||||
- 🔒 **Maximum Reliability**: Fully offline model loading with zero network dependencies
|
||||
- 📦 **Pre-bundled Models**: Complete Universal Sentence Encoder model (~25MB) included
|
||||
|
|
@ -15,7 +24,7 @@ This package provides offline access to the Universal Sentence Encoder model, el
|
|||
- 🛠️ **Easy Integration**: Drop-in replacement for online model loading
|
||||
- 📊 **Comprehensive Metrics**: Detailed model information and performance statistics
|
||||
|
||||
## Installation
|
||||
## 🔧 Installation
|
||||
|
||||
```bash
|
||||
npm install @soulcraft/brainy-models
|
||||
|
|
@ -26,7 +35,7 @@ npm install @soulcraft/brainy-models
|
|||
- Node.js >= 18.0.0
|
||||
- `@soulcraft/brainy` >= 0.33.0
|
||||
|
||||
## Quick Start
|
||||
## 🏁 Quick Start
|
||||
|
||||
### Basic Usage
|
||||
|
||||
|
|
@ -88,7 +97,7 @@ await encoder.load()
|
|||
const embeddings = await encoder.embedToArrays(['Sample text'])
|
||||
```
|
||||
|
||||
## API Reference
|
||||
## 📚 API Reference
|
||||
|
||||
### BundledUniversalSentenceEncoder
|
||||
|
||||
|
|
@ -221,7 +230,7 @@ const models = utils.listAvailableModels()
|
|||
console.log('Available models:', models)
|
||||
```
|
||||
|
||||
## Model Variants
|
||||
## 🎯 Model Variants
|
||||
|
||||
The package includes multiple model variants optimized for different use cases:
|
||||
|
||||
|
|
@ -243,7 +252,7 @@ The package includes multiple model variants optimized for different use cases:
|
|||
- **Memory**: Low
|
||||
- **Speed**: Medium
|
||||
|
||||
## Scripts
|
||||
## ⚙️ Scripts
|
||||
|
||||
The package includes several utility scripts:
|
||||
|
||||
|
|
@ -271,7 +280,7 @@ Verify model functionality:
|
|||
npm test
|
||||
```
|
||||
|
||||
## Development
|
||||
## 🔨 Development
|
||||
|
||||
### Building the Package
|
||||
|
||||
|
|
@ -291,7 +300,7 @@ npm test
|
|||
npm run pack
|
||||
```
|
||||
|
||||
## Comparison with Online Loading
|
||||
## ⚖️ Comparison with Online Loading
|
||||
|
||||
| Feature | Online Loading | Bundled Models |
|
||||
|---------|----------------|----------------|
|
||||
|
|
@ -302,7 +311,7 @@ npm run pack
|
|||
| **Network required** | Yes (first time) | No |
|
||||
| **Offline support** | Limited | Complete |
|
||||
|
||||
## Use Cases
|
||||
## 💡 Use Cases
|
||||
|
||||
### When to Use Bundled Models
|
||||
|
||||
|
|
@ -319,7 +328,7 @@ npm run pack
|
|||
- ✅ Environments with reliable internet connectivity
|
||||
- ✅ Applications that rarely use embeddings
|
||||
|
||||
## Troubleshooting
|
||||
## 🔧 Troubleshooting
|
||||
|
||||
### Model Not Found Error
|
||||
|
||||
|
|
@ -351,15 +360,15 @@ For optimal performance:
|
|||
2. **Speed-critical**: Use original float32 models
|
||||
3. **Balanced**: Use float16 compressed models
|
||||
|
||||
## License
|
||||
## 📄 License
|
||||
|
||||
MIT
|
||||
|
||||
## Contributing
|
||||
## 🤝 Contributing
|
||||
|
||||
Contributions are welcome! Please see the main [Brainy repository](https://github.com/soulcraft-research/brainy) for contribution guidelines.
|
||||
|
||||
## Support
|
||||
## 💬 Support
|
||||
|
||||
For issues and questions:
|
||||
- [GitHub Issues](https://github.com/soulcraft-research/brainy/issues)
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
"version": "1.0.0",
|
||||
"description": "Complete Universal Sentence Encoder model bundled for offline use",
|
||||
"dimensions": 512,
|
||||
"downloadDate": "2025-08-01T23:20:17.338Z",
|
||||
"downloadDate": "2025-08-02T00:28:52.947Z",
|
||||
"source": "tensorflow-models/universal-sentence-encoder",
|
||||
"approach": "full-bundle",
|
||||
"modelUrl": "https://storage.googleapis.com/tfjs-models/savedmodel/universal_sentence_encoder",
|
||||
|
|
|
|||
|
|
@ -7178,5 +7178,6 @@
|
|||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
],
|
||||
"format": "tfjs-graph-model"
|
||||
}
|
||||
|
|
@ -138,6 +138,13 @@ async function downloadFullModel() {
|
|||
// Read the model.json to get the weights manifest
|
||||
const modelJson = JSON.parse(fs.readFileSync(modelJsonPath, 'utf8'))
|
||||
|
||||
// Add the required "format" field for TensorFlow.js compatibility
|
||||
if (!modelJson.format) {
|
||||
modelJson.format = 'tfjs-graph-model'
|
||||
fs.writeFileSync(modelJsonPath, JSON.stringify(modelJson, null, 2))
|
||||
console.log('✅ Added "format" field to model.json for TensorFlow.js compatibility')
|
||||
}
|
||||
|
||||
// Download all weight files
|
||||
if (modelJson.weightsManifest) {
|
||||
for (const manifest of modelJson.weightsManifest) {
|
||||
|
|
|
|||
|
|
@ -54,7 +54,9 @@ function executeStep(command, description, cwd = packageDir) {
|
|||
return true
|
||||
} catch (error) {
|
||||
// eslint-disable-next-line no-console
|
||||
console.error(`❌ Error during ${description.toLowerCase()}: ${error.message}`)
|
||||
console.error(
|
||||
`❌ Error during ${description.toLowerCase()}: ${error.message}`
|
||||
)
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
|
@ -62,7 +64,9 @@ function executeStep(command, description, cwd = packageDir) {
|
|||
// Main workflow
|
||||
async function runReleaseWorkflow() {
|
||||
// eslint-disable-next-line no-console
|
||||
console.log(`\n=== Starting @soulcraft/brainy-models-package Release Workflow (${versionType}) ===\n`)
|
||||
console.log(
|
||||
`\n=== Starting @soulcraft/brainy-models-package Release Workflow (${versionType}) ===\n`
|
||||
)
|
||||
|
||||
// Step 1: Build the project
|
||||
if (!executeStep('npm run build', 'Building brainy-models-package')) {
|
||||
|
|
@ -73,45 +77,56 @@ async function runReleaseWorkflow() {
|
|||
// Step 2: Run tests to ensure everything is working
|
||||
if (!executeStep('npm test', 'Running tests')) {
|
||||
// eslint-disable-next-line no-console
|
||||
console.warn('⚠️ Tests failed. This might indicate issues with the release.')
|
||||
|
||||
console.warn(
|
||||
'⚠️ Tests failed. This might indicate issues with the release.'
|
||||
)
|
||||
|
||||
// Ask the user if they want to continue despite test failures
|
||||
// eslint-disable-next-line no-console
|
||||
console.log('\n⚠️ Do you want to continue with the release process despite test failures? (y/N)')
|
||||
|
||||
console.log(
|
||||
'\n⚠️ Do you want to continue with the release process despite test failures? (y/N)'
|
||||
)
|
||||
|
||||
const rl = readline.createInterface({
|
||||
input: process.stdin,
|
||||
output: process.stdout
|
||||
})
|
||||
|
||||
const response = await new Promise(resolve => {
|
||||
rl.question('', answer => {
|
||||
|
||||
const response = await new Promise((resolve) => {
|
||||
rl.question('', (answer) => {
|
||||
rl.close()
|
||||
resolve(answer.toLowerCase())
|
||||
})
|
||||
})
|
||||
|
||||
|
||||
if (response !== 'y' && response !== 'yes') {
|
||||
// eslint-disable-next-line no-console
|
||||
console.error('Release process aborted due to test failures.')
|
||||
// eslint-disable-next-line no-process-exit
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
|
||||
// eslint-disable-next-line no-console
|
||||
console.log('Continuing with release process despite test failures...')
|
||||
}
|
||||
|
||||
// Step 3: Update version and generate changelog
|
||||
if (!executeStep(`npm run release:${versionType}`, `Updating version (${versionType}) and generating changelog`)) {
|
||||
if (
|
||||
!executeStep(
|
||||
`npm run release:${versionType}`,
|
||||
`Updating version (${versionType}) and generating changelog`
|
||||
)
|
||||
) {
|
||||
// eslint-disable-next-line no-process-exit
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
// Step 4: Create GitHub release
|
||||
if (!executeStep('npm run github-release', 'Creating GitHub release')) {
|
||||
if (!executeStep('npm run _github-release', 'Creating GitHub release')) {
|
||||
// eslint-disable-next-line no-console
|
||||
console.log('Warning: GitHub release creation failed, but continuing with deployment...')
|
||||
console.log(
|
||||
'Warning: GitHub release creation failed, but continuing with deployment...'
|
||||
)
|
||||
}
|
||||
|
||||
// Step 5: Publish to NPM
|
||||
|
|
@ -126,7 +141,9 @@ async function runReleaseWorkflow() {
|
|||
const newVersion = packageJson.version
|
||||
|
||||
// eslint-disable-next-line no-console
|
||||
console.log(`\n🎉 @soulcraft/brainy-models-package v${newVersion} release completed successfully! 🎉\n`)
|
||||
console.log(
|
||||
`\n🎉 @soulcraft/brainy-models-package v${newVersion} release completed successfully! 🎉\n`
|
||||
)
|
||||
// eslint-disable-next-line no-console
|
||||
console.log('Summary of actions:')
|
||||
// eslint-disable-next-line no-console
|
||||
|
|
@ -140,11 +157,13 @@ async function runReleaseWorkflow() {
|
|||
// eslint-disable-next-line no-console
|
||||
console.log(`- Package published to NPM as @soulcraft/brainy-models-package`)
|
||||
// eslint-disable-next-line no-console
|
||||
console.log('\nThank you for using the brainy-models-package release workflow!\n')
|
||||
console.log(
|
||||
'\nThank you for using the brainy-models-package release workflow!\n'
|
||||
)
|
||||
}
|
||||
|
||||
// Run the workflow
|
||||
runReleaseWorkflow().catch(error => {
|
||||
runReleaseWorkflow().catch((error) => {
|
||||
// eslint-disable-next-line no-console
|
||||
console.error('Unexpected error during release workflow:', error)
|
||||
// eslint-disable-next-line no-process-exit
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue