2025-08-05 16:09:30 -07:00
|
|
|
<div align="center>
|
2025-06-24 11:41:30 -07:00
|
|
|
<img src="./brainy.png" alt="Brainy Logo" width="200"/>
|
|
|
|
|
<br/><br/>
|
|
|
|
|
|
|
|
|
|
[](LICENSE)
|
**feat(tests, docs, storage): add statistics storage tests and enhance documentation**
- **Tests**: Added new `statistics-storage.test.ts` to validate statistics storage functionality across scenarios including saving, retrieving, time-based partitioning, and backward compatibility. Ensured tests dynamically handle missing environment variables by skipping S3-related tests when credentials are unavailable.
- **Docs**: Enhanced `statistics.md` with detailed explanations of scalability improvements, including adaptive flush timing, batched updates, and time-based partitioning. Improved readability and structure.
- **Storage**: Updated all storage adapters to integrate time-based partitioning and maintain backward compatibility with legacy statistics storage formats.
- **Dependencies**: Added `dotenv` to support environmental variable management for storage adapter tests.
**Purpose**: Strengthen system reliability by adding comprehensive test coverage for statistics storage, improve scalability documentation, and ensure consistency across storage adapters with robust implementations.
2025-07-24 16:24:02 -07:00
|
|
|
[](https://nodejs.org/)
|
2025-07-04 12:19:48 -07:00
|
|
|
[](https://www.typescriptlang.org/)
|
2025-06-24 11:41:30 -07:00
|
|
|
[](CONTRIBUTING.md)
|
|
|
|
|
|
2025-08-06 16:06:03 -07:00
|
|
|
**The world's only true Vector + Graph database - unified semantic search and knowledge graphs**
|
2025-06-24 11:41:30 -07:00
|
|
|
|
|
|
|
|
</div>
|
|
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
---
|
2025-08-06 14:39:33 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
# The Search Problem Every Developer Faces
|
2025-08-06 14:39:33 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
**"I need to find similar content, explore relationships, AND filter by metadata - but that means juggling 3+ databases"**
|
2025-08-05 19:29:59 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
❌ **Current Reality**: Pinecone + Neo4j + Elasticsearch + Custom Sync Logic
|
|
|
|
|
✅ **Brainy Reality**: One database. One API. All three search types.
|
2025-08-05 19:29:59 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
## 🔥 The Power of Three-in-One Search
|
2025-08-05 19:29:59 -07:00
|
|
|
|
2025-08-06 12:29:32 -07:00
|
|
|
```javascript
|
2025-08-06 16:26:39 -07:00
|
|
|
// This ONE query does what used to require 3 databases:
|
|
|
|
|
const results = await brainy.search("AI startups in healthcare", 10, {
|
|
|
|
|
// 🔍 Vector: Semantic similarity
|
|
|
|
|
includeVerbs: true,
|
|
|
|
|
|
|
|
|
|
// 🔗 Graph: Relationship traversal
|
|
|
|
|
verbTypes: ["invests_in", "partners_with"],
|
|
|
|
|
|
|
|
|
|
// 📊 Faceted: MongoDB-style filtering
|
2025-08-06 12:29:32 -07:00
|
|
|
metadata: {
|
2025-08-06 16:26:39 -07:00
|
|
|
industry: "healthcare",
|
|
|
|
|
funding: { $gte: 1000000 },
|
|
|
|
|
stage: { $in: ["Series A", "Series B"] }
|
2025-08-06 12:29:32 -07:00
|
|
|
}
|
|
|
|
|
})
|
2025-08-06 16:26:39 -07:00
|
|
|
// Returns: Companies similar to your query + their relationships + matching your criteria
|
2025-08-06 12:29:32 -07:00
|
|
|
```
|
|
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
**Three search paradigms. One lightning-fast query. Zero complexity.**
|
**feat(web-service): add new web service package and deployment scripts**
- **New Web Service Package**:
- Introduced `@soulcraft/brainy-web-service`, a REST API wrapper for the Brainy vector graph database.
- Added documentation and features to support secure, read-only search and retrieval operations.
- **Deployment Support**:
- Included comprehensive deployment instructions in the `web-service-package/README.md`:
- Options for Docker, serverless platforms, and cloud providers (AWS, GCP, Azure, Cloudflare).
- Example configurations for systemd, Nginx, and Docker Compose.
- **Scripts and Documentation Updates**:
- Added `deploy:web-service` script to `package.json` for streamlined build and publishing.
- Enhanced `README.md` to reflect the introduction of the web service package and its capabilities.
- **Purpose**:
- This update extends Brainy’s functionality by providing a production-ready, easy-to-deploy REST API for search operations. It ensures flexibility for diverse deployment scenarios while maintaining security and high performance.
2025-07-22 10:22:38 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
## 🚀 Install & Go
|
2025-08-05 15:34:39 -07:00
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
npm install @soulcraft/brainy
|
|
|
|
|
```
|
|
|
|
|
|
2025-08-04 14:33:39 -07:00
|
|
|
```javascript
|
2025-08-05 15:34:39 -07:00
|
|
|
import { BrainyData } from '@soulcraft/brainy'
|
2025-08-04 14:33:39 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
const brainy = new BrainyData() // Auto-detects your environment
|
|
|
|
|
await brainy.init() // Auto-configures everything
|
2025-08-04 14:33:39 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
// Add data with relationships
|
|
|
|
|
const openai = await brainy.add("OpenAI", { type: "company", funding: 11000000 })
|
|
|
|
|
const gpt4 = await brainy.add("GPT-4", { type: "product", users: 100000000 })
|
2025-08-06 09:52:45 -07:00
|
|
|
await brainy.relate(openai, gpt4, "develops")
|
2025-08-06 12:29:32 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
// Search across all dimensions
|
|
|
|
|
const results = await brainy.search("AI language models", 5, {
|
|
|
|
|
metadata: { funding: { $gte: 10000000 } },
|
|
|
|
|
includeVerbs: true
|
2025-08-06 12:29:32 -07:00
|
|
|
})
|
2025-08-04 08:56:05 -07:00
|
|
|
```
|
2025-08-04 14:33:39 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
**That's it. You just built a knowledge graph with semantic search and faceted filtering in 8 lines.**
|
2025-08-04 08:56:05 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
## 🏆 Why Brainy Wins
|
2025-08-05 16:09:30 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
- 🧠 **Triple Search Power** - Vector + Graph + Faceted filtering in one query
|
|
|
|
|
- 🌍 **Runs Everywhere** - Same code: React, Node.js, serverless, edge
|
|
|
|
|
- ⚡ **Zero Config** - Auto-detects environment, optimizes itself
|
|
|
|
|
- 🔄 **Always Synced** - No data consistency nightmares between systems
|
|
|
|
|
- 📦 **Truly Offline** - Works without internet after initial setup
|
|
|
|
|
- 🔒 **Your Data** - Run locally, in browser, or your own cloud
|
2025-08-03 17:22:32 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
## 🔮 Coming Soon
|
2025-08-05 15:34:39 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
- **🤖 MCP Integration** - Let Claude, GPT, and other AI models query your data directly
|
|
|
|
|
- **⚡ LLM Generation** - Built-in content generation powered by your knowledge graph
|
|
|
|
|
- **🌊 Real-time Sync** - Live updates across distributed instances
|
2025-08-05 15:34:39 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
## 🎯 Perfect For
|
2025-08-05 15:34:39 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
**🤖 AI Chat Applications** - ChatGPT-like apps with long-term memory and context
|
|
|
|
|
**🔍 Semantic Search** - Find "that thing like a cat but bigger" → returns "tiger"
|
|
|
|
|
**🧬 Knowledge Graphs** - Connect everything. Wikipedia meets Neo4j meets magic
|
|
|
|
|
**🎯 Recommendation Engines** - "Users who liked this also liked..." but actually good
|
|
|
|
|
**📚 Smart Documentation** - Docs that answer questions before you ask them
|
2025-08-05 15:34:39 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
## 🌍 Works Everywhere - Same Code
|
2025-08-05 15:34:39 -07:00
|
|
|
|
2025-08-06 09:52:45 -07:00
|
|
|
```javascript
|
2025-08-06 16:26:39 -07:00
|
|
|
// This EXACT code works in ALL environments
|
2025-08-06 09:52:45 -07:00
|
|
|
import { BrainyData } from '@soulcraft/brainy'
|
|
|
|
|
|
|
|
|
|
const brainy = new BrainyData()
|
2025-08-06 16:26:39 -07:00
|
|
|
await brainy.init()
|
2025-08-06 09:52:45 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
// Works in: React, Vue, Angular, Node.js, Deno, Bun,
|
|
|
|
|
// Cloudflare Workers, Vercel Edge, AWS Lambda, browsers, anywhere
|
2025-08-05 15:34:39 -07:00
|
|
|
```
|
|
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
Brainy automatically detects and optimizes for your environment:
|
2025-08-06 12:29:32 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
| Environment | Storage | Optimization |
|
|
|
|
|
|-------------|---------|-------------|
|
|
|
|
|
| 🌐 Browser | OPFS | Web Workers, Memory Cache |
|
|
|
|
|
| 🟢 Node.js | FileSystem / S3 | Worker Threads, Clustering |
|
|
|
|
|
| ⚡ Serverless | S3 / Memory | Cold Start Optimization |
|
|
|
|
|
| 🔥 Edge | Memory / KV | Minimal Footprint |
|
2025-08-06 12:29:32 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
## 🆚 Why Not Just Use...?
|
2025-08-06 12:29:32 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
### vs. Multiple Databases
|
|
|
|
|
❌ **Pinecone + Neo4j + Elasticsearch** - 3 databases, sync nightmares, 3x the cost
|
|
|
|
|
✅ **Brainy** - One database, always synced, built-in intelligence
|
2025-08-05 15:34:39 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
### vs. Traditional Solutions
|
|
|
|
|
❌ **PostgreSQL + pgvector + extensions** - Complex setup, performance issues
|
|
|
|
|
✅ **Brainy** - Zero config, purpose-built for AI, works everywhere
|
2025-08-05 15:34:39 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
### vs. Cloud-Only Vector DBs
|
|
|
|
|
❌ **Pinecone/Weaviate/Qdrant** - Vendor lock-in, expensive, cloud-only
|
|
|
|
|
✅ **Brainy** - Run anywhere, your data stays yours, cost-effective
|
2025-08-05 15:34:39 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
### vs. Graph Databases with "Vector Features"
|
|
|
|
|
❌ **Neo4j + vector plugin** - Bolt-on solution, not native, limited
|
|
|
|
|
✅ **Brainy** - Native vector+graph architecture from the ground up
|
2025-08-06 12:29:32 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
## 📦 Advanced Features
|
2025-08-05 15:34:39 -07:00
|
|
|
|
2025-08-06 09:52:45 -07:00
|
|
|
<details>
|
2025-08-06 16:26:39 -07:00
|
|
|
<summary>🔧 <strong>MongoDB-Style Metadata Filtering</strong></summary>
|
2025-08-05 15:34:39 -07:00
|
|
|
|
|
|
|
|
```javascript
|
2025-08-06 16:26:39 -07:00
|
|
|
const results = await brainy.search("machine learning", 10, {
|
|
|
|
|
metadata: {
|
|
|
|
|
// Comparison operators
|
|
|
|
|
price: { $gte: 100, $lte: 1000 },
|
|
|
|
|
category: { $in: ["AI", "ML", "Data"] },
|
|
|
|
|
rating: { $gt: 4.5 },
|
|
|
|
|
|
|
|
|
|
// Logical operators
|
|
|
|
|
$and: [
|
|
|
|
|
{ status: "active" },
|
|
|
|
|
{ verified: true }
|
|
|
|
|
],
|
|
|
|
|
|
|
|
|
|
// Text operators
|
|
|
|
|
description: { $regex: "neural.*network", $options: "i" },
|
|
|
|
|
|
|
|
|
|
// Array operators
|
|
|
|
|
tags: { $includes: "tensorflow" }
|
2025-08-06 09:52:45 -07:00
|
|
|
}
|
2025-08-01 08:27:39 -07:00
|
|
|
})
|
2025-08-05 15:34:39 -07:00
|
|
|
```
|
|
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
**15+ operators supported**: `$gt`, `$gte`, `$lt`, `$lte`, `$eq`, `$ne`, `$in`, `$nin`, `$and`, `$or`, `$not`, `$regex`, `$includes`, `$exists`, `$size`
|
2025-08-05 15:34:39 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
</details>
|
2025-08-04 08:56:05 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
<details>
|
|
|
|
|
<summary>🔗 <strong>Graph Relationships & Traversal</strong></summary>
|
2025-08-04 14:33:39 -07:00
|
|
|
|
2025-08-04 12:18:58 -07:00
|
|
|
```javascript
|
2025-08-06 16:26:39 -07:00
|
|
|
// Create entities and relationships
|
|
|
|
|
const company = await brainy.add("OpenAI", { type: "company" })
|
|
|
|
|
const product = await brainy.add("GPT-4", { type: "product" })
|
|
|
|
|
const person = await brainy.add("Sam Altman", { type: "person" })
|
2025-08-04 12:18:58 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
// Create meaningful relationships
|
|
|
|
|
await brainy.relate(company, product, "develops")
|
|
|
|
|
await brainy.relate(person, company, "leads")
|
|
|
|
|
await brainy.relate(product, person, "created_by")
|
2025-08-04 12:18:58 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
// Traverse relationships
|
|
|
|
|
const products = await brainy.getVerbsBySource(company) // What OpenAI develops
|
|
|
|
|
const leaders = await brainy.getVerbsByTarget(company) // Who leads OpenAI
|
|
|
|
|
const connections = await brainy.findSimilar(product, {
|
|
|
|
|
relationType: "develops"
|
2025-08-04 12:18:58 -07:00
|
|
|
})
|
|
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
// Search with relationship context
|
|
|
|
|
const results = await brainy.search("AI models", 10, {
|
|
|
|
|
includeVerbs: true,
|
|
|
|
|
verbTypes: ["develops", "created_by"],
|
|
|
|
|
searchConnectedNouns: true
|
2025-08-04 12:18:58 -07:00
|
|
|
})
|
|
|
|
|
```
|
|
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
</details>
|
2025-08-05 15:34:39 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
<details>
|
|
|
|
|
<summary>🌐 <strong>Universal Storage & Deployment</strong></summary>
|
2025-08-05 15:34:39 -07:00
|
|
|
|
|
|
|
|
```javascript
|
2025-08-06 16:26:39 -07:00
|
|
|
// Development: File system
|
|
|
|
|
const dev = new BrainyData({
|
|
|
|
|
storage: { fileSystem: { path: './data' } }
|
2025-08-05 15:34:39 -07:00
|
|
|
})
|
|
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
// Production: S3/R2
|
|
|
|
|
const prod = new BrainyData({
|
|
|
|
|
storage: { s3Storage: { bucketName: 'my-vectors' } }
|
2025-08-05 15:34:39 -07:00
|
|
|
})
|
|
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
// Browser: OPFS
|
|
|
|
|
const browser = new BrainyData() // Auto-detects OPFS
|
2025-08-05 15:34:39 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
// Edge: Memory
|
|
|
|
|
const edge = new BrainyData({
|
|
|
|
|
storage: { memory: {} }
|
2025-08-05 15:34:39 -07:00
|
|
|
})
|
|
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
// Redis: High performance
|
|
|
|
|
const redis = new BrainyData({
|
|
|
|
|
storage: { redis: { connectionString: 'redis://...' } }
|
2025-08-04 10:14:33 -07:00
|
|
|
})
|
|
|
|
|
```
|
2025-08-04 08:56:05 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
**Extend with any storage**: MongoDB, PostgreSQL, DynamoDB - [see storage adapters guide](docs/api-reference/storage-adapters.md)
|
2025-08-05 15:34:39 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
</details>
|
2025-08-05 15:34:39 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
<details>
|
|
|
|
|
<summary>🐳 <strong>Docker & Cloud Deployment</strong></summary>
|
2025-08-05 15:34:39 -07:00
|
|
|
|
|
|
|
|
```dockerfile
|
2025-08-06 16:26:39 -07:00
|
|
|
# Production-ready Dockerfile
|
2025-08-05 20:39:39 -07:00
|
|
|
FROM node:24-slim AS builder
|
2025-08-05 15:34:39 -07:00
|
|
|
WORKDIR /app
|
|
|
|
|
COPY package*.json ./
|
|
|
|
|
RUN npm ci
|
|
|
|
|
COPY . .
|
2025-08-06 16:26:39 -07:00
|
|
|
RUN npm run download-models # Embed models for offline operation
|
2025-08-05 15:34:39 -07:00
|
|
|
RUN npm run build
|
|
|
|
|
|
2025-08-05 20:39:39 -07:00
|
|
|
FROM node:24-slim AS production
|
2025-08-05 15:34:39 -07:00
|
|
|
WORKDIR /app
|
|
|
|
|
COPY package*.json ./
|
2025-08-06 16:26:39 -07:00
|
|
|
RUN npm ci --only=production
|
2025-08-05 15:34:39 -07:00
|
|
|
COPY --from=builder /app/dist ./dist
|
2025-08-06 16:26:39 -07:00
|
|
|
COPY --from=builder /app/models ./models # Offline models included
|
2025-08-05 15:34:39 -07:00
|
|
|
CMD ["node", "dist/server.js"]
|
|
|
|
|
```
|
2025-08-04 08:56:05 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
Deploy to: Google Cloud Run, AWS Lambda/ECS, Azure Container Instances, Cloudflare Workers, Railway, Render, Vercel, anywhere Docker runs.
|
2025-08-04 08:56:05 -07:00
|
|
|
|
2025-08-06 12:29:32 -07:00
|
|
|
</details>
|
2025-08-04 08:56:05 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
## 📚 Documentation & Resources
|
2025-08-04 08:56:05 -07:00
|
|
|
|
2025-08-06 16:26:39 -07:00
|
|
|
- **[🚀 Quick Start Guide](docs/getting-started/)** - Get up and running in minutes
|
|
|
|
|
- **[📖 API Reference](docs/api-reference/)** - Complete method documentation
|
|
|
|
|
- **[💡 Examples](docs/examples/)** - Real-world usage patterns
|
|
|
|
|
- **[⚡ Performance Guide](docs/optimization-guides/)** - Scale to millions of vectors
|
|
|
|
|
- **[🔧 Storage Adapters](docs/api-reference/storage-adapters.md)** - Universal storage compatibility
|
2025-08-04 08:56:05 -07:00
|
|
|
|
2025-08-04 10:14:33 -07:00
|
|
|
## 🤝 Contributing
|
2025-08-04 08:56:05 -07:00
|
|
|
|
2025-08-04 10:14:33 -07:00
|
|
|
We welcome contributions! Please see:
|
2025-08-04 14:33:39 -07:00
|
|
|
|
2025-08-04 10:14:33 -07:00
|
|
|
- [Contributing Guidelines](CONTRIBUTING.md)
|
|
|
|
|
- [Developer Documentation](docs/development/DEVELOPERS.md)
|
|
|
|
|
- [Code of Conduct](CODE_OF_CONDUCT.md)
|
2025-08-04 08:56:05 -07:00
|
|
|
|
2025-08-04 10:14:33 -07:00
|
|
|
## 📄 License
|
2025-08-04 08:56:05 -07:00
|
|
|
|
2025-08-04 10:14:33 -07:00
|
|
|
[MIT](LICENSE)
|
2025-08-04 08:56:05 -07:00
|
|
|
|
2025-08-04 10:14:33 -07:00
|
|
|
---
|
2025-08-04 08:56:05 -07:00
|
|
|
|
2025-08-04 10:14:33 -07:00
|
|
|
<div align="center">
|
2025-08-06 16:26:39 -07:00
|
|
|
<strong>Ready to build the future of search? Get started with Brainy today!</strong>
|
|
|
|
|
|
|
|
|
|
**[Get Started →](docs/getting-started/) | [View Examples →](docs/examples/) | [Join Community →](https://github.com/soulcraft-research/brainy/discussions)**
|
|
|
|
|
</div>
|