soulcraft/brainy
2
0
Fork 0
Code Issues 1 Pull requests Projects Releases 335 Packages Wiki Activity Actions

refactor(8.0)!: remove orphaned zero-config subsystem + dead cloud/progressive-init storage vestige

Browse source 
The old config-generation subsystem (src/config/ + autoConfiguration.ts) was
superseded during the 8.0 rework and never wired into init(): it emitted settings
for a partitioning subsystem that no longer exists and probed deleted cloud env
vars. The live zero-config path is inline — recall preset → HNSW knobs, storage
auto-detect, auto persistMode, container-memory-aware cache sizing.

The storage progressive-init / cloud-detection cluster was equally dead after the
cloud adapters were dropped: isCloudStorage() is permanently false (no overriders),
scheduleBackgroundInit/runBackgroundInit were never called (the latter an empty
body), initMode was never assigned, and Brainy.isFullyInitialized()/
awaitBackgroundInit() were always-trivial with zero callers. scheduleCountPersist()
collapses to its only-ever-taken immediate write-through path.

Removed:
- src/config/{index,zeroConfig,storageAutoConfig,modelAutoConfig,sharedConfigManager}.ts
- src/utils/autoConfiguration.ts + the inert BrainyZeroConfig export
- Brainy.isFullyInitialized()/awaitBackgroundInit() (+ BrainyInterface decls)
- InitMode type, isCloudStorage/detectCloudEnvironment/resolveInitMode,
  scheduleBackgroundInit/runBackgroundInit/ensureValidatedForWrite and their state
- Dead cloud env-var probes (K_SERVICE/K_REVISION/AWS_LAMBDA_FUNCTION_NAME/
  FUNCTIONS_TARGET/AZURE_FUNCTIONS_ENVIRONMENT)

Kept (verified live): production-detection logging (environment.ts), container-
memory cache sizing (memoryDetection/paramValidation), on-disk hash bucketing
(sharding.ts).

Docs: scrubbed deleted-subsystem references (JS quantization knobs, cloud/OPFS
adapters, partitioning, old zero-config API) across 14 files; deleted two wholly-
obsolete feature docs (complete-feature-list, v3-features); rewrote
architecture/zero-config for 8.0.

~3,700 LOC removed. Build clean; 1392 unit + 24 db-mvcc green.
This commit is contained in:
David Snelling 2026-06-15 11:11:21 -07:00
parent 00d3203d68
commit 35b9d7ef43
28 changed files with 596 additions and 3752 deletions
Download patch file Download diff file Expand all files Collapse all files

539
docs/guides/framework-integration.md
View file

@ -1,15 +1,17 @@
# Framework Integration Guide
Brainy 3.0 is **framework-first** - designed from the ground up to work seamlessly with modern JavaScript frameworks. This guide shows you how to integrate Brainy into any framework.
Brainy is **framework-friendly** - designed to drop into the server side of any modern JavaScript framework. This guide shows you how to integrate Brainy into framework-based apps.
## 🎯 Why Framework-First?
> **Runtime**: Brainy 8.0 runs on Node.js 22+ and Bun (server-side only). It is not a browser library. In framework apps, run Brainy in API routes, server components, server actions, loaders, or a dedicated backend service - never in client-side bundles.
Traditional AI databases require complex browser polyfills and bundler configurations. Brainy 3.0 trusts your framework to handle this:
## 🎯 Why Server-Side?
Brainy embeds an HNSW vector index, a graph engine, and a filesystem-backed persistence layer. These belong on the server:
- **Zero configuration**: Just `import { Brainy } from '@soulcraft/brainy'`
- **Framework responsibility**: Let Next.js, Vite, Webpack handle Node.js polyfills
- **Cleaner code**: No browser-specific entry points or conditional imports
- **Better DX**: Same API everywhere - browser, server, edge
- **Auto storage detection**: `new Brainy()` auto-selects filesystem persistence on Node
- **Cleaner code**: No browser polyfills, no conditional client/server imports
- **Better DX**: One instance shared across your server routes
## 🚀 Quick Start
@ -24,7 +26,8 @@ npm install @soulcraft/brainy
```javascript
import { Brainy } from '@soulcraft/brainy'
// Works in any framework!
// Run on the server (API route, server component, backend service)
// new Brainy() auto-detects filesystem persistence on Node
const brain = new Brainy()
await brain.init()
@ -41,52 +44,48 @@ const results = await brain.find("framework integration")
## ⚛️ React Integration
Brainy runs on the server, so a React client component talks to it through an API endpoint (see the Next.js API route below). The hook fetches results; it never instantiates Brainy in the browser.
### Basic Hook Pattern
```jsx
import { useState, useEffect, useCallback } from 'react'
import { Brainy } from '@soulcraft/brainy'
import { useState, useCallback } from 'react'
function useBrainy() {
const [brain, setBrain] = useState(null)
const [isReady, setIsReady] = useState(false)
function useBrainySearch(endpoint = '/api/search') {
const [results, setResults] = useState([])
const [loading, setLoading] = useState(false)
useEffect(() => {
const initBrain = async () => {
const newBrain = new Brainy({
storage: { type: 'opfs' } // Browser storage
const search = useCallback(async (query) => {
if (!query) return
setLoading(true)
try {
const res = await fetch(endpoint, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ query })
})
await newBrain.init()
setBrain(newBrain)
setIsReady(true)
const { results } = await res.json()
setResults(results)
} finally {
setLoading(false)
}
}, [endpoint])
initBrain()
}, [])
return { brain, isReady }
return { results, loading, search }
}
// Usage in component
function SearchComponent() {
const { brain, isReady } = useBrainy()
const [results, setResults] = useState([])
const handleSearch = useCallback(async (query) => {
if (!isReady) return
const searchResults = await brain.find(query)
setResults(searchResults)
}, [brain, isReady])
if (!isReady) return <div>Loading AI...</div>
const { results, loading, search } = useBrainySearch()
return (
<div>
<input
type="text"
placeholder="Search..."
onChange={(e) => handleSearch(e.target.value)}
onChange={(e) => search(e.target.value)}
/>
{loading && <div>Searching...</div>}
<div>
{results.map(result => (
<div key={result.id}>
@ -100,48 +99,34 @@ function SearchComponent() {
}
```
### React Context Pattern
### Shared Server Instance
```jsx
import React, { createContext, useContext, useEffect, useState } from 'react'
On the server, create one Brainy instance and reuse it across requests. This module is imported only by server code (API routes, server components), never by client components:
```javascript
// lib/brain.server.js
import { Brainy } from '@soulcraft/brainy'
const BrainyContext = createContext()
let brainPromise
export function BrainyProvider({ children }) {
const [brain, setBrain] = useState(null)
const [isReady, setIsReady] = useState(false)
useEffect(() => {
const initBrain = async () => {
const newBrain = new Brainy()
await newBrain.init()
setBrain(newBrain)
setIsReady(true)
}
initBrain()
}, [])
return (
<BrainyContext.Provider value={{ brain, isReady }}>
{children}
</BrainyContext.Provider>
)
}
export function useBrainContext() {
const context = useContext(BrainyContext)
if (!context) {
throw new Error('useBrainContext must be used within BrainyProvider')
export function getBrain() {
if (!brainPromise) {
brainPromise = (async () => {
// new Brainy() auto-detects filesystem persistence on Node
const brain = new Brainy()
await brain.init()
return brain
})()
}
return context
return brainPromise
}
```
## 🟢 Vue.js Integration
### Composition API
Vue components call a server endpoint (see the Nuxt server route in the [Vue.js Integration Guide](vue-integration.md)); Brainy itself runs on the server.
### Composition API (client component)
```vue
<template>
@ -155,100 +140,70 @@ export function useBrainContext() {
</template>
<script setup>
import { ref, onMounted } from 'vue'
import { Brainy } from '@soulcraft/brainy'
import { ref } from 'vue'
const brain = ref(null)
const isReady = ref(false)
const query = ref('')
const results = ref([])
onMounted(async () => {
brain.value = new Brainy({
storage: { type: 'opfs' }
})
await brain.value.init()
isReady.value = true
})
const search = async () => {
if (!isReady.value || !query.value) return
results.value = await brain.value.find(query.value)
if (!query.value) return
const res = await fetch('/api/search', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ query: query.value })
})
results.value = (await res.json()).results
}
</script>
```
### Vue 3 Plugin
### Shared Server Instance
On the server, create one Brainy instance and reuse it across requests:
```javascript
// plugins/brainy.js
// server/brain.js (server-only module)
import { Brainy } from '@soulcraft/brainy'
export default {
install(app, options) {
const brain = new Brainy(options)
let brainPromise
app.config.globalProperties.$brain = brain
app.provide('brain', brain)
// Initialize on app mount
brain.init()
export function getBrain() {
if (!brainPromise) {
brainPromise = (async () => {
// new Brainy() auto-detects filesystem persistence on Node
const brain = new Brainy()
await brain.init()
return brain
})()
}
return brainPromise
}
// main.js
import { createApp } from 'vue'
import BrainyPlugin from './plugins/brainy'
const app = createApp(App)
app.use(BrainyPlugin, {
storage: { type: 'opfs' }
})
```
## 🅰️ Angular Integration
### Service Pattern
The Angular service calls your backend over HTTP; Brainy lives in that backend, not in the browser.
### Service Pattern (calls the backend)
```typescript
// brainy.service.ts
import { Injectable } from '@angular/core'
import { BehaviorSubject, Observable } from 'rxjs'
import { Brainy } from '@soulcraft/brainy'
import { HttpClient } from '@angular/common/http'
import { Observable } from 'rxjs'
@Injectable({
providedIn: 'root'
})
export class BrainyService {
private brain: Brainy
private readySubject = new BehaviorSubject<boolean>(false)
constructor(private http: HttpClient) {}
ready$: Observable<boolean> = this.readySubject.asObservable()
constructor() {
this.initBrain()
search(query: string): Observable<{ results: any[] }> {
return this.http.post<{ results: any[] }>('/api/search', { query })
}
private async initBrain() {
this.brain = new Brainy({
storage: { type: 'opfs' }
})
await this.brain.init()
this.readySubject.next(true)
}
async search(query: string): Promise<any[]> {
if (!this.readySubject.value) {
throw new Error('Brain not ready')
}
return await this.brain.find(query)
}
async add(data: any, type: string, metadata?: any): Promise<string> {
if (!this.readySubject.value) {
throw new Error('Brain not ready')
}
return await this.brain.add({ data, type, metadata })
add(data: any, type: string, metadata?: any): Observable<{ id: string }> {
return this.http.post<{ id: string }>('/api/add', { data, type, metadata })
}
}
```
@ -280,64 +235,52 @@ export class SearchComponent {
constructor(private brainyService: BrainyService) {}
async search() {
search() {
if (!this.query) return
this.results = await this.brainyService.search(this.query)
this.brainyService.search(this.query).subscribe(({ results }) => {
this.results = results
})
}
}
```
The matching backend endpoint uses Brainy directly (Node/Bun):
```typescript
// server: api/search
import { Brainy } from '@soulcraft/brainy'
const brain = new Brainy() // auto-detects filesystem persistence on Node
await brain.init()
export async function handleSearch(query: string) {
return await brain.find(query)
}
```
## 🚀 Next.js Integration
### App Router (Next.js 13+)
In Next.js, Brainy lives in server code only: API routes, server components, or server actions. Create one shared instance in a server-only module.
```jsx
// app/providers.jsx
'use client'
import { createContext, useContext, useEffect, useState } from 'react'
### Shared Server Instance
```javascript
// lib/brain.server.js (imported only by server code)
import { Brainy } from '@soulcraft/brainy'
const BrainyContext = createContext()
let brainPromise
export function BrainyProvider({ children }) {
const [brain, setBrain] = useState(null)
const [isReady, setIsReady] = useState(false)
useEffect(() => {
const initBrain = async () => {
const newBrain = new Brainy()
await newBrain.init()
setBrain(newBrain)
setIsReady(true)
}
initBrain()
}, [])
return (
<BrainyContext.Provider value={{ brain, isReady }}>
{children}
</BrainyContext.Provider>
)
}
export const useBrainy = () => useContext(BrainyContext)
```
```jsx
// app/layout.jsx
import { BrainyProvider } from './providers'
export default function RootLayout({ children }) {
return (
<html>
<body>
<BrainyProvider>
{children}
</BrainyProvider>
</body>
</html>
)
export function getBrain() {
if (!brainPromise) {
brainPromise = (async () => {
const brain = new Brainy({
storage: { type: 'filesystem', rootDirectory: './data' }
})
await brain.init()
return brain
})()
}
return brainPromise
}
```
@ -345,45 +288,78 @@ export default function RootLayout({ children }) {
```javascript
// app/api/search/route.js
import { Brainy } from '@soulcraft/brainy'
const brain = new Brainy({
storage: { type: 'filesystem', rootDirectory: './data' }
})
await brain.init()
import { getBrain } from '@/lib/brain.server'
export async function POST(request) {
const { query } = await request.json()
const brain = await getBrain()
const results = await brain.find(query)
return Response.json({ results })
}
```
## 🔷 Svelte Integration
### Server Action
```javascript
// app/actions.js
'use server'
import { getBrain } from '@/lib/brain.server'
export async function search(query) {
const brain = await getBrain()
return await brain.find(query)
}
```
## 🔷 SvelteKit Integration
Brainy runs in a server-only module (`*.server.js`); the component fetches results from an endpoint.
```javascript
// src/lib/server/brain.js (server-only — note the .server suffix)
import { Brainy } from '@soulcraft/brainy'
let brainPromise
export function getBrain() {
if (!brainPromise) {
brainPromise = (async () => {
const brain = new Brainy() // auto-detects filesystem persistence
await brain.init()
return brain
})()
}
return brainPromise
}
```
```javascript
// src/routes/api/search/+server.js
import { json } from '@sveltejs/kit'
import { getBrain } from '$lib/server/brain'
export async function POST({ request }) {
const { query } = await request.json()
const brain = await getBrain()
return json({ results: await brain.find(query) })
}
```
```svelte
<!-- SearchComponent.svelte -->
<script>
import { onMount } from 'svelte'
import { Brainy } from '@soulcraft/brainy'
let brain = null
let isReady = false
let query = ''
let results = []
onMount(async () => {
brain = new Brainy({
storage: { type: 'opfs' }
})
await brain.init()
isReady = true
})
async function search() {
if (!isReady || !query) return
results = await brain.find(query)
if (!query) return
const res = await fetch('/api/search', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ query })
})
results = (await res.json()).results
}
</script>
@ -401,27 +377,23 @@ export async function POST(request) {
## 🌟 Solid.js Integration
The component calls a server endpoint (use SolidStart server routes, or any backend, to host Brainy):
```jsx
import { createSignal, onMount } from 'solid-js'
import { Brainy } from '@soulcraft/brainy'
import { createSignal } from 'solid-js'
function SearchComponent() {
const [brain, setBrain] = createSignal(null)
const [isReady, setIsReady] = createSignal(false)
const [query, setQuery] = createSignal('')
const [results, setResults] = createSignal([])
onMount(async () => {
const newBrain = new Brainy()
await newBrain.init()
setBrain(newBrain)
setIsReady(true)
})
const search = async () => {
if (!isReady() || !query()) return
const searchResults = await brain().find(query())
setResults(searchResults)
if (!query()) return
const res = await fetch('/api/search', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ query: query() })
})
setResults((await res.json()).results)
}
return (
@ -450,57 +422,25 @@ function SearchComponent() {
## 📦 Bundler Configuration
### Vite (Recommended)
Brainy is a server-side dependency, so keep it out of client bundles. Import it only from server-only modules (`*.server.js`, API routes, server components, server actions). If your bundler ever tries to pull Brainy into a client bundle, that's a sign it's being imported from a client component — move the import to a server module.
For server builds, mark Brainy as external so the bundler doesn't inline it:
```javascript
// vite.config.js
// vite.config.js (SSR build)
import { defineConfig } from 'vite'
export default defineConfig({
define: {
global: 'globalThis'
},
optimizeDeps: {
include: ['@soulcraft/brainy']
ssr: {
external: ['@soulcraft/brainy']
}
})
```
### Webpack
```javascript
// webpack.config.js
module.exports = {
resolve: {
fallback: {
"fs": false,
"path": require.resolve("path-browserify"),
"crypto": require.resolve("crypto-browserify")
}
},
plugins: [
new webpack.ProvidePlugin({
global: 'global'
})
]
}
```
### Rollup
```javascript
// rollup.config.js
import { nodeResolve } from '@rollup/plugin-node-resolve'
import commonjs from '@rollup/plugin-commonjs'
// rollup.config.js (server bundle)
export default {
plugins: [
nodeResolve({
browser: true,
preferBuiltins: false
}),
commonjs()
]
external: ['@soulcraft/brainy', 'node:fs', 'node:path', 'node:crypto']
}
```
@ -508,30 +448,24 @@ export default {
### Server-Side Rendering
Instantiate Brainy on the server and feed its results into the rendered page. Never construct it in client code.
```javascript
// Check if running in browser
if (typeof window !== 'undefined') {
// Browser-only code
const brain = new Brainy({
storage: { type: 'opfs' }
})
}
// Server-side data loading (framework loader / getServerSideProps / load fn)
import { getBrain } from './brain.server'
// Or use dynamic imports
const initBrainForBrowser = async () => {
if (typeof window === 'undefined') return null
const { Brainy } = await import('@soulcraft/brainy')
const brain = new Brainy()
await brain.init()
return brain
export async function load({ url }) {
const brain = await getBrain()
const query = url.searchParams.get('q') ?? ''
const results = query ? await brain.find(query) : []
return { results }
}
```
### Static Site Generation
```javascript
// For build-time usage
// For build-time usage (runs in Node during the build)
import { Brainy } from '@soulcraft/brainy'
export async function generateStaticProps() {
@ -552,67 +486,46 @@ export async function generateStaticProps() {
## 🔧 Framework-Specific Tips
### React
- Use `useCallback` for search functions to prevent re-renders
- Consider `useMemo` for expensive brain operations
- Implement cleanup in `useEffect` for proper memory management
- Keep components client-side and call a Brainy-backed API route
- Use `useCallback` for fetch handlers to prevent re-renders
- Debounce keystroke-driven searches before hitting the endpoint
### Vue
- Use `shallowRef` for the brain instance (it's not reactive data)
- Consider Pinia for global brain state management
- Use `watchEffect` for reactive search queries
- Components call an endpoint; the shared instance lives in a server module
- Consider Pinia for caching results client-side
- Debounce reactive search queries
### Angular
- Implement proper dependency injection with services
- Use RxJS observables for reactive search
- Consider lazy loading brain in feature modules
- Use `HttpClient` and RxJS to call the backend
- Hold the shared Brainy instance in your Node backend, not the app
- Consider lazy loading search features in feature modules
### Next.js
- Use dynamic imports for client-side only features
- Consider API routes for server-side brain operations
- Implement proper error boundaries
- Put Brainy in server-only modules (`*.server.js`), API routes, or server actions
- Reuse one shared instance across requests
- Implement proper error boundaries for failed fetches
## 🚨 Common Issues & Solutions
### Issue: "crypto is not defined"
**Solution**: Your framework should handle this automatically. If not:
```javascript
// Add to your bundle config
define: {
global: 'globalThis'
}
```
### Issue: "fs module not found" / "crypto is not defined" in the browser
**Cause**: Brainy was imported into a client bundle. Brainy 8.0 is a server-side library (Node 22+/Bun) and uses Node built-ins like `fs` and `crypto`.
**Solution**: Import Brainy only from server code — server-only modules (`*.server.js`), API routes, server components, or server actions. From client components, call those endpoints instead.
### Issue: "fs module not found"
**Solution**: This is expected in browsers. Use browser-compatible storage:
```javascript
const brain = new Brainy({
storage: { type: 'opfs' } // Or 'memory' for development
})
```
### Issue: Large bundle size
**Solution**: Use dynamic imports for optional features:
```javascript
const brain = await import('@soulcraft/brainy').then(m => new m.Brainy())
```
### Issue: Large client bundle size
**Cause**: A client module is pulling in Brainy.
**Solution**: Move the `import { Brainy } from '@soulcraft/brainy'` into a server-only module so it never reaches the browser bundle.
### Issue: SSR hydration mismatch
**Solution**: Initialize brain only on client:
```javascript
useEffect(() => {
// Browser-only initialization
initBrain()
}, [])
```
**Solution**: Run the search on the server (loader / server action / API route) and pass the results down as props, so server and client render the same markup.
## 🎯 Best Practices
1. **Initialize Once**: Create brain instance at app level, not component level
2. **Use Context**: Share brain instance across components with context/providers
3. **Handle Loading**: Always show loading states during brain initialization
4. **Error Boundaries**: Implement proper error handling for brain operations
5. **Memory Management**: Clean up brain instances on unmount
6. **Storage Strategy**: Choose appropriate storage for your deployment target
1. **Initialize Once**: Create one shared Brainy instance per server process, not per request
2. **Server-Only**: Import Brainy only from server modules — never from client components
3. **Endpoint Boundary**: Expose search/add through API routes or server actions
4. **Handle Loading**: Show loading states in the client while the fetch is in flight
5. **Error Handling**: Catch and surface failed endpoint calls gracefully
6. **Storage**: Use `filesystem` for persistence (the default on Node) or `memory` for ephemeral/tests
## 📚 Next Steps