feat: progressive init and readiness API for cloud storage
- Add `initMode` option to all cloud storage adapters (GCS, S3, Azure) - 'auto' (default): progressive in cloud, strict locally - 'progressive': <200ms cold starts, lazy bucket validation - 'strict': blocking validation (current behavior) - Add cloud environment auto-detection for: - Cloud Run (K_SERVICE, K_REVISION) - AWS Lambda (AWS_LAMBDA_FUNCTION_NAME) - Cloud Functions (FUNCTIONS_TARGET) - Azure Functions (AZURE_FUNCTIONS_ENVIRONMENT) - Add readiness API to Brainy class: - `brain.ready`: Promise that resolves when init() completes - `brain.isFullyInitialized()`: checks if background tasks done - `brain.awaitBackgroundInit()`: waits for background tasks - Lazy bucket validation on first write (not during init) - Background count synchronization - Update AWS/GCP deployment docs with readiness patterns 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
parent
19c5e67035
commit
d938a6bd4f
10 changed files with 1470 additions and 62 deletions
|
|
@ -386,8 +386,53 @@ jobs:
|
|||
```
|
||||
|
||||
3. **Cold Starts (Lambda)**
|
||||
|
||||
**v7.3.0+ Progressive Initialization (Zero-Config)**
|
||||
|
||||
Brainy automatically detects Lambda environments (AWS_LAMBDA_FUNCTION_NAME)
|
||||
and uses progressive initialization for <200ms cold starts:
|
||||
|
||||
```javascript
|
||||
// Zero-config - Brainy auto-detects Lambda
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 's3',
|
||||
s3Storage: {
|
||||
bucketName: 'my-bucket',
|
||||
region: 'us-east-1',
|
||||
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
|
||||
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
|
||||
}
|
||||
}
|
||||
})
|
||||
await brain.init() // Returns in <200ms
|
||||
|
||||
// First write validates bucket (lazy validation)
|
||||
await brain.add('noun', { name: 'test' }) // Validates here
|
||||
```
|
||||
|
||||
**Manual Override (if needed)**
|
||||
```javascript
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 's3',
|
||||
s3Storage: {
|
||||
bucketName: 'my-bucket',
|
||||
// Force specific mode
|
||||
initMode: 'progressive' // 'auto' | 'progressive' | 'strict'
|
||||
}
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
| Mode | Cold Start | Best For |
|
||||
|------|------------|----------|
|
||||
| `auto` (default) | <200ms in Lambda | Zero-config, auto-detects |
|
||||
| `progressive` | <200ms always | Force fast init everywhere |
|
||||
| `strict` | 100-500ms+ | Local dev, tests, debugging |
|
||||
|
||||
**Warm-up (Alternative)**
|
||||
```javascript
|
||||
// Warm-up configuration
|
||||
exports.warmup = async () => {
|
||||
if (!brain) {
|
||||
brain = new Brainy({ warmup: true })
|
||||
|
|
@ -396,6 +441,50 @@ jobs:
|
|||
}
|
||||
```
|
||||
|
||||
**Readiness Detection (v7.3.0+)**
|
||||
|
||||
Use the `brain.ready` Promise to ensure Brainy is initialized before handling requests:
|
||||
|
||||
```javascript
|
||||
let brain
|
||||
|
||||
exports.handler = async (event) => {
|
||||
if (!brain) {
|
||||
brain = new Brainy({ storage: { type: 's3', ... } })
|
||||
brain.init() // Fire and forget
|
||||
}
|
||||
|
||||
// Wait for initialization to complete
|
||||
await brain.ready
|
||||
|
||||
// Now safe to use brain methods
|
||||
const results = await brain.find({ query: event.queryStringParameters.q })
|
||||
return { statusCode: 200, body: JSON.stringify(results) }
|
||||
}
|
||||
```
|
||||
|
||||
For health checks, use `isFullyInitialized()` to verify all background tasks are complete:
|
||||
|
||||
```javascript
|
||||
exports.healthCheck = async () => {
|
||||
try {
|
||||
await brain.ready
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: JSON.stringify({
|
||||
status: 'ready',
|
||||
fullyInitialized: brain.isFullyInitialized()
|
||||
})
|
||||
}
|
||||
} catch (error) {
|
||||
return {
|
||||
statusCode: 503,
|
||||
body: JSON.stringify({ status: 'initializing' })
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Production Checklist
|
||||
|
||||
- [ ] IAM roles configured with minimal permissions
|
||||
|
|
|
|||
|
|
@ -506,6 +506,47 @@ const brain = new Brainy({
|
|||
```
|
||||
|
||||
2. **Cold Starts**
|
||||
|
||||
**v7.3.0+ Progressive Initialization (Zero-Config)**
|
||||
|
||||
Brainy automatically detects Cloud Run and Cloud Functions environments
|
||||
and uses progressive initialization for <200ms cold starts:
|
||||
|
||||
```javascript
|
||||
// Zero-config - Brainy auto-detects Cloud Run (K_SERVICE env var)
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 'gcs',
|
||||
gcsNativeStorage: { bucketName: 'my-bucket' }
|
||||
}
|
||||
})
|
||||
await brain.init() // Returns in <200ms
|
||||
|
||||
// First write validates bucket (lazy validation)
|
||||
await brain.add('noun', { name: 'test' }) // Validates here
|
||||
```
|
||||
|
||||
**Manual Override (if needed)**
|
||||
```javascript
|
||||
const brain = new Brainy({
|
||||
storage: {
|
||||
type: 'gcs',
|
||||
gcsNativeStorage: {
|
||||
bucketName: 'my-bucket',
|
||||
// Force specific mode
|
||||
initMode: 'progressive' // 'auto' | 'progressive' | 'strict'
|
||||
}
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
| Mode | Cold Start | Best For |
|
||||
|------|------------|----------|
|
||||
| `auto` (default) | <200ms in cloud | Zero-config, auto-detects |
|
||||
| `progressive` | <200ms always | Force fast init everywhere |
|
||||
| `strict` | 100-500ms+ | Local dev, tests, debugging |
|
||||
|
||||
**Keep Warm (Alternative)**
|
||||
```javascript
|
||||
// Keep minimum instances warm
|
||||
const brain = new Brainy({
|
||||
|
|
@ -516,6 +557,45 @@ const brain = new Brainy({
|
|||
})
|
||||
```
|
||||
|
||||
**Readiness Detection (v7.3.0+)**
|
||||
|
||||
Use the `brain.ready` Promise to ensure Brainy is initialized before handling requests:
|
||||
|
||||
```javascript
|
||||
let brain
|
||||
|
||||
async function handleRequest(req, res) {
|
||||
if (!brain) {
|
||||
brain = new Brainy({ storage: { type: 'gcs', ... } })
|
||||
brain.init() // Fire and forget
|
||||
}
|
||||
|
||||
// Wait for initialization to complete
|
||||
await brain.ready
|
||||
|
||||
// Now safe to use brain methods
|
||||
const results = await brain.find({ query: req.query.q })
|
||||
res.json(results)
|
||||
}
|
||||
```
|
||||
|
||||
For Cloud Run health checks, use `isFullyInitialized()` to verify all background tasks are complete:
|
||||
|
||||
```javascript
|
||||
// Health check endpoint for Cloud Run
|
||||
app.get('/health', async (req, res) => {
|
||||
try {
|
||||
await brain.ready
|
||||
res.json({
|
||||
status: 'ready',
|
||||
fullyInitialized: brain.isFullyInitialized()
|
||||
})
|
||||
} catch (error) {
|
||||
res.status(503).json({ status: 'initializing' })
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
3. **Memory Pressure**
|
||||
```javascript
|
||||
// Optimize for GCP memory constraints
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue