docs: clarify GCS storage type and config object pairing
Improve documentation to explicitly show that: - type: 'gcs' requires gcsStorage config (S3-compatible, HMAC) - type: 'gcs-native' requires gcsNativeStorage config (native SDK, ADC) - Mixing type and config objects will fall back to memory storage - Auto-detection works when type is omitted Added 'Common Mistakes' section with examples of incorrect usage. Enhanced migration guide with step-by-step instructions.
This commit is contained in:
parent
17898104e0
commit
dcbd0fd136
1 changed files with 95 additions and 29 deletions
|
|
@ -609,24 +609,6 @@ const brain = new Brainy({
|
||||||
})
|
})
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Google Cloud Storage (S3-Compatible)
|
|
||||||
GCS using HMAC keys for S3-compatible access.
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
const brain = new Brainy({
|
|
||||||
storage: {
|
|
||||||
type: 'gcs',
|
|
||||||
gcsStorage: {
|
|
||||||
bucketName: 'my-bucket',
|
|
||||||
region: 'us-central1',
|
|
||||||
accessKeyId: process.env.GCS_ACCESS_KEY_ID,
|
|
||||||
secretAccessKey: process.env.GCS_SECRET_ACCESS_KEY,
|
|
||||||
endpoint: 'https://storage.googleapis.com'
|
|
||||||
}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Google Cloud Storage (Native SDK) 🆕
|
#### Google Cloud Storage (Native SDK) 🆕
|
||||||
**Recommended for GCS deployments.** Uses native `@google-cloud/storage` SDK with automatic authentication.
|
**Recommended for GCS deployments.** Uses native `@google-cloud/storage` SDK with automatic authentication.
|
||||||
|
|
||||||
|
|
@ -636,11 +618,13 @@ const brain = new Brainy({
|
||||||
- ✅ No HMAC key management required
|
- ✅ No HMAC key management required
|
||||||
- ✅ Automatic service account integration
|
- ✅ Automatic service account integration
|
||||||
|
|
||||||
**With Application Default Credentials (Cloud Run/GCE):**
|
**Option 1: Explicit Type (Recommended)**
|
||||||
|
|
||||||
|
With Application Default Credentials (Cloud Run/GCE):
|
||||||
```typescript
|
```typescript
|
||||||
const brain = new Brainy({
|
const brain = new Brainy({
|
||||||
storage: {
|
storage: {
|
||||||
type: 'gcs-native',
|
type: 'gcs-native', // ⚠️ Must be 'gcs-native' for native SDK
|
||||||
gcsNativeStorage: {
|
gcsNativeStorage: {
|
||||||
bucketName: 'my-bucket'
|
bucketName: 'my-bucket'
|
||||||
// No credentials needed - ADC automatic!
|
// No credentials needed - ADC automatic!
|
||||||
|
|
@ -649,7 +633,7 @@ const brain = new Brainy({
|
||||||
})
|
})
|
||||||
```
|
```
|
||||||
|
|
||||||
**With Service Account Key File:**
|
With Service Account Key File:
|
||||||
```typescript
|
```typescript
|
||||||
const brain = new Brainy({
|
const brain = new Brainy({
|
||||||
storage: {
|
storage: {
|
||||||
|
|
@ -662,7 +646,7 @@ const brain = new Brainy({
|
||||||
})
|
})
|
||||||
```
|
```
|
||||||
|
|
||||||
**With Service Account Credentials Object:**
|
With Service Account Credentials Object:
|
||||||
```typescript
|
```typescript
|
||||||
const brain = new Brainy({
|
const brain = new Brainy({
|
||||||
storage: {
|
storage: {
|
||||||
|
|
@ -678,6 +662,64 @@ const brain = new Brainy({
|
||||||
})
|
})
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**Option 2: Auto-Detection**
|
||||||
|
|
||||||
|
You can omit the `type` field and let Brainy auto-detect based on the config object:
|
||||||
|
```typescript
|
||||||
|
const brain = new Brainy({
|
||||||
|
storage: {
|
||||||
|
gcsNativeStorage: {
|
||||||
|
bucketName: 'my-bucket'
|
||||||
|
// type defaults to 'auto', will use native SDK
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Google Cloud Storage (S3-Compatible) - Legacy
|
||||||
|
GCS using HMAC keys for S3-compatible access. **Consider migrating to 'gcs-native' for better performance.**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
const brain = new Brainy({
|
||||||
|
storage: {
|
||||||
|
type: 'gcs', // ⚠️ Must be 'gcs' for S3-compatible mode
|
||||||
|
gcsStorage: { // ⚠️ Must use 'gcsStorage' (not 'gcsNativeStorage')
|
||||||
|
bucketName: 'my-bucket',
|
||||||
|
region: 'us-central1',
|
||||||
|
accessKeyId: process.env.GCS_ACCESS_KEY_ID,
|
||||||
|
secretAccessKey: process.env.GCS_SECRET_ACCESS_KEY,
|
||||||
|
endpoint: 'https://storage.googleapis.com'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
**⚠️ Common Mistakes:**
|
||||||
|
```typescript
|
||||||
|
// ❌ WRONG - type/config mismatch (will fall back to memory storage)
|
||||||
|
{
|
||||||
|
type: 'gcs',
|
||||||
|
gcsNativeStorage: { bucketName: 'my-bucket' }
|
||||||
|
}
|
||||||
|
|
||||||
|
// ❌ WRONG - type/config mismatch (will fall back to memory storage)
|
||||||
|
{
|
||||||
|
type: 'gcs-native',
|
||||||
|
gcsStorage: { bucketName: 'my-bucket', accessKeyId: '...', secretAccessKey: '...' }
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ CORRECT - type matches config object
|
||||||
|
{
|
||||||
|
type: 'gcs-native',
|
||||||
|
gcsNativeStorage: { bucketName: 'my-bucket' }
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ CORRECT - auto-detection
|
||||||
|
{
|
||||||
|
gcsNativeStorage: { bucketName: 'my-bucket' }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
### Storage Features
|
### Storage Features
|
||||||
|
|
||||||
All storage adapters support:
|
All storage adapters support:
|
||||||
|
|
@ -692,12 +734,12 @@ All storage adapters support:
|
||||||
|
|
||||||
If you're currently using `type: 'gcs'` with HMAC keys, migrating to `type: 'gcs-native'` is straightforward:
|
If you're currently using `type: 'gcs'` with HMAC keys, migrating to `type: 'gcs-native'` is straightforward:
|
||||||
|
|
||||||
**Before (HMAC):**
|
**Before (S3-Compatible with HMAC):**
|
||||||
```typescript
|
```typescript
|
||||||
const brain = new Brainy({
|
const brain = new Brainy({
|
||||||
storage: {
|
storage: {
|
||||||
type: 'gcs',
|
type: 'gcs', // ⚠️ Old: S3-compatible mode
|
||||||
gcsStorage: {
|
gcsStorage: { // ⚠️ Old: HMAC credentials
|
||||||
bucketName: 'my-bucket',
|
bucketName: 'my-bucket',
|
||||||
accessKeyId: process.env.GCS_ACCESS_KEY_ID,
|
accessKeyId: process.env.GCS_ACCESS_KEY_ID,
|
||||||
secretAccessKey: process.env.GCS_SECRET_ACCESS_KEY
|
secretAccessKey: process.env.GCS_SECRET_ACCESS_KEY
|
||||||
|
|
@ -706,12 +748,12 @@ const brain = new Brainy({
|
||||||
})
|
})
|
||||||
```
|
```
|
||||||
|
|
||||||
**After (Native with ADC):**
|
**After (Native SDK with ADC):**
|
||||||
```typescript
|
```typescript
|
||||||
const brain = new Brainy({
|
const brain = new Brainy({
|
||||||
storage: {
|
storage: {
|
||||||
type: 'gcs-native',
|
type: 'gcs-native', // ✅ New: Native SDK mode
|
||||||
gcsNativeStorage: {
|
gcsNativeStorage: { // ✅ New: ADC authentication
|
||||||
bucketName: 'my-bucket'
|
bucketName: 'my-bucket'
|
||||||
// ADC handles authentication automatically
|
// ADC handles authentication automatically
|
||||||
}
|
}
|
||||||
|
|
@ -719,12 +761,36 @@ const brain = new Brainy({
|
||||||
})
|
})
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**⚠️ Important Migration Notes:**
|
||||||
|
|
||||||
|
1. **Change BOTH the type AND the config object:**
|
||||||
|
- `type: 'gcs'` → `type: 'gcs-native'`
|
||||||
|
- `gcsStorage` → `gcsNativeStorage`
|
||||||
|
|
||||||
|
2. **Remove HMAC keys** - Not needed with ADC:
|
||||||
|
- Remove `accessKeyId`
|
||||||
|
- Remove `secretAccessKey`
|
||||||
|
- Remove `region` (optional with native SDK)
|
||||||
|
|
||||||
|
3. **Set up ADC in your environment:**
|
||||||
|
```bash
|
||||||
|
# Cloud Run/GCE: Nothing needed, ADC is automatic
|
||||||
|
|
||||||
|
# Local development:
|
||||||
|
gcloud auth application-default login
|
||||||
|
|
||||||
|
# Or set GOOGLE_APPLICATION_CREDENTIALS:
|
||||||
|
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json"
|
||||||
|
```
|
||||||
|
|
||||||
**Data Migration:**
|
**Data Migration:**
|
||||||
No data migration required! Both adapters use the same path structure:
|
✅ **No data migration required!** Both adapters use the same path structure:
|
||||||
- `entities/nouns/vectors/{shard}/{id}.json`
|
- `entities/nouns/vectors/{shard}/{id}.json`
|
||||||
- `entities/nouns/metadata/{shard}/{id}.json`
|
- `entities/nouns/metadata/{shard}/{id}.json`
|
||||||
- `entities/verbs/vectors/{shard}/{id}.json`
|
- `entities/verbs/vectors/{shard}/{id}.json`
|
||||||
|
|
||||||
|
Simply change your code and restart your application. Existing data will work immediately.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Configuration API
|
## Configuration API
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue