@geekmidas/仓库
一个全面,类型安全的存储客户端,用于云存储服务,支持多个供应商,以及版本和预签名的URL等高级功能.
特征
- 多供应商支助: AWS S3, 有谷歌云存储和Azure Blob存储的可扩展接口
- 类型安全: 包含全面类型定义的完整类型脚本支持
- 已指定 URL: 生成安全上传和下载 URL而不暴露证书
- 文件版本: 支持检索和管理文件版本
- 直接上传: 直接上传文件到没有中间服务器的存储
- 灵活配置: 支持自定义端点(用于 MinIO, LocalStack等)
- 现代合成/等待 API: 整个基于承诺的界面
- URL 缓存: 用于缓存预签名 URL 的内置支持以减少 API 调用并改进性能
安装
npm install @geekmidas/storage
同伴依赖关系
对于 AWS S3 支持,您需要安装 AWS SDK v3 包 :
npm install @aws-sdk/client-s3 @aws-sdk/s3-presigned-post @aws-sdk/s3-request-presigner
快速启动
AWS S3 导弹
import { AmazonStorageClient } from '@geekmidas/storage/aws';
import { InMemoryCache } from '@geekmidas/cache/memory';
// Create client with credentials
const storage = AmazonStorageClient.create({
bucket: 'my-bucket',
region: 'us-east-1',
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
});
// Create client with caching enabled
const cache = new InMemoryCache<string>();
const storageWithCache = AmazonStorageClient.create({
bucket: 'my-bucket',
region: 'us-east-1',
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
cache,
});
// Upload a file directly
await storage.upload('documents/readme.txt', 'Hello, World!', 'text/plain');
// Generate a download URL
const downloadUrl = await storage.getDownloadURL({
path: 'documents/readme.txt',
name: 'README.txt' // Optional: sets Content-Disposition header
});
// Generate a presigned upload URL
const uploadUrl = await storage.getUploadURL({
path: 'uploads/new-file.pdf',
contentType: 'application/pdf',
contentLength: 1024 * 1024, // 1MB
});
MinIO / 本地软件包
import { AmazonStorageClient } from '@geekmidas/storage/aws';
// For local development with MinIO
const storage = AmazonStorageClient.create({
bucket: 'test-bucket',
region: 'us-east-1',
accessKeyId: 'minioadmin',
secretAccessKey: 'minioadmin',
endpoint: 'http://localhost:9000',
});
API 参考
存储客户端接口
所有存储提供者执行的核心接口 :
interface StorageClient {
readonly provider: StorageProvider;
readonly cache?: Cache<string>;
// Direct upload
upload(key: string, data: string | Buffer, contentType: string): Promise<void>;
// Download URLs
getDownloadURL(file: File, expiresIn?: number): Promise<string>;
// Upload URLs
getUploadURL(params: GetUploadParams, expiresIn?: number): Promise<string>;
getUpload(params: GetUploadParams, expiresIn?: number): Promise<GetUploadResponse>;
// Versioning
getVersions(key: string): Promise<DocumentVersion[]>;
getVersionDownloadURL(file: File, versionId: string): Promise<string>;
}
亚马逊Storage客户端
工厂方法
AmazonStorageClient.create(options: AmazonStorageClientCreateOptions)
选项 :
bucket(需要):S3桶名region: AWS 区域( 默认: 使用 AWS SDK 默认)accessKeyId: AWS 访问密钥 IDsecretAccessKey: AWS 秘密访问密钥endpoint: 自定义 S3 端点(用于 MinIO, LocalStack)acl: Canned ACL 用于上传(默认:authenticated-read(中文(简体) )cache: 用于存储预签名的 URL 的可选缓存执行forcePathStyle: 强制路径样式 URL(用于 MinIO)
方法
upload(key: string, data: string | Buffer, contentType: string): Promise<void>
将数据直接上传到存储中 .
// Upload text
await storage.upload('documents/hello.txt', 'Hello, World!', 'text/plain');
// Upload binary data
const buffer = Buffer.from('binary data');
await storage.upload('files/binary.dat', buffer, 'application/octet-stream');
getDownloadURL(file: File, expiresIn?: number): Promise<string>
生成一个预先签名的下载 URL 。 当缓存被配置时,会根据文件路径缓存 URL.
// Simple download URL
const url = await storage.getDownloadURL({ path: 'documents/file.pdf' });
// With custom filename in Content-Disposition
const url = await storage.getDownloadURL({
path: 'documents/file.pdf',
name: 'My Document.pdf'
});
// Custom expiration (in seconds)
const url = await storage.getDownloadURL({ path: 'documents/file.pdf' }, 3600);
缓存行为 :
- URL以密钥格式缓存 :
download-url:{file.path} - 缓存 TTL 设置为
expiresIn - 60秒(带有1分缓冲) - 过期 < 60 秒的 URL 不缓存
- 缓存的 URL 立即返回, 不生成新的预签 URL
getUploadURL(params: GetUploadParams, expiresIn?: number): Promise<string>
生成一个 PUT 上传 URL .
const uploadUrl = await storage.getUploadURL({
path: 'uploads/new-file.pdf',
contentType: 'application/pdf',
contentLength: 1024 * 1024,
});
// Use the URL to upload
const response = await fetch(uploadUrl, {
method: 'PUT',
headers: {
'Content-Type': 'application/pdf',
'Content-Length': '1048576',
},
body: fileData,
});
getUpload(params: GetUploadParams, expiresIn?: number): Promise<GetUploadResponse>
生成一个带有窗体字段的预签名 POST 上传 .
const upload = await storage.getUpload({
path: 'uploads/form-upload.jpg',
contentType: 'image/jpeg',
contentLength: 500000,
});
// Use with HTML form
const formData = new FormData();
upload.fields.forEach(({ key, value }) => {
formData.append(key, value);
});
formData.append('file', fileInput.files[0]);
const response = await fetch(upload.url, {
method: 'POST',
body: formData,
});
getVersions(key: string): Promise<DocumentVersion[]>
获取文件的所有版本( 需要 S3 版本) .
const versions = await storage.getVersions('documents/versioned-file.txt');
console.log(versions); // [{ id: 'version-1', createdAt: Date }, ...]
getVersionDownloadURL(file: File, versionId: string): Promise<string>
生成特定版本的下载 URL .
const url = await storage.getVersionDownloadURL(
{ path: 'documents/file.txt' },
'version-12345'
);
类型
文件
interface File {
path: string;
name?: string; // Optional display name for Content-Disposition
}
获取上行程序
interface GetUploadParams {
path: string;
contentType: string;
contentLength: number;
}
文档版本
interface DocumentVersion {
id: string;
createdAt: Date;
}
存储器
enum StorageProvider {
AWSS3 = 'geekimdas.toolbox.storage.aws.s3',
GCP = 'geekimdas.toolbox.storage.gcp',
AZURE = 'geekimdas.toolbox.storage.azure',
}
高级使用
自定义 S3 客户端
import { S3Client } from '@aws-sdk/client-s3';
import { AmazonStorageClient, AmazonCannedAccessControlList } from '@geekmidas/storage/aws';
import { InMemoryCache } from '@geekmidas/cache/memory';
const s3Client = new S3Client({
region: 'us-east-1',
credentials: {
accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
},
});
// Without cache
const storage = new AmazonStorageClient(
s3Client,
'my-bucket',
AmazonCannedAccessControlList.PublicRead
);
// With cache
const cache = new InMemoryCache<string>();
const storageWithCache = new AmazonStorageClient(
s3Client,
'my-bucket',
AmazonCannedAccessControlList.PublicRead,
cache
);
访问控制列表
import { AmazonCannedAccessControlList } from '@geekmidas/storage/aws';
const storage = AmazonStorageClient.create({
bucket: 'my-bucket',
acl: AmazonCannedAccessControlList.PublicRead, // Files will be publicly readable
});
可用的控制列表 :
Private- 所有人得到全能控制, 没有其他人可以进入PublicRead- 所有人得到全控,其他人得到读取权限PublicReadWrite- 所有者得到全控,其他人得到读取/写取AuthenticatedRead- 所有者获得全控,认证用户获得阅读访问BucketOwnerRead- 对象拥有者获得全控,桶拥有者获得读取权限BucketOwnerFullControl- 物体和桶主得到全面的控制LogDeliveryWrite- 寄出日志服务获得写入权限AwsExecRead- Amazon EC2 读取 AMI 捆绑
缓存
缓存预签名的URL可以显著地减少API呼叫AWS S3的次数并改进性能.
import { AmazonStorageClient } from '@geekmidas/storage/aws';
import { InMemoryCache } from '@geekmidas/cache/memory';
import { UpstashCache } from '@geekmidas/cache/upstash';
// In-memory cache for development/testing
const memoryCache = new InMemoryCache<string>();
const storage = AmazonStorageClient.create({
bucket: 'my-bucket',
cache: memoryCache,
});
// Redis cache for production
const redisCache = new UpstashCache<string>({
url: process.env.UPSTASH_REDIS_URL,
token: process.env.UPSTASH_REDIS_TOKEN,
});
const productionStorage = AmazonStorageClient.create({
bucket: 'my-bucket',
cache: redisCache,
});
// Cache behavior example
const url1 = await storage.getDownloadURL({ path: 'file.pdf' }); // Generates new URL
const url2 = await storage.getDownloadURL({ path: 'file.pdf' }); // Returns cached URL
console.log(url1 === url2); // true
快取密钥和 TTL :
- 下载 URL 以密钥缓存 :
download-url:{path} - 缓存 TTL 自动计算为
expiresIn - 60秒数 - 60秒后到期的 URL 不缓存
- 上传 URL 不缓存( 通常是单用)
处理错误
try {
await storage.upload('documents/file.txt', 'content', 'text/plain');
} catch (error) {
if (error.name === 'NoSuchBucket') {
console.error('Bucket does not exist');
} else if (error.name === 'AccessDenied') {
console.error('Access denied');
} else {
console.error('Upload failed:', error);
}
}
发展
运行测试
# Run all tests
npm test
# Run unit tests only
npm run test:unit
# Run integration tests only (requires MinIO)
npm run test:integration
# Run tests once
npm run test:once
地方发展与MINIO
-
使用 Docker 作曲启动 MiNIO :
docker-compose up -d minio -
MinIO将在下列地点提供:
- API:http://localhost:9000 (英语)
- 控制台:http://localhost:9001
- 全权证书:米诺阿明/米诺阿明
-
运行集成测试 :
npm run test:integration
项目结构
src/
├── index.ts # Main exports
├── aws.ts # AWS-specific exports
├── StorageClient.ts # Core interfaces and types
├── AmazonStorageClient.ts # AWS S3 implementation
└── __tests__/
├── StorageClient.spec.ts # Interface tests
├── AmazonStorageClient.spec.ts # Unit tests
└── AmazonStorageClient.integration.spec.ts # Integration tests
捐款
- 遵循现有的代码样式(2个空格,单引号,分号)
- 添加新特性的综合测试
- 更新 API 更改的文档
- 使用“ 整合比单位” 测试哲学 - 倾向于真正的依赖性而不是模拟
许可证
麻省理工学院执照 - 详情见LICENSE文件.