Minioに関する処理を追加

This commit is contained in:
ry.yamafuji 2025-03-22 12:49:06 +09:00
parent 3a1b88493f
commit 2454db7d13
17 changed files with 1237 additions and 2 deletions

199
docs/minio.md Normal file
View File

@ -0,0 +1,199 @@
# MinIO 利用ガイド
## 概要
[MinIO](https://min.io/) は Amazon S3 互換のオブジェクトストレージサーバー。軽量かつシンプルな構成で、オンプレミスやクラウド上に S3 互換のストレージ環境を構築できる。
高可用構成やセキュリティ機能も備えており、小規模から大規模用途まで幅広く利用できる。本記事では MinIO の導入方法、仕様要件、ユースケース、基本的な利用方法についてまとめる。
---
## 1. インストール方法
### 1.1 Docker を使う場合
```bash
docker run -p 9000:9000 -p 9001:9001 \
--name minio \
-e "MINIO_ROOT_USER=admin" \
-e "MINIO_ROOT_PASSWORD=password123" \
-v /mnt/data:/data \
quay.io/minio/minio server /data --console-address ":9001"
```
- `9000`: S3互換API用ポート
- `9001`: Webコンソール用ポート
![mino-top.png](/mino-top.png)
---
### 1.2 Docker Compose を使う場合
```yaml
version: '3.8'
services:
minio:
image: quay.io/minio/minio
ports:
- "9000:9000"
- "9001:9001"
volumes:
- ./data:/data
environment:
MINIO_ROOT_USER: admin
MINIO_ROOT_PASSWORD: password123
command: server /data --console-address ":9001"
restart: unless-stopped
```
起動コマンド:
```bash
docker compose up -d
```
---
### 1.3 バイナリから直接実行する場合Linux
```bash
wget https://dl.min.io/server/minio/release/linux-amd64/minio
chmod +x minio
sudo mv minio /usr/local/bin/
export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=password123
minio server /mnt/data --console-address ":9001"
```
---
## 2. 仕様要件
| 項目 | 推奨構成例 |
|--------------|------------------------------|
| OS | LinuxUbuntu 22.04 など) |
| CPU | 1コア以上最低限 |
| メモリ | 1GB 以上2GB 以上推奨) |
| ストレージ | SSD または HDD |
| 通信 | ポート 9000 / 9001開放 |
- HTTPS を使用する場合は TLS 証明書Let's Encrypt など)を導入
- リバースプロキシNginx/Caddyなど経由での公開も可能
---
## 3. ユースケース・機能
### 主なユースケース
- バックアップ保存
- Webアプリの画像・動画ストレージ
- ログデータ保管
- オンプレS3互換ストレージ
### 主な機能
- S3 互換API対応
- Webベース管理コンソール
- IAM風のユーザー・ポリシー制御
- TLS対応HTTPS通信
- サーバーサイド暗号化AES-256
- マルチテナント対応(認証連携含む)
- オブジェクトのバージョン管理(設定時)
---
## 4. 実際の使い方
サンプルコードでは以下を作成しております
1. バケット作成 : sample-data(readwrite)
2. ユーザー設定 : appuser:password123
### Webコンソールを利用する場合
#### Webコンソールにアクセス
ブラウザで `http://<ホスト>:9001` にアクセス
管理者アカウントでログイン(`MINIO_ROOT_USER` / `MINIO_ROOT_PASSWORD`
---
#### バケット作成・ファイルアップロード
- Web UI からバケット作成
- ローカルからファイルをアップロード可能
- バケット単位で「パブリックアクセス許可」も可能
### `mc` コマンドによる操作
#### インストール
```bash
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
sudo mv mc /usr/local/bin/
```
#### MinIOへの接続
```bash
mc alias set local http://localhost:9000 admin password123
```
#### 操作例
```bash
mc mb local/mybucket
mc cp ./example.jpg local/mybucket/
mc ls local/mybucket
```
### 各プログラム言語のライブラリから利用する場合
MinIOはAmazon S3互換のAPIを提供しているためS3対応の
クライアントライブラリを使用すればPythonやNode.jsから
簡単に接続・操作できます
* Python: `boto3`
* Node.js: `@aws-sdk/client-s3`
**接続時の注意点**
* リージョン
* 仮に指定しますが実際は無視される
* バージョニング/ライフサイクル/通知
* 一部のS3機能はMinIOでサポートされていない、または挙動が違う場合あり
* 署名付きURL
* boto3などで生成する署名付きURLも動作しますが、MinIOの設定によって挙動が異なることがある
### minioライブラリ(MinIO公式)
MinIO専用に最適化されていて、軽量・シンプルで扱いやすいのが特徴です。
S3互換APIを使ってはいますがAmazon純正のSDK@aws-sdk/*と違いMinIOのユースケースに特化してます
#### インストール方法
```sh
npm install minio
```
#### インストール方法
---
## 付録Tips
- リバースプロキシNginxを通して HTTPS 公開することで、ブラウザ互換性やセキュリティを向上できる
- Cloudflare などのCDNと組み合わせれば、簡易的なグローバル配信も可能
- アプリからは AWS SDK を使ってアクセス可能S3互換
---
## まとめ
MinIO は S3互換かつ軽量で、開発・検証・軽量サービスにおけるストレージニーズに柔軟に対応できる。Docker環境でも素早く構築可能なため、内部ツールやバックアップ用途として導入検討する価値がある。

25
examles/sampleMinio.js Normal file
View File

@ -0,0 +1,25 @@
async function main() {
const MinioStorage = require('../src/classes/MinioStorage');
const config = {
endPoint: 'localhost',
port: 9000,
useSSL: false,
accessKey: 'appuser',
secretKey: 'password123',
bucketName: 'sample-data'
}
const storage = new MinioStorage(config);
// ファイルを取得する
files = await storage.getFileList();
console.log(files);
// ファイルをアップロードする
await storage.uploadFile("./src/data/sample2.csv","sample2.csv");
files = await storage.getFileList();
console.log(files);
}
main();

460
package-lock.json generated
View File

@ -12,7 +12,8 @@
"@google-cloud/storage": "^7.15.2", "@google-cloud/storage": "^7.15.2",
"archiver": "^7.0.1", "archiver": "^7.0.1",
"csv-writer": "^1.6.0", "csv-writer": "^1.6.0",
"dotenv": "^16.4.7" "dotenv": "^16.4.7",
"minio": "^8.0.5"
}, },
"devDependencies": { "devDependencies": {
"jsdoc": "^4.0.4" "jsdoc": "^4.0.4"
@ -232,6 +233,13 @@
"integrity": "sha512-/Ad8+nIOV7Rl++6f1BdKxFSMgmoqEoYbHRpPcx3JEfv8VRsQe9Z4mCXeJBzxs7mbHY/XOZZuXlRNfhpVPbs6ZA==", "integrity": "sha512-/Ad8+nIOV7Rl++6f1BdKxFSMgmoqEoYbHRpPcx3JEfv8VRsQe9Z4mCXeJBzxs7mbHY/XOZZuXlRNfhpVPbs6ZA==",
"license": "MIT" "license": "MIT"
}, },
"node_modules/@zxing/text-encoding": {
"version": "0.9.0",
"resolved": "https://registry.npmjs.org/@zxing/text-encoding/-/text-encoding-0.9.0.tgz",
"integrity": "sha512-U/4aVJ2mxI0aDNI8Uq0wEhMgY+u4CNtEb0om3+y3+niDAsoTCOB33UF0sxpzqzdqXLqmvc+vZyAt4O8pPdfkwA==",
"license": "(Unlicense OR Apache-2.0)",
"optional": true
},
"node_modules/abort-controller": { "node_modules/abort-controller": {
"version": "3.0.0", "version": "3.0.0",
"resolved": "https://registry.npmjs.org/abort-controller/-/abort-controller-3.0.0.tgz", "resolved": "https://registry.npmjs.org/abort-controller/-/abort-controller-3.0.0.tgz",
@ -350,6 +358,21 @@
"integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==", "integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==",
"license": "MIT" "license": "MIT"
}, },
"node_modules/available-typed-arrays": {
"version": "1.0.7",
"resolved": "https://registry.npmjs.org/available-typed-arrays/-/available-typed-arrays-1.0.7.tgz",
"integrity": "sha512-wvUjBtSGN7+7SjNpq/9M2Tg350UZD3q62IFZLbRAR1bSMlCo1ZaeW+BJ+D090e4hIIZLBcTDWe4Mh4jvUDajzQ==",
"license": "MIT",
"dependencies": {
"possible-typed-array-names": "^1.0.0"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/b4a": { "node_modules/b4a": {
"version": "1.6.7", "version": "1.6.7",
"resolved": "https://registry.npmjs.org/b4a/-/b4a-1.6.7.tgz", "resolved": "https://registry.npmjs.org/b4a/-/b4a-1.6.7.tgz",
@ -398,6 +421,29 @@
"node": "*" "node": "*"
} }
}, },
"node_modules/block-stream2": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/block-stream2/-/block-stream2-2.1.0.tgz",
"integrity": "sha512-suhjmLI57Ewpmq00qaygS8UgEq2ly2PCItenIyhMqVjo4t4pGzqMvfgJuX8iWTeSDdfSSqS6j38fL4ToNL7Pfg==",
"license": "MIT",
"dependencies": {
"readable-stream": "^3.4.0"
}
},
"node_modules/block-stream2/node_modules/readable-stream": {
"version": "3.6.2",
"resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz",
"integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==",
"license": "MIT",
"dependencies": {
"inherits": "^2.0.3",
"string_decoder": "^1.1.1",
"util-deprecate": "^1.0.1"
},
"engines": {
"node": ">= 6"
}
},
"node_modules/bluebird": { "node_modules/bluebird": {
"version": "3.7.2", "version": "3.7.2",
"resolved": "https://registry.npmjs.org/bluebird/-/bluebird-3.7.2.tgz", "resolved": "https://registry.npmjs.org/bluebird/-/bluebird-3.7.2.tgz",
@ -414,6 +460,12 @@
"balanced-match": "^1.0.0" "balanced-match": "^1.0.0"
} }
}, },
"node_modules/browser-or-node": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/browser-or-node/-/browser-or-node-2.1.1.tgz",
"integrity": "sha512-8CVjaLJGuSKMVTxJ2DpBl5XnlNDiT4cQFeuCJJrvJmts9YrTZDizTX7PjC2s6W4x+MBGZeEY6dGMrF04/6Hgqg==",
"license": "MIT"
},
"node_modules/buffer": { "node_modules/buffer": {
"version": "6.0.3", "version": "6.0.3",
"resolved": "https://registry.npmjs.org/buffer/-/buffer-6.0.3.tgz", "resolved": "https://registry.npmjs.org/buffer/-/buffer-6.0.3.tgz",
@ -453,6 +505,24 @@
"integrity": "sha512-zRpUiDwd/xk6ADqPMATG8vc9VPrkck7T07OIx0gnjmJAnHnTVXNQG3vfvWNuiZIkwu9KrKdA1iJKfsfTVxE6NA==", "integrity": "sha512-zRpUiDwd/xk6ADqPMATG8vc9VPrkck7T07OIx0gnjmJAnHnTVXNQG3vfvWNuiZIkwu9KrKdA1iJKfsfTVxE6NA==",
"license": "BSD-3-Clause" "license": "BSD-3-Clause"
}, },
"node_modules/call-bind": {
"version": "1.0.8",
"resolved": "https://registry.npmjs.org/call-bind/-/call-bind-1.0.8.tgz",
"integrity": "sha512-oKlSFMcMwpUg2ednkhQ454wfWiU/ul3CkJe/PEHcTKuiX6RpbehUiFMXu13HalGZxfUwCQzZG747YXBn1im9ww==",
"license": "MIT",
"dependencies": {
"call-bind-apply-helpers": "^1.0.0",
"es-define-property": "^1.0.0",
"get-intrinsic": "^1.2.4",
"set-function-length": "^1.2.2"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/call-bind-apply-helpers": { "node_modules/call-bind-apply-helpers": {
"version": "1.0.2", "version": "1.0.2",
"resolved": "https://registry.npmjs.org/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.2.tgz", "resolved": "https://registry.npmjs.org/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.2.tgz",
@ -466,6 +536,22 @@
"node": ">= 0.4" "node": ">= 0.4"
} }
}, },
"node_modules/call-bound": {
"version": "1.0.4",
"resolved": "https://registry.npmjs.org/call-bound/-/call-bound-1.0.4.tgz",
"integrity": "sha512-+ys997U96po4Kx/ABpBCqhA9EuxJaQWDQg7295H4hBphv3IZg0boBKuwYpt4YXp6MZ5AmZQnU/tyMTlRpaSejg==",
"license": "MIT",
"dependencies": {
"call-bind-apply-helpers": "^1.0.2",
"get-intrinsic": "^1.3.0"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/catharsis": { "node_modules/catharsis": {
"version": "0.9.0", "version": "0.9.0",
"resolved": "https://registry.npmjs.org/catharsis/-/catharsis-0.9.0.tgz", "resolved": "https://registry.npmjs.org/catharsis/-/catharsis-0.9.0.tgz",
@ -593,6 +679,32 @@
} }
} }
}, },
"node_modules/decode-uri-component": {
"version": "0.2.2",
"resolved": "https://registry.npmjs.org/decode-uri-component/-/decode-uri-component-0.2.2.tgz",
"integrity": "sha512-FqUYQ+8o158GyGTrMFJms9qh3CqTKvAqgqsTnkLI8sKu0028orqBhxNMFkFen0zGyg6epACD32pjVk58ngIErQ==",
"license": "MIT",
"engines": {
"node": ">=0.10"
}
},
"node_modules/define-data-property": {
"version": "1.1.4",
"resolved": "https://registry.npmjs.org/define-data-property/-/define-data-property-1.1.4.tgz",
"integrity": "sha512-rBMvIzlpA8v6E+SJZoo++HAYqsLrkg7MSfIinMPFhmkorw7X+dOXVJQs+QT69zGkzMyfDnIMN2Wid1+NbL3T+A==",
"license": "MIT",
"dependencies": {
"es-define-property": "^1.0.0",
"es-errors": "^1.3.0",
"gopd": "^1.0.1"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/delayed-stream": { "node_modules/delayed-stream": {
"version": "1.0.0", "version": "1.0.0",
"resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz", "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
@ -761,6 +873,12 @@
"node": ">=6" "node": ">=6"
} }
}, },
"node_modules/eventemitter3": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-5.0.1.tgz",
"integrity": "sha512-GWkBvjiSZK87ELrYOSESUYeVIc9mvLLf/nXalMOS5dYrgZq9o5OVkbZAVM06CVxYsCwH9BDZFPlQTlPA1j4ahA==",
"license": "MIT"
},
"node_modules/events": { "node_modules/events": {
"version": "3.3.0", "version": "3.3.0",
"resolved": "https://registry.npmjs.org/events/-/events-3.3.0.tgz", "resolved": "https://registry.npmjs.org/events/-/events-3.3.0.tgz",
@ -800,6 +918,30 @@
"fxparser": "src/cli/cli.js" "fxparser": "src/cli/cli.js"
} }
}, },
"node_modules/filter-obj": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/filter-obj/-/filter-obj-1.1.0.tgz",
"integrity": "sha512-8rXg1ZnX7xzy2NGDVkBVaAy+lSlPNwad13BtgSlLuxfIslyt5Vg64U7tFcCt4WS1R0hvtnQybT/IyCkGZ3DpXQ==",
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/for-each": {
"version": "0.3.5",
"resolved": "https://registry.npmjs.org/for-each/-/for-each-0.3.5.tgz",
"integrity": "sha512-dKx12eRCVIzqCxFGplyFKJMPvLEWgmNtUrpTiJIR5u97zEhRG8ySrtboPHZXx7daLxQVrl643cTzbab2tkQjxg==",
"license": "MIT",
"dependencies": {
"is-callable": "^1.2.7"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/foreground-child": { "node_modules/foreground-child": {
"version": "3.3.1", "version": "3.3.1",
"resolved": "https://registry.npmjs.org/foreground-child/-/foreground-child-3.3.1.tgz", "resolved": "https://registry.npmjs.org/foreground-child/-/foreground-child-3.3.1.tgz",
@ -998,6 +1140,18 @@
"node": ">=14.0.0" "node": ">=14.0.0"
} }
}, },
"node_modules/has-property-descriptors": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/has-property-descriptors/-/has-property-descriptors-1.0.2.tgz",
"integrity": "sha512-55JNKuIW+vq4Ke1BjOTjM2YctQIvCT7GFzHwmfZPGo5wnrgkid0YQtnAleFSqumZm4az3n2BS+erby5ipJdgrg==",
"license": "MIT",
"dependencies": {
"es-define-property": "^1.0.0"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/has-symbols": { "node_modules/has-symbols": {
"version": "1.1.0", "version": "1.1.0",
"resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.1.0.tgz", "resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.1.0.tgz",
@ -1118,6 +1272,43 @@
"integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==", "integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
"license": "ISC" "license": "ISC"
}, },
"node_modules/ipaddr.js": {
"version": "2.2.0",
"resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-2.2.0.tgz",
"integrity": "sha512-Ag3wB2o37wslZS19hZqorUnrnzSkpOVy+IiiDEiTqNubEYpYuHWIf6K4psgN2ZWKExS4xhVCrRVfb/wfW8fWJA==",
"license": "MIT",
"engines": {
"node": ">= 10"
}
},
"node_modules/is-arguments": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/is-arguments/-/is-arguments-1.2.0.tgz",
"integrity": "sha512-7bVbi0huj/wrIAOzb8U1aszg9kdi3KN/CyU19CTI7tAoZYEZoL9yCDXpbXN+uPsuWnP02cyug1gleqq+TU+YCA==",
"license": "MIT",
"dependencies": {
"call-bound": "^1.0.2",
"has-tostringtag": "^1.0.2"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/is-callable": {
"version": "1.2.7",
"resolved": "https://registry.npmjs.org/is-callable/-/is-callable-1.2.7.tgz",
"integrity": "sha512-1BC0BVFhS/p0qtw6enp8e+8OD0UrK0oFLztSjNzhcKA3WDuJxxAPXzPuPtKkjEY9UUoEWlX/8fgKeu2S8i9JTA==",
"license": "MIT",
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/is-fullwidth-code-point": { "node_modules/is-fullwidth-code-point": {
"version": "3.0.0", "version": "3.0.0",
"resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-3.0.0.tgz", "resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-3.0.0.tgz",
@ -1127,6 +1318,42 @@
"node": ">=8" "node": ">=8"
} }
}, },
"node_modules/is-generator-function": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/is-generator-function/-/is-generator-function-1.1.0.tgz",
"integrity": "sha512-nPUB5km40q9e8UfN/Zc24eLlzdSf9OfKByBw9CIdw4H1giPMeA0OIJvbchsCu4npfI2QcMVBsGEBHKZ7wLTWmQ==",
"license": "MIT",
"dependencies": {
"call-bound": "^1.0.3",
"get-proto": "^1.0.0",
"has-tostringtag": "^1.0.2",
"safe-regex-test": "^1.1.0"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/is-regex": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/is-regex/-/is-regex-1.2.1.tgz",
"integrity": "sha512-MjYsKHO5O7mCsmRGxWcLWheFqN9DJ/2TmngvjKXihe6efViPqc274+Fx/4fYj/r03+ESvBdTXK0V6tA3rgez1g==",
"license": "MIT",
"dependencies": {
"call-bound": "^1.0.2",
"gopd": "^1.2.0",
"has-tostringtag": "^1.0.2",
"hasown": "^2.0.2"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/is-stream": { "node_modules/is-stream": {
"version": "2.0.1", "version": "2.0.1",
"resolved": "https://registry.npmjs.org/is-stream/-/is-stream-2.0.1.tgz", "resolved": "https://registry.npmjs.org/is-stream/-/is-stream-2.0.1.tgz",
@ -1139,6 +1366,21 @@
"url": "https://github.com/sponsors/sindresorhus" "url": "https://github.com/sponsors/sindresorhus"
} }
}, },
"node_modules/is-typed-array": {
"version": "1.1.15",
"resolved": "https://registry.npmjs.org/is-typed-array/-/is-typed-array-1.1.15.tgz",
"integrity": "sha512-p3EcsicXjit7SaskXHs1hA91QxgTw46Fv6EFKKGS5DRFLD8yKnohjF3hxoju94b/OcMZoQukzpPpBE9uLVKzgQ==",
"license": "MIT",
"dependencies": {
"which-typed-array": "^1.1.16"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/isarray": { "node_modules/isarray": {
"version": "1.0.0", "version": "1.0.0",
"resolved": "https://registry.npmjs.org/isarray/-/isarray-1.0.0.tgz", "resolved": "https://registry.npmjs.org/isarray/-/isarray-1.0.0.tgz",
@ -1416,6 +1658,31 @@
"url": "https://github.com/sponsors/isaacs" "url": "https://github.com/sponsors/isaacs"
} }
}, },
"node_modules/minio": {
"version": "8.0.5",
"resolved": "https://registry.npmjs.org/minio/-/minio-8.0.5.tgz",
"integrity": "sha512-/vAze1uyrK2R/DSkVutE4cjVoAowvIQ18RAwn7HrqnLecLlMazFnY0oNBqfuoAWvu7mZIGX75AzpuV05TJeoHg==",
"license": "Apache-2.0",
"dependencies": {
"async": "^3.2.4",
"block-stream2": "^2.1.0",
"browser-or-node": "^2.1.1",
"buffer-crc32": "^1.0.0",
"eventemitter3": "^5.0.1",
"fast-xml-parser": "^4.4.1",
"ipaddr.js": "^2.0.1",
"lodash": "^4.17.21",
"mime-types": "^2.1.35",
"query-string": "^7.1.3",
"stream-json": "^1.8.0",
"through2": "^4.0.2",
"web-encoding": "^1.1.5",
"xml2js": "^0.5.0 || ^0.6.2"
},
"engines": {
"node": "^16 || ^18 || >=20"
}
},
"node_modules/minipass": { "node_modules/minipass": {
"version": "7.1.2", "version": "7.1.2",
"resolved": "https://registry.npmjs.org/minipass/-/minipass-7.1.2.tgz", "resolved": "https://registry.npmjs.org/minipass/-/minipass-7.1.2.tgz",
@ -1528,6 +1795,15 @@
"url": "https://github.com/sponsors/isaacs" "url": "https://github.com/sponsors/isaacs"
} }
}, },
"node_modules/possible-typed-array-names": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/possible-typed-array-names/-/possible-typed-array-names-1.1.0.tgz",
"integrity": "sha512-/+5VFTchJDoVj3bhoqi6UeymcD00DAwb1nJwamzPvHEszJ4FpF6SNNbUbOS8yI56qHzdV8eK0qEfOSiodkTdxg==",
"license": "MIT",
"engines": {
"node": ">= 0.4"
}
},
"node_modules/process": { "node_modules/process": {
"version": "0.11.10", "version": "0.11.10",
"resolved": "https://registry.npmjs.org/process/-/process-0.11.10.tgz", "resolved": "https://registry.npmjs.org/process/-/process-0.11.10.tgz",
@ -1553,6 +1829,24 @@
"node": ">=6" "node": ">=6"
} }
}, },
"node_modules/query-string": {
"version": "7.1.3",
"resolved": "https://registry.npmjs.org/query-string/-/query-string-7.1.3.tgz",
"integrity": "sha512-hh2WYhq4fi8+b+/2Kg9CEge4fDPvHS534aOOvOZeQ3+Vf2mCFsaFBYj0i+iXcAq6I9Vzp5fjMFBlONvayDC1qg==",
"license": "MIT",
"dependencies": {
"decode-uri-component": "^0.2.2",
"filter-obj": "^1.1.0",
"split-on-first": "^1.0.0",
"strict-uri-encode": "^2.0.0"
},
"engines": {
"node": ">=6"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/readable-stream": { "node_modules/readable-stream": {
"version": "4.7.0", "version": "4.7.0",
"resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-4.7.0.tgz", "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-4.7.0.tgz",
@ -1643,6 +1937,46 @@
], ],
"license": "MIT" "license": "MIT"
}, },
"node_modules/safe-regex-test": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/safe-regex-test/-/safe-regex-test-1.1.0.tgz",
"integrity": "sha512-x/+Cz4YrimQxQccJf5mKEbIa1NzeCRNI5Ecl/ekmlYaampdNLPalVyIcCZNNH3MvmqBugV5TMYZXv0ljslUlaw==",
"license": "MIT",
"dependencies": {
"call-bound": "^1.0.2",
"es-errors": "^1.3.0",
"is-regex": "^1.2.1"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/sax": {
"version": "1.4.1",
"resolved": "https://registry.npmjs.org/sax/-/sax-1.4.1.tgz",
"integrity": "sha512-+aWOz7yVScEGoKNd4PA10LZ8sk0A/z5+nXQG5giUO5rprX9jgYsTdov9qCchZiPIZezbZH+jRut8nPodFAX4Jg==",
"license": "ISC"
},
"node_modules/set-function-length": {
"version": "1.2.2",
"resolved": "https://registry.npmjs.org/set-function-length/-/set-function-length-1.2.2.tgz",
"integrity": "sha512-pgRc4hJ4/sNjWCSS9AmnS40x3bNMDTknHgL5UaMBTMyJnU90EgWh1Rz+MC9eFu4BuN/UwZjKQuY/1v3rM7HMfg==",
"license": "MIT",
"dependencies": {
"define-data-property": "^1.1.4",
"es-errors": "^1.3.0",
"function-bind": "^1.1.2",
"get-intrinsic": "^1.2.4",
"gopd": "^1.0.1",
"has-property-descriptors": "^1.0.2"
},
"engines": {
"node": ">= 0.4"
}
},
"node_modules/shebang-command": { "node_modules/shebang-command": {
"version": "2.0.0", "version": "2.0.0",
"resolved": "https://registry.npmjs.org/shebang-command/-/shebang-command-2.0.0.tgz", "resolved": "https://registry.npmjs.org/shebang-command/-/shebang-command-2.0.0.tgz",
@ -1676,6 +2010,21 @@
"url": "https://github.com/sponsors/isaacs" "url": "https://github.com/sponsors/isaacs"
} }
}, },
"node_modules/split-on-first": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/split-on-first/-/split-on-first-1.1.0.tgz",
"integrity": "sha512-43ZssAJaMusuKWL8sKUBQXHWOpq8d6CfN/u1p4gUzfJkM05C8rxTmYrkIPTXapZpORA6LkkzcUulJ8FqA7Uudw==",
"license": "MIT",
"engines": {
"node": ">=6"
}
},
"node_modules/stream-chain": {
"version": "2.2.5",
"resolved": "https://registry.npmjs.org/stream-chain/-/stream-chain-2.2.5.tgz",
"integrity": "sha512-1TJmBx6aSWqZ4tx7aTpBDXK0/e2hhcNSTV8+CbFJtDjbb+I1mZ8lHit0Grw9GRT+6JbIrrDd8esncgBi8aBXGA==",
"license": "BSD-3-Clause"
},
"node_modules/stream-events": { "node_modules/stream-events": {
"version": "1.0.5", "version": "1.0.5",
"resolved": "https://registry.npmjs.org/stream-events/-/stream-events-1.0.5.tgz", "resolved": "https://registry.npmjs.org/stream-events/-/stream-events-1.0.5.tgz",
@ -1685,6 +2034,15 @@
"stubs": "^3.0.0" "stubs": "^3.0.0"
} }
}, },
"node_modules/stream-json": {
"version": "1.9.1",
"resolved": "https://registry.npmjs.org/stream-json/-/stream-json-1.9.1.tgz",
"integrity": "sha512-uWkjJ+2Nt/LO9Z/JyKZbMusL8Dkh97uUBTv3AJQ74y07lVahLY4eEFsPsE97pxYBwr8nnjMAIch5eqI0gPShyw==",
"license": "BSD-3-Clause",
"dependencies": {
"stream-chain": "^2.2.5"
}
},
"node_modules/stream-shift": { "node_modules/stream-shift": {
"version": "1.0.3", "version": "1.0.3",
"resolved": "https://registry.npmjs.org/stream-shift/-/stream-shift-1.0.3.tgz", "resolved": "https://registry.npmjs.org/stream-shift/-/stream-shift-1.0.3.tgz",
@ -1704,6 +2062,15 @@
"bare-events": "^2.2.0" "bare-events": "^2.2.0"
} }
}, },
"node_modules/strict-uri-encode": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/strict-uri-encode/-/strict-uri-encode-2.0.0.tgz",
"integrity": "sha512-QwiXZgpRcKkhTj2Scnn++4PKtWsH0kpzZ62L2R6c/LUVYv7hVnZqcg2+sMuT6R7Jusu1vviK/MFsu6kNJfWlEQ==",
"license": "MIT",
"engines": {
"node": ">=4"
}
},
"node_modules/string_decoder": { "node_modules/string_decoder": {
"version": "1.3.0", "version": "1.3.0",
"resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz", "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",
@ -1914,6 +2281,29 @@
"b4a": "^1.6.4" "b4a": "^1.6.4"
} }
}, },
"node_modules/through2": {
"version": "4.0.2",
"resolved": "https://registry.npmjs.org/through2/-/through2-4.0.2.tgz",
"integrity": "sha512-iOqSav00cVxEEICeD7TjLB1sueEL+81Wpzp2bY17uZjZN0pWZPuo4suZ/61VujxmqSGFfgOcNuTZ85QJwNZQpw==",
"license": "MIT",
"dependencies": {
"readable-stream": "3"
}
},
"node_modules/through2/node_modules/readable-stream": {
"version": "3.6.2",
"resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz",
"integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==",
"license": "MIT",
"dependencies": {
"inherits": "^2.0.3",
"string_decoder": "^1.1.1",
"util-deprecate": "^1.0.1"
},
"engines": {
"node": ">= 6"
}
},
"node_modules/tr46": { "node_modules/tr46": {
"version": "0.0.3", "version": "0.0.3",
"resolved": "https://registry.npmjs.org/tr46/-/tr46-0.0.3.tgz", "resolved": "https://registry.npmjs.org/tr46/-/tr46-0.0.3.tgz",
@ -1940,6 +2330,19 @@
"integrity": "sha512-Ny6QZ2Nju20vw1SRHe3d9jVu6gJ+4e3+MMpqu7pqE5HT6WsTSlce++GQmK5UXS8mzV8DSYHrQH+Xrf2jVcuKNg==", "integrity": "sha512-Ny6QZ2Nju20vw1SRHe3d9jVu6gJ+4e3+MMpqu7pqE5HT6WsTSlce++GQmK5UXS8mzV8DSYHrQH+Xrf2jVcuKNg==",
"license": "MIT" "license": "MIT"
}, },
"node_modules/util": {
"version": "0.12.5",
"resolved": "https://registry.npmjs.org/util/-/util-0.12.5.tgz",
"integrity": "sha512-kZf/K6hEIrWHI6XqOFUiiMa+79wE/D8Q+NCNAWclkyg3b4d2k7s0QGepNjiABc+aR3N1PAyHL7p6UcLY6LmrnA==",
"license": "MIT",
"dependencies": {
"inherits": "^2.0.3",
"is-arguments": "^1.0.4",
"is-generator-function": "^1.0.7",
"is-typed-array": "^1.1.3",
"which-typed-array": "^1.1.2"
}
},
"node_modules/util-deprecate": { "node_modules/util-deprecate": {
"version": "1.0.2", "version": "1.0.2",
"resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz", "resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz",
@ -1955,6 +2358,18 @@
"uuid": "dist/bin/uuid" "uuid": "dist/bin/uuid"
} }
}, },
"node_modules/web-encoding": {
"version": "1.1.5",
"resolved": "https://registry.npmjs.org/web-encoding/-/web-encoding-1.1.5.tgz",
"integrity": "sha512-HYLeVCdJ0+lBYV2FvNZmv3HJ2Nt0QYXqZojk3d9FJOLkwnuhzM9tmamh8d7HPM8QqjKH8DeHkFTx+CFlWpZZDA==",
"license": "MIT",
"dependencies": {
"util": "^0.12.3"
},
"optionalDependencies": {
"@zxing/text-encoding": "0.9.0"
}
},
"node_modules/webidl-conversions": { "node_modules/webidl-conversions": {
"version": "3.0.1", "version": "3.0.1",
"resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-3.0.1.tgz", "resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-3.0.1.tgz",
@ -1986,6 +2401,27 @@
"node": ">= 8" "node": ">= 8"
} }
}, },
"node_modules/which-typed-array": {
"version": "1.1.19",
"resolved": "https://registry.npmjs.org/which-typed-array/-/which-typed-array-1.1.19.tgz",
"integrity": "sha512-rEvr90Bck4WZt9HHFC4DJMsjvu7x+r6bImz0/BrbWb7A2djJ8hnZMrWnHo9F8ssv0OMErasDhftrfROTyqSDrw==",
"license": "MIT",
"dependencies": {
"available-typed-arrays": "^1.0.7",
"call-bind": "^1.0.8",
"call-bound": "^1.0.4",
"for-each": "^0.3.5",
"get-proto": "^1.0.1",
"gopd": "^1.2.0",
"has-tostringtag": "^1.0.2"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/wrap-ansi": { "node_modules/wrap-ansi": {
"version": "8.1.0", "version": "8.1.0",
"resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-8.1.0.tgz", "resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-8.1.0.tgz",
@ -2083,6 +2519,28 @@
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==", "integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
"license": "ISC" "license": "ISC"
}, },
"node_modules/xml2js": {
"version": "0.6.2",
"resolved": "https://registry.npmjs.org/xml2js/-/xml2js-0.6.2.tgz",
"integrity": "sha512-T4rieHaC1EXcES0Kxxj4JWgaUQHDk+qwHcYOCFHfiwKz7tOVPLq7Hjq9dM1WCMhylqMEfP7hMcOIChvotiZegA==",
"license": "MIT",
"dependencies": {
"sax": ">=0.6.0",
"xmlbuilder": "~11.0.0"
},
"engines": {
"node": ">=4.0.0"
}
},
"node_modules/xmlbuilder": {
"version": "11.0.1",
"resolved": "https://registry.npmjs.org/xmlbuilder/-/xmlbuilder-11.0.1.tgz",
"integrity": "sha512-fDlsI/kFEx7gLvbecc0/ohLG50fugQp8ryHzMTuW9vSa1GJ0XYWKnhsUx7oie3G98+r56aTQIUB4kht42R3JvA==",
"license": "MIT",
"engines": {
"node": ">=4.0"
}
},
"node_modules/xmlcreate": { "node_modules/xmlcreate": {
"version": "2.0.4", "version": "2.0.4",
"resolved": "https://registry.npmjs.org/xmlcreate/-/xmlcreate-2.0.4.tgz", "resolved": "https://registry.npmjs.org/xmlcreate/-/xmlcreate-2.0.4.tgz",

View File

@ -16,7 +16,8 @@
"@google-cloud/storage": "^7.15.2", "@google-cloud/storage": "^7.15.2",
"archiver": "^7.0.1", "archiver": "^7.0.1",
"csv-writer": "^1.6.0", "csv-writer": "^1.6.0",
"dotenv": "^16.4.7" "dotenv": "^16.4.7",
"minio": "^8.0.5"
}, },
"devDependencies": { "devDependencies": {
"jsdoc": "^4.0.4" "jsdoc": "^4.0.4"

View File

@ -0,0 +1,43 @@
// @ts-check を付けると型チェックが有効に!
/** @ts-check */
// 型定義をインポート
const Minio = require('minio');
const fs = require('fs');
class MinioStorage {
constructor(config) {
this.minioClient = new Minio.Client({
endPoint: config.endPoint,
port: config.port,
useSSL: config.useSSL,
accessKey: config.accessKey,
secretKey: config.secretKey
});
this.bucketName = config.bucketName;
}
getFileList(prefix = '') {
return new Promise((resolve, reject) => {
const objectKeys = [];
const stream = this.minioClient.listObjects(this.bucketName, prefix, true);
stream.on('data', obj => objectKeys.push(obj.name));
stream.on('end', () => resolve(objectKeys));
stream.on('error', err => reject(err));
});
}
async uploadFile(filePath, objectKey) {
return this.minioClient.fPutObject(this.bucketName, objectKey, filePath)
.then(() => {
console.log('File uploaded successfully.');
})
.catch(err => {
console.error('File upload error:', err);
throw err;
});
}
}
module.exports = MinioStorage;

11
src/data/sample1.csv Normal file
View File

@ -0,0 +1,11 @@
ID,Name,Age,Email
1,User_1,26,user1@example.com
2,User_2,65,user2@example.com
3,User_3,39,user3@example.com
4,User_4,51,user4@example.com
5,User_5,25,user5@example.com
6,User_6,26,user6@example.com
7,User_7,69,user7@example.com
8,User_8,47,user8@example.com
9,User_9,40,user9@example.com
10,User_10,28,user10@example.com
1 ID Name Age Email
2 1 User_1 26 user1@example.com
3 2 User_2 65 user2@example.com
4 3 User_3 39 user3@example.com
5 4 User_4 51 user4@example.com
6 5 User_5 25 user5@example.com
7 6 User_6 26 user6@example.com
8 7 User_7 69 user7@example.com
9 8 User_8 47 user8@example.com
10 9 User_9 40 user9@example.com
11 10 User_10 28 user10@example.com

11
src/data/sample2.csv Normal file
View File

@ -0,0 +1,11 @@
ID,Name,Age,Email
1,User_1,63,user1@example.com
2,User_2,46,user2@example.com
3,User_3,69,user3@example.com
4,User_4,62,user4@example.com
5,User_5,64,user5@example.com
6,User_6,52,user6@example.com
7,User_7,41,user7@example.com
8,User_8,64,user8@example.com
9,User_9,22,user9@example.com
10,User_10,62,user10@example.com
1 ID Name Age Email
2 1 User_1 63 user1@example.com
3 2 User_2 46 user2@example.com
4 3 User_3 69 user3@example.com
5 4 User_4 62 user4@example.com
6 5 User_5 64 user5@example.com
7 6 User_6 52 user6@example.com
8 7 User_7 41 user7@example.com
9 8 User_8 64 user8@example.com
10 9 User_9 22 user9@example.com
11 10 User_10 62 user10@example.com

199
src/script/minio/README.md Normal file
View File

@ -0,0 +1,199 @@
# MinIO 利用ガイド
## 概要
[MinIO](https://min.io/) は Amazon S3 互換のオブジェクトストレージサーバー。軽量かつシンプルな構成で、オンプレミスやクラウド上に S3 互換のストレージ環境を構築できる。
高可用構成やセキュリティ機能も備えており、小規模から大規模用途まで幅広く利用できる。本記事では MinIO の導入方法、仕様要件、ユースケース、基本的な利用方法についてまとめる。
---
## 1. インストール方法
### 1.1 Docker を使う場合
```bash
docker run -p 9000:9000 -p 9001:9001 \
--name minio \
-e "MINIO_ROOT_USER=admin" \
-e "MINIO_ROOT_PASSWORD=password123" \
-v /mnt/data:/data \
quay.io/minio/minio server /data --console-address ":9001"
```
- `9000`: S3互換API用ポート
- `9001`: Webコンソール用ポート
![mino-top.png](/mino-top.png)
---
### 1.2 Docker Compose を使う場合
```yaml
version: '3.8'
services:
minio:
image: quay.io/minio/minio
ports:
- "9000:9000"
- "9001:9001"
volumes:
- ./data:/data
environment:
MINIO_ROOT_USER: admin
MINIO_ROOT_PASSWORD: password123
command: server /data --console-address ":9001"
restart: unless-stopped
```
起動コマンド:
```bash
docker compose up -d
```
---
### 1.3 バイナリから直接実行する場合Linux
```bash
wget https://dl.min.io/server/minio/release/linux-amd64/minio
chmod +x minio
sudo mv minio /usr/local/bin/
export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=password123
minio server /mnt/data --console-address ":9001"
```
---
## 2. 仕様要件
| 項目 | 推奨構成例 |
|--------------|------------------------------|
| OS | LinuxUbuntu 22.04 など) |
| CPU | 1コア以上最低限 |
| メモリ | 1GB 以上2GB 以上推奨) |
| ストレージ | SSD または HDD |
| 通信 | ポート 9000 / 9001開放 |
- HTTPS を使用する場合は TLS 証明書Let's Encrypt など)を導入
- リバースプロキシNginx/Caddyなど経由での公開も可能
---
## 3. ユースケース・機能
### 主なユースケース
- バックアップ保存
- Webアプリの画像・動画ストレージ
- ログデータ保管
- オンプレS3互換ストレージ
### 主な機能
- S3 互換API対応
- Webベース管理コンソール
- IAM風のユーザー・ポリシー制御
- TLS対応HTTPS通信
- サーバーサイド暗号化AES-256
- マルチテナント対応(認証連携含む)
- オブジェクトのバージョン管理(設定時)
---
## 4. 実際の使い方
サンプルコードでは以下を作成しております
1. バケット作成 : sample-data(readwrite)
2. ユーザー設定 : appuser:password123
### Webコンソールを利用する場合
#### Webコンソールにアクセス
ブラウザで `http://<ホスト>:9001` にアクセス
管理者アカウントでログイン(`MINIO_ROOT_USER` / `MINIO_ROOT_PASSWORD`
---
#### バケット作成・ファイルアップロード
- Web UI からバケット作成
- ローカルからファイルをアップロード可能
- バケット単位で「パブリックアクセス許可」も可能
### `mc` コマンドによる操作
#### インストール
```bash
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
sudo mv mc /usr/local/bin/
```
#### MinIOへの接続
```bash
mc alias set local http://localhost:9000 admin password123
```
#### 操作例
```bash
mc mb local/mybucket
mc cp ./example.jpg local/mybucket/
mc ls local/mybucket
```
### 各プログラム言語のライブラリから利用する場合
MinIOはAmazon S3互換のAPIを提供しているためS3対応の
クライアントライブラリを使用すればPythonやNode.jsから
簡単に接続・操作できます
* Python: `boto3`
* Node.js: `@aws-sdk/client-s3`
**接続時の注意点**
* リージョン
* 仮に指定しますが実際は無視される
* バージョニング/ライフサイクル/通知
* 一部のS3機能はMinIOでサポートされていない、または挙動が違う場合あり
* 署名付きURL
* boto3などで生成する署名付きURLも動作しますが、MinIOの設定によって挙動が異なることがある
### minioライブラリ(MinIO公式)
MinIO専用に最適化されていて、軽量・シンプルで扱いやすいのが特徴です。
S3互換APIを使ってはいますがAmazon純正のSDK@aws-sdk/*と違いMinIOのユースケースに特化してます
#### インストール方法
```sh
npm install minio
```
#### インストール方法
---
## 付録Tips
- リバースプロキシNginxを通して HTTPS 公開することで、ブラウザ互換性やセキュリティを向上できる
- Cloudflare などのCDNと組み合わせれば、簡易的なグローバル配信も可能
- アプリからは AWS SDK を使ってアクセス可能S3互換
---
## まとめ
MinIO は S3互換かつ軽量で、開発・検証・軽量サービスにおけるストレージニーズに柔軟に対応できる。Docker環境でも素早く構築可能なため、内部ツールやバックアップ用途として導入検討する価値がある。

View File

@ -0,0 +1,20 @@
const Minio = require('minio');
const minioClient = new Minio.Client({
endPoint: 'localhost',
port: 9000,
useSSL: false,
accessKey: 'appuser',
secretKey: 'password123'
});
const BUCKET_NAME = 'sample-data';
const OBJECT_NAME = 'sample2.csv';
minioClient.removeObject(BUCKET_NAME, OBJECT_NAME, (err) => {
if (err) {
return console.log("file delete error:", err);
}
console.log('File deleted successfully.');
});

View File

@ -0,0 +1,19 @@
const Minio = require('minio');
const minioClient = new Minio.Client({
endPoint: 'localhost',
port: 9000,
useSSL: false,
accessKey: 'appuser',
secretKey: 'password123'
});
const BUCKET_NAME = 'sample-data';
const OBJECT_NAMES = ['file1.csv', 'file2.csv'];
minioClient.removeObjects(BUCKET_NAME, OBJECT_NAMES, (err) => {
if (err) {
return console.log("files delete error:", err);
}
console.log('Files deleted successfully.');
});

View File

@ -0,0 +1,35 @@
/**
* MinIOに保存されているファイル一覧を取得する
*/
const Minio = require('minio');
const fs = require('fs');
const minioClient = new Minio.Client({
endPoint: 'localhost',
port: 9000,
useSSL: false,
accessKey: 'appuser',
secretKey: 'password123'
});
const BUCKET_NAME = 'sample-data';
const OBJECT_NAME = 'sample1.csv';
minioClient.getObject(BUCKET_NAME, OBJECT_NAME, (err, dataStream) => {
if (err) {
return console.log("file get error:", err);
}
const file = fs.createWriteStream('./sample1_direct.csv');
dataStream.pipe(file);
console.log('File downloaded successfully.');
dataStream.on('end', () => {
console.log('File Download File end');
});
dataStream.on('error', err => {
console.error('File Download Err:', err);
});
});

View File

@ -0,0 +1,28 @@
/**
* MinIOに保存されているファイル一覧を取得する
*/
const Minio = require('minio');
const minioClient = new Minio.Client({
endPoint: 'localhost',
port: 9000,
useSSL: false,
accessKey: 'appuser',
secretKey: 'password123'
});
const BUCKET_NAME = 'sample-data';
// listObjects はストリームなので順に読み取る必要があります
const stream = minioClient.listObjects(BUCKET_NAME, '', true); // true: 再帰的にすべて取得
stream.on('data', obj => {
console.log(`data ${obj.name} (${obj.size} bytes)`);
});
stream.on('end', () => {
console.log('Get File list end');
});
stream.on('error', err => {
console.error('Files get Err:', err);
});

View File

@ -0,0 +1,27 @@
/**
* MinIOに保存されているファイル一覧を非同期関数で取得する
*/
const Minio = require('minio');
const minioClient = new Minio.Client({
endPoint: 'localhost',
port: 9000,
useSSL: false,
accessKey: 'appuser',
secretKey: 'password123'
});
const BUCKET_NAME = 'sample-data';
function listAllObjects(bucket) {
return new Promise((resolve, reject) => {
const result = [];
const stream = minioClient.listObjects(bucket, '', true);
stream.on('data', obj => result.push(obj));
stream.on('end', () => resolve(result));
stream.on('error', reject);
});
}
const objects = await listAllObjects(BUCKET_NAME);
console.log(objects.map(obj => obj.name));

View File

@ -0,0 +1,32 @@
const Minio = require('minio');
const BUCKET_NAME = 'sample-data';
const minioClient = new Minio.Client({
endPoint: 'localhost',
port: 9000,
useSSL: false,
accessKey: 'appuser',
secretKey: 'password123'
});
// presigned GET URL を生成(有効期限: 1時間
minioClient.presignedGetObject(BUCKET_NAME, 'sample2.csv', 60 * 60, (err, url) => {
if (err) {
return console.log("file get signatureURL error:", err);
}
console.log('✅ Presigned GET URL:', url)
// fetchで取得してファイルに保存任意
// fetch(url)
// .then(res => res.buffer())
// .then(data => {
// // Bufferを使ってファイルに保存オプション
// import('fs').then(fs => {
// fs.writeFileSync('./downloaded_sample2.csv', data);
// console.log('File downloaded successfully.');
// });
// });
});

View File

@ -0,0 +1,22 @@
const Minio = require('minio');
const BUCKET_NAME = 'sample-data';
const minioClient = new Minio.Client({
endPoint: 'localhost',
port: 9000,
useSSL: false,
accessKey: 'appuser',
secretKey: 'password123'
});
// ファイルアップロード(src/data/sample1.csv)
minioClient.fPutObject(BUCKET_NAME, 'sample1.csv', './src/data/sample1.csv', function(err, etag) {
if (err){
return console.log("file upload error:", err);
}
console.log('File uploaded successfully.',etag);
});

View File

@ -0,0 +1,30 @@
const Minio = require('minio');
const BUCKET_NAME = 'sample-data';
const minioClient = new Minio.Client({
endPoint: 'localhost',
port: 9000,
useSSL: false,
accessKey: 'appuser',
secretKey: 'password123'
});
// ファイルアップロード(src/data/sample1.csv)
minioClient.presignedPutObject(BUCKET_NAME, 'sample2.csv',60 * 60, function(err, presignedUrl) {
if (err){
return console.log("file upload error:", err);
}
console.log('File uploaded URL:',presignedUrl);
// ダウンロードしたURLにPUTリクエストを送信する
});
// curl -X PUT -T .\src\data\sample2.csv "presignedUrl"
// const fileData = await fs.readFile(filePath);
// const res = await fetch(uploadUrl, {
// method: 'PUT',
// body: fileData,
// headers: {
// 'Content-Type': 'text/csv'
// }
// });

75
src/types/IStorage.d.ts vendored Normal file
View File

@ -0,0 +1,75 @@
/**
* S3 / GCS / MinIO
*/
export enum ProcessType {
GET = 0,
PUT = 1
}
export interface IStorage {
/**
*
* @param filePath
* @param objectKey
*/
uploadFile: (filePath: string, objectKey: string) => Promise<void>;
/**
*
* @param fileData
* @param objectKey
* @param contentType Content-Type application/octet-stream
*/
uploadContents?: (
fileData: Buffer | string,
objectKey: string,
contentType?: string
) => Promise<void>;
/**
*
* @param objectKey
* @param destinationPath
* @returns Buffer
*/
download(objectKey: string, destinationPath: string): Promise<void>;
/**
*
* @param objectKey
* @returns Buffer
*/
downloadContents?(objectKey: string): Promise<Buffer>;
/**
*
* @param objectKey
*/
delete(objectKey: string): Promise<void>;
/**
*
* @param objectKeys
*/
deleteMany?(objectKeys: string[]): Promise<void>;
/**
*
* @param prefix
* @returns
*/
getFileList(prefix?: string): Promise<string[]>;
/**
* URLを発行する
* @param objectKey
* @param processType : 0 = GET, 1 = PUT
* @param options
* @returns URL
*/
getSignedUrl(
objectKey: string,
processType: ProcessType,
options: { expiresIn: number }
): Promise<string>;
}