240 lines
5.7 KiB
Markdown
240 lines
5.7 KiB
Markdown
# swfss3
|
||
|
||
`swfss3` is a small CLI for performing **filesystem-like operations** (list/read/write/delete)
|
||
against a **SeaweedFS S3-compatible** API endpoint using the Rust `s3` crate.
|
||
|
||
## Build
|
||
|
||
```bash
|
||
cargo build --release
|
||
```
|
||
|
||
Binary will be at `target/release/swfss3`.
|
||
|
||
## Authentication and configuration
|
||
|
||
You must provide:
|
||
|
||
- **Endpoint**: `--endpoint` (custom domain / S3 API URL)
|
||
- **Bucket**: `--bucket`
|
||
- **Region**: `--region` (defaults to `us-east-1`)
|
||
|
||
Authentication (choose one):
|
||
|
||
- **Flags**: `--access-key-id` and `--secret-access-key` (optional `--session-token`)
|
||
- **Environment variables**: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`
|
||
- **Anonymous**: `--anonymous` (unsigned requests)
|
||
|
||
If you see `invalid config: missing AWS_ACCESS_KEY_ID`, it means you did not pass credentials
|
||
flags and `swfss3` could not find the AWS environment variables. Fix it by using **one** of:
|
||
|
||
- **Provide keys via flags**:
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
--access-key-id YOUR_KEY --secret-access-key YOUR_SECRET \
|
||
list --prefix some/path/
|
||
```
|
||
|
||
- **Export keys via environment variables**:
|
||
|
||
```bash
|
||
export AWS_ACCESS_KEY_ID=YOUR_KEY
|
||
export AWS_SECRET_ACCESS_KEY=YOUR_SECRET
|
||
# optional (temporary credentials)
|
||
export AWS_SESSION_TOKEN=YOUR_TOKEN
|
||
```
|
||
|
||
- **Use unsigned requests** (only works if the bucket/object is publicly readable/writable):
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style --anonymous \
|
||
list --prefix some/path/
|
||
```
|
||
|
||
Addressing style (optional, but often important for S3-compatible servers):
|
||
|
||
- `--path-style` (recommended for many self-hosted endpoints)
|
||
- `--virtual-hosted` (bucket as a subdomain)
|
||
- default is automatic selection
|
||
|
||
## Commands
|
||
|
||
All commands share the common flags above and then take their own arguments.
|
||
|
||
### List objects
|
||
|
||
List a prefix (like listing a directory):
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
list --prefix some/path/
|
||
```
|
||
|
||
Recursive listing (no delimiter grouping):
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
list --prefix some/path/ --recursive
|
||
```
|
||
|
||
Fetch all pages:
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
list --prefix some/path/ --all
|
||
```
|
||
|
||
### Read an object
|
||
|
||
To stdout:
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
read some/path/file.txt
|
||
```
|
||
|
||
To a local file:
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
read some/path/file.txt --out ./file.txt
|
||
```
|
||
|
||
### Write an object
|
||
|
||
Upload a local file to an object key:
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
write ./local.bin some/path/local.bin
|
||
```
|
||
|
||
Optionally set `Content-Type`:
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
write ./index.html some/path/index.html --content-type text/html
|
||
```
|
||
|
||
### Delete an object
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
delete some/path/local.bin
|
||
```
|
||
|
||
### Stat (HEAD) an object
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
stat some/path/file.txt
|
||
```
|
||
|
||
### Copy / move (rename) an object
|
||
|
||
Server-side copy:
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
cp some/path/file.txt some/path/file-copy.txt
|
||
```
|
||
|
||
Move (copy then delete):
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
mv some/path/file.txt some/path/renamed.txt
|
||
```
|
||
|
||
### Presigned URLs
|
||
|
||
Create a temporary download URL (default 900s expiry):
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
presign-get some/path/file.txt --expires 300
|
||
```
|
||
|
||
Create a temporary upload URL:
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
presign-put some/path/upload.bin --expires 300
|
||
```
|
||
|
||
### Delete everything under a prefix
|
||
|
||
Dry-run (prints keys that would be deleted):
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
rm-prefix some/path/
|
||
```
|
||
|
||
Actually delete:
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
rm-prefix some/path/ --yes
|
||
```
|
||
|
||
### Mount bucket via FUSE3 (Linux)
|
||
|
||
Mount the whole bucket read-only:
|
||
|
||
```bash
|
||
mkdir -p /tmp/s3mnt
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
mount /tmp/s3mnt
|
||
```
|
||
|
||
Mount only a prefix (the prefix becomes the filesystem root):
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
mount /tmp/s3mnt --prefix photos/2025/
|
||
```
|
||
|
||
Enable writes (naive buffered uploads on flush/close):
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
mount /tmp/s3mnt --read-write
|
||
```
|
||
|
||
Allow access by other users (requires `user_allow_other` in `/etc/fuse.conf`):
|
||
|
||
```bash
|
||
swfss3 --endpoint http://localhost:8333 --bucket mybucket --path-style \
|
||
mount /tmp/s3mnt --allow-other
|
||
```
|
||
|
||
## Troubleshooting
|
||
|
||
### `NoSuchBucket` but the bucket “exists”
|
||
|
||
If you get an error like `code=NoSuchBucket`, double-check:
|
||
|
||
- **You are hitting the S3 gateway endpoint**, not the filer UI or another reverse-proxied service.
|
||
- **The bucket exists in the same endpoint/cluster**. Buckets are endpoint-scoped.
|
||
- **Addressing style**: if one style doesn’t work, try the other:
|
||
- `--path-style` (common for self-hosted)
|
||
- `--virtual-hosted` (requires bucket DNS/wildcard)
|
||
|
||
## Help
|
||
|
||
```bash
|
||
swfss3 --help
|
||
swfss3 list --help
|
||
swfss3 read --help
|
||
swfss3 write --help
|
||
swfss3 delete --help
|
||
swfss3 stat --help
|
||
swfss3 cp --help
|
||
swfss3 mv --help
|
||
swfss3 presign-get --help
|
||
swfss3 presign-put --help
|
||
swfss3 rm-prefix --help
|
||
swfss3 mount --help
|
||
```
|