83 lines
2.6 KiB
Markdown
83 lines
2.6 KiB
Markdown
# s3watch
|
|
|
|
`s3watch` is a Go daemon that periodically keeps an S3 bucket or prefix and a
|
|
local directory in sync. When a sync cycle downloads, uploads, updates, or
|
|
deletes files, it runs an optional hook script.
|
|
|
|
The daemon uses the AWS SDK directly. It does not shell out to `aws s3 sync` or
|
|
any other external sync tool.
|
|
|
|
## Usage
|
|
|
|
```sh
|
|
s3watch -bucket my-bucket -prefix app/config -dir /srv/config -hook /usr/local/bin/reload-app
|
|
```
|
|
|
|
You can also use a JSON config file:
|
|
|
|
```sh
|
|
s3watch -config examples/s3watch.json
|
|
```
|
|
|
|
Useful flags:
|
|
|
|
- `-config examples/s3watch.json`: loads daemon settings from JSON.
|
|
- `-interval 1m`: controls the sync period.
|
|
- `-once`: runs a single sync cycle and exits.
|
|
- `-prune`: propagates deletes for files that were previously synced and are unchanged on the remaining side.
|
|
- `-region us-east-1`: overrides the AWS SDK's default region resolution.
|
|
- `-key-id KEY`: uses explicit S3 access key credentials.
|
|
- `-application-key SECRET`: uses explicit S3 secret or application key credentials.
|
|
- `-endpoint-url https://s3.example.com`: uses a custom S3-compatible endpoint.
|
|
- `-http-timeout 30s`: sets a timeout for AWS HTTP requests.
|
|
|
|
Flags override values loaded from `-config`.
|
|
|
|
Example config:
|
|
|
|
```json
|
|
{
|
|
"bucket": "my-application-config",
|
|
"prefix": "production/web",
|
|
"dir": "/srv/myapp/config",
|
|
"hook": "/usr/local/bin/reload-myapp",
|
|
"region": "us-east-1",
|
|
"key_id": "REPLACE_WITH_KEY_ID",
|
|
"application_key": "REPLACE_WITH_APPLICATION_KEY",
|
|
"endpoint_url": "https://s3.us-east-1.amazonaws.com",
|
|
"interval": "1m",
|
|
"http_timeout": "30s",
|
|
"once": false,
|
|
"prune": true
|
|
}
|
|
```
|
|
|
|
The hook runs with the synced directory as its working directory and receives:
|
|
|
|
- `S3WATCH_CHANGED`
|
|
- `S3WATCH_DOWNLOADED`
|
|
- `S3WATCH_UPLOADED`
|
|
- `S3WATCH_UPDATED`
|
|
- `S3WATCH_DELETED`
|
|
- `S3WATCH_UNCHANGED`
|
|
|
|
## Sync Semantics
|
|
|
|
Local-only files are uploaded to S3. Remote-only objects are downloaded locally.
|
|
When a file exists on both sides and differs, the side with the newer modified
|
|
time wins.
|
|
|
|
With `prune` disabled, missing files are treated as new files on the side where
|
|
they still exist. With `prune` enabled, `s3watch` uses local state in
|
|
`.s3watch/state.json` to propagate deletes only when the remaining copy is
|
|
unchanged since the last successful sync.
|
|
|
|
## Credentials
|
|
|
|
AWS credentials are resolved by the standard AWS SDK chain, including
|
|
environment variables, shared config files, web identity, and instance or task
|
|
roles.
|
|
|
|
If `key_id` and `application_key` are set, those static credentials override the
|
|
standard AWS SDK credential chain. They must be provided together.
|