s3watch is a Go daemon that periodically keeps an S3 bucket or prefix and a local directory in sync. When a sync cycle downloads, uploads, updates, or deletes files, it runs an optional hook script.

The daemon uses the AWS SDK directly. It does not shell out to aws s3 sync or any other external sync tool.

Usage

s3watch -bucket my-bucket -prefix app/config -dir /srv/config -hook /usr/local/bin/reload-app

You can also use a JSON config file:

s3watch -config examples/s3watch.json

Useful flags:

-config examples/s3watch.json: loads daemon settings from JSON.
-interval 1m: controls the sync period.
-once: runs a single sync cycle and exits.
-prune: propagates deletes for files that were previously synced and are unchanged on the remaining side.
-region us-east-1: overrides the AWS SDK's default region resolution.
-key-id KEY: uses explicit S3 access key credentials.
-application-key SECRET: uses explicit S3 secret or application key credentials.
-endpoint-url https://s3.example.com: uses a custom S3-compatible endpoint.
-insecure-skip-verify: skips TLS certificate verification for the S3 server. Use only with trusted endpoints.
-http-timeout 30s: sets a timeout for AWS HTTP requests.

Flags override values loaded from -config.

Example config:

{
  "bucket": "my-application-config",
  "prefix": "production/web",
  "dir": "/srv/myapp/config",
  "hook": "/usr/local/bin/reload-myapp",
  "region": "us-east-1",
  "key_id": "REPLACE_WITH_KEY_ID",
  "application_key": "REPLACE_WITH_APPLICATION_KEY",
  "endpoint_url": "https://s3.us-east-1.amazonaws.com",
  "insecure_skip_verify": false,
  "interval": "1m",
  "http_timeout": "30s",
  "once": false,
  "prune": true
}

The hook runs with the synced directory as its working directory and receives:

S3WATCH_CHANGED
S3WATCH_DOWNLOADED
S3WATCH_UPLOADED
S3WATCH_UPDATED
S3WATCH_DELETED
S3WATCH_UNCHANGED

Sync Semantics

Local-only files are uploaded to S3. Remote-only objects are downloaded locally. When a file exists on both sides and differs, the side with the newer modified time wins.

With prune disabled, missing files are treated as new files on the side where they still exist. With prune enabled, s3watch uses local state in .s3watch/state.json to propagate deletes only when the remaining copy is unchanged since the last successful sync.

Credentials

AWS credentials are resolved by the standard AWS SDK chain, including environment variables, shared config files, web identity, and instance or task roles.

If key_id and application_key are set, those static credentials override the standard AWS SDK credential chain. They must be provided together.