datasketch-io

Setting your Docker secrets

The docker compose file for the Datasketch stack requires that the pg_passwd_datasketch and datasketch_key secret be set before deployment.

You will need to ssh into the node manager and create these secrets:

echo "mysecret" | docker secret create <secret_name> -

Build docker image

docker build -t androiddrew/datasketch:latest -f ./services/cms/Dockerfile ./services/cms

To test the build locally on your laptop:

docker run -it --name datasketch_cms -e DB_HOST="host.docker.internal" -e DB_PORT=5432 -p 8888:5000 androiddrew/datasketch:<tag> gunicorn cms.wsgi:application --bind 0.0.0.0:5000 --workers 3

If using the postgres.app on Mac OS you will need to update both the postgresql.conf and pg_hba.conf files

/Users/<user_name>/Library/Application Support/Postgres/var-10/postgresql.conf

listen_addresses = '*'

/Users/<user_name>/Library/Application Support/Postgres/var-10/pg_hba.conf

# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     trust
# IPv4 local connections:
host    all             all             127.0.0.1/32            trust
# IPv6 local connections:
host    all             all             ::1/128                 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local   replication     all                                     trust
host    replication     all             127.0.0.1/32            trust
host    replication     all             ::1/128                 trust
host    datasketch      datasketch      host.docker.internal    trust

Trouble shooting

You can run the following command to get the logs for a task, running or not.

docker logs $(docker inspect --format "{{.Status.ContainerStatus.ContainerID}}" <task_id>)