Pre-1.0 hoardDB is pre-1.0. Expect breaking changes.

Security

What hoardDB protects out of the box, what you have to turn on, what it does not protect, and how to tell us about a vulnerability. Every feature on this page is free and unlocked in every build; there is no enterprise tier for TLS, users or encryption.

What a fresh install does

With no configuration, hoarddb-server:

• serves every connection over TLS 1.3, with no way to downgrade;
• requires a login on every connection before it will answer anything;
• creates the root user the first time it starts: it asks for the password on a terminal, and with no terminal it refuses until you set HOARDDB_ROOT_USER and HOARDDB_ROOT_PASSWORD. It never makes a password up or prints one, and it shows the cluster token only on a terminal. Only the password’s hash is kept, in <data dir>/users.json; the password is never written to disk, so a copy or backup of the data directory does not carry it;
• creates its own private keys with mode 0600, in the data directory;
• stores passwords only as hashes, and leaves the audit log, the metrics endpoint and the profiling endpoint off.

It does not encrypt the data on disk until you ask it to, and it listens on loopback only until you change HOARDDB_TCP_LISTEN. Both are covered below.

TLS and self-signed certificates

A node generates an Ed25519 key and a self-signed certificate on its first start and serves them from <data dir>/keys/server.key and server.crt. No certificate authority is involved, so there is nothing to renew and nothing for a client to download — and nothing that proves to a client that it reached the right machine, other than the certificate’s fingerprint.

The server prints its fingerprint every time it starts, in the line that says it is listening:

{"level":"INFO","msg":"TCP driver listener listening","address":"[::]:7433","fingerprint":"SHA256:d8aa96c0…","alpn":"hoarddb/1"}

Trusting a server from the CLI

The CLI treats a server it has never seen the way ssh treats a new host. It shows the fingerprint and asks, and it remembers the answer, for that address, in ~/.hoarddb/known_servers. A later connection to the same address whose key is different fails with WARNING: server key for <address> has changed! and shows both fingerprints, instead of asking again. If you did replace the server, remove the old entry from that file; if you did not, treat the warning as an attack until you know otherwise.

“The same address” means the same server under any spelling: the CLI records and looks up host:port in one form, with the name in lower case, a trailing dot dropped, an IPv6 address in its short form and the port written out (7433 when you give none). DB.example:7433, db.example.:7433 and db.example are one entry. localhost and 127.0.0.1 are two, because the CLI never resolves a name.

The file is one server per line, host:port <key in hex>, and anything after the key is ignored, so you can note where a key came from. Lines starting with # are comments. The CLI only ever adds a line to the end of the file, so your comments and notes stay where you put them.

The CLI refuses to use the file if another user could change it: the file and ~/.hoarddb must be owned by you or by root, not writable by your group or by others, and the file must not be a symbolic link (the directory may be). The error names the command that fixes it, for example chmod 700 ~/.hoarddb or chmod 600 ~/.hoarddb/known_servers.

A script has nobody to ask, so a non-interactive session refuses:

The authenticity of server '127.0.0.1:7433' cannot be established.
ED25519 key fingerprint is SHA256:5612612c….
Refusing to trust it non-interactively: pre-populate ~/.hoarddb/known_servers or pass --insecure to bypass.

Compare the fingerprint with the one in the server’s start-up log, then answer y. For unattended use, put the server’s entry in ~/.hoarddb/known_servers ahead of time.

-insecure skips the check altogether. It prints a warning that says exactly what it gives up, and it is for a server you started yourself on the same machine. It is a setting of the client and never makes a server less strict. Do not put it in a script that talks to a server across a network.

Nodes trusting each other

Nodes use the same certificates between themselves. A node that is told to join a cluster must be given the fingerprints it should expect (HOARDDB_SEED_FINGERPRINTS) and refuses to start without them; a certificate that changes for a node it already knows is a hard failure, not a quiet re-trust. Configuration has the details.

What each side proves is not symmetric. A node that opens a connection checks the other node’s certificate against its pins. A node that accepts a connection checks only the cluster token: it asks for no certificate, so any process that holds the token is accepted as a member. On each accepted connection, the node records the member that the first request names as its sender, and refuses any later request on that connection that names a different member. It does not prove the connection’s first claim: a token holder can open a new connection in another member’s name. Checking that claim, by having the accepting node connect back to the member and verify its certificate, is planned for v1.1. Until then, keep the cluster token on the members only, and keep the internode port on a private network.

When a node copies a bucket in from another node, it sets its own limit on how much it holds. It refuses a copy announced as larger than half of its memory limit before it fetches any of it, and refuses any piece larger than the copy’s piece size when it arrives.

Your own certificate

You cannot substitute your own certificate in this version. A node started with HOARDDB_TLS_MODE=custom, or with HOARDDB_TLS_CERT, HOARDDB_TLS_KEY or HOARDDB_TLS_CA set, refuses to start: it exits with an error that names the setting, rather than quietly serving the certificate it generated. Unset the setting and use the generated certificate, which clients trust by its fingerprint. Operator-supplied certificates come after 1.0. There is no certificate authority check to configure; the fingerprint is the identity.

Users and authentication

Every connection logs in once, and what the connection may do is decided by the roles the user held at that moment.

• Passwords are hashed with Argon2id, with a server-side secret added, and the plain text is never stored.

• A new password must be at least 12 characters (HOARDDB_PASSWORD_MIN_LENGTH, 1 to 1024). create user and alter user <name> password refuse a shorter one and change nothing:

  the password for user "bob" is 8 characters; this server requires at least 12.
  Choose a longer password, or ask an administrator to change HOARDDB_PASSWORD_MIN_LENGTH; nothing was changed
  

  The CLI does the measuring, against the number the server it is connected to reports when it logs in. It has to: the CLI hashes the new password itself and sends the server only the hash, so the server never sees the password and cannot count its characters. Two things follow. A program that builds its own user-creation request (a hand-written client, or a driver that does not apply the check) is not measured by the server, so only give the user-administration role to people and tools you trust to choose long passwords. And set the same value on every node of a cluster, because the CLI uses the value of the node it is connected to. hoarddb-server -reset-password applies the same minimum itself, since it runs with the password in hand. The root password you give in HOARDDB_ROOT_PASSWORD for the first start is measured too: a first start that seeds the root user refuses a password shorter than HOARDDB_PASSWORD_MIN_LENGTH and names the setting that held it, HOARDDB_ROOT_PASSWORD or HOARDDB_ROOT_PASSWORD_FILE. A restart over an existing user store does not seed from it, so a stale value there is only warned about, as before.

• Give the CLI a password without putting it on the command line. Set HOARDDB_ROOT_PASSWORD (it stays out of ps), or give -password with no value and the CLI asks for it on a terminal. -password=VALUE puts the password on the command line, where ps and shell history can show it; do not use it on a shared host. -password-file and -password-stdin supply the new password for create user, not the login.

• Repeated failures slow an address down. After five failed logins in a row from one address, each further attempt is delayed, doubling from one second up to thirty. At HOARDDB_TCP_AUTH_MAX_FAILS failures in a row (default 20) the address is refused outright for sixty seconds, before any password is checked. A successful password login clears the count, and so does fifteen minutes without a failure. A login whose password is still being checked counts toward the limit until it settles, so many guesses sent at once cannot all be checked before the first ones fail: at most HOARDDB_TCP_AUTH_MAX_FAILS logins from one address are checked at once, and the rest wait up to five seconds for a place (up to four times that many may wait). A login refused because the wait ran out is answered RATE_LIMITED with “too many logins from this address are being checked at once”; it is not a failed login and opens no sixty-second window. With the audit log on, it is recorded as an auth.failure entry whose result reads “login refused: N being checked, M waiting; no password was checked”.

• Roles are per database, from reader (also accepted as read) up to root. A user with no grant on a database is refused there, and the refusal names the roles the user has and the command that would fix it.

• Disabling or dropping a user, or changing their password, ends that user’s open connections on every node: at once on the node that takes the change, and on each other node as soon as the change reaches it, normally within a second. Granting or revoking a role takes effect at the user’s next login; an open connection keeps the roles it logged in with. If you need a role change to bite immediately, change the password or disable the user.

Creating users, roles and the full list of what each can do: User management. The honest list of what this model does not do: Limitations.

Requiring a root password in production

A first start with no root password set asks for one on a terminal, and refuses when there is none. To refuse even at a terminal, so that every node’s root password comes from your secret store, turn the prompt off:

HOARDDB_REQUIRE_ROOT_PASSWORD=true HOARDDB_ROOT_PASSWORD='<choose one>' hoarddb-server

With HOARDDB_REQUIRE_ROOT_PASSWORD=true (YAML: auth.require_root_password: true), a first start that is not given HOARDDB_ROOT_PASSWORD or HOARDDB_ROOT_PASSWORD_FILE refuses to start and names the variable, rather than asking or inventing a password nobody chose. The Docker image sets it, so a container must always be given HOARDDB_ROOT_PASSWORD (docker run -e HOARDDB_ROOT_PASSWORD=... hoarddb). Restarts are unaffected: the root user already exists, and nothing is asked for.

Rotating the root password

Stop the server and run:

hoarddb-server -reset-password admin

It asks for the new password twice, without echo, and writes only its hash. The new password has to meet the minimum length above.

On a node with encryption at rest, run it with the same encryption settings as the server (the same configuration file or HOARDDB_ENCRYPTION_* variables): it opens the sealed users.json and the credential files under the node key, never creates one, and writes users.json back sealed. Without them it cannot open the sealed files and refuses, telling you to run it with the encryption settings the data directory was created with. Under systemd-creds the key is handed to a unit, so run it through systemd-run (Configuration).

On a cluster, run it on the node that is the auth master; on any other node it refuses and names the master. After an upgrade from a version that recorded the master by its host name, the master itself can be refused once: run hoarddb-server -reset-password admin -force on it. The next user change records the master’s cluster id and the refusal does not come back.

Upgrading from a version that kept root.password

Earlier versions saved a generated root password in plain text, in <data dir>/root.password. The first start after an upgrade converts it: the root user’s hash in users.json is kept (or, if the earlier first start never got as far as writing users.json, created from the file), the file is deleted, and the log says so once. If the file cannot be deleted, the server still starts and logs a warning with the path and the command to remove it by hand. Copies and backups of a data directory taken before the upgrade still hold the old file: delete them, or rotate the root password.

The cluster token

A cluster has a token that every node holds (<data dir>/cluster.token, mode 0600, sealed under the node key when encryption at rest is on). On a clustered node the keys that sign login tokens are derived from it, so whoever holds it can forge a login and can join a node to the cluster. Treat it like a signing key: keep it out of backups that other people can read, and delete it, and the node’s users.json, when you decommission a node. It does not expire, and this release cannot rotate it on a running cluster: rotate-token refuses on a cluster member, because the member’s users.json is sealed with keys derived from the token (see the CLI reference).

What a query can cost

A filter runs once for every document a query examines, so a hostile or careless filter could otherwise tie up a node. A query is limited in three ways: the shape of its filter (the size of a $regex pattern, the values in an $in, how deep $and and $or nest, the clauses in all), the regex work done on any one document, and the bytes a result may hold. A query over a limit is refused with INVALID_ARGUMENT or RESULT_TOO_LARGE, naming the limit and what to change, and nothing is partly done. The numbers are on Limits. The filter and regex limits are fixed; the result limit can be set from 1 MiB to 24 MiB with HOARDDB_QUERY_MAX_RESULT_BYTES.

Driver connection strings carry a password, so a driver that refuses one (an unencoded /, ?, # or extra @ in the password, a bad percent-escape) names the problem without repeating any part of the user info. Percent-encode those characters in the password and the string is accepted.

Encryption at rest

Encryption at rest is built for a whole node, and it is off by default.

When it is off, the data directory is plaintext on disk, and only the users.json file is sealed, which is not encryption at rest for your data. When you turn it on, every bucket on that node is encrypted under a key that you provide and that is not kept in the data directory:

encryption:
  keystore: keyfile
  key_file: /etc/hoarddb/kek
  mode: require

What to know before you rely on it:

• It belongs to the node. One key opens every bucket on the node; there is no per-bucket key. Per-database and per-bucket keys are planned and are not built.
• It is decided when the data directory is first started. You cannot switch a node between encrypted and plaintext afterwards; you dump, and restore into a fresh directory started the way you want.
• An encrypted node seals blob payloads too. Blob content of 64 KiB and over is stored in files that are encrypted and authenticated in pieces, under keys kept in keys/blobs.wrapped. Back that file up with the data directory: the node-master escrow does not cover it, and without it sealed blob files cannot be read.
• A dump is plaintext whatever the node’s mode. Protect the file the way you protect the data.
• Losing the key is unrecoverable data loss. Back the key up apart from the data directory.
• It seals the node’s credentials too. The cluster token, the signing key of a standalone node (auth.token.key) and the TLS private key (keys/server.key) are sealed under the node key, so a copy of the data directory alone cannot sign a login, join the cluster or impersonate the node. A node that already ran with encryption on and finds these files in plaintext seals them in place on its next start, keeping the same token, keys and certificate fingerprint. A sealed file belongs to its node: copying cluster.token to another node does not work, so give a new node the token that get-token prints, in HOARDDB_CLUSTER_TOKEN. get-token and rotate-token need the server’s encryption settings (the same configuration file or HOARDDB_ENCRYPTION_* variables) to open the file. This is one-way: do not start a sealed data directory with a build from before this release, which cannot read the sealed files.
• It protects a copied disk or data directory. It does not protect against someone who can run code as the server’s user, who can ask the same keystore for the key. It does not replace disk encryption either, which also covers swap and freed blocks.

How to set it up, the three places the key can come from, export and import of the node key, and what a cluster does with an unencrypted peer: Configuration.

Default network exposure

A node you start with no arguments listens on this machine only. Nothing on the network can reach it until you say so.

The start-up log says where the node listens, and when it is on loopback it names the settings that open it up.

The start-up log shows the node’s address in the ring, which is HOARDDB_LISTEN. Its default, 127.0.0.1:4433 for HOARDDB_LISTEN, is the name other nodes use for it, not a port the node opens.

Opening a node to other machines

• Applications on other machines. Set the client address. 0.0.0.0 means every interface; use one interface’s address to be narrower:

  HOARDDB_TCP_LISTEN=0.0.0.0:7433 hoarddb-server
  

• A cluster across machines. Set HOARDDB_LISTEN to the address the other nodes reach this node on, and list that same address in HOARDDB_SEED_NODES on every node:

  HOARDDB_LISTEN=192.0.2.1:4433 HOARDDB_SEED_NODES=192.0.2.1:4433,192.0.2.2:4433,192.0.2.3:4433 ...
  

  A node whose seeds are on another machine but which is still on the loopback default is refused at start, with the setting to change in the message: it would tell its peers to reach it on 127.0.0.1, which only it can do. A cluster whose nodes all run on one machine (every seed on 127.0.0.1) needs no change.

Two things follow:

• On a host that is reachable from outside your network, bind to the interface your applications use rather than 0.0.0.0, or put a firewall in front of the client port and the node-to-node port. Peers must hold the cluster token to use the node-to-node port, but the port accepts a connection from anywhere it is reachable, so keep it to the nodes of your cluster.
• The client port is protected by login and TLS, not by obscurity. Anyone who can reach it can try passwords, which the failure delay slows down but does not stop. Choose long root and user passwords.

Metrics exposure

When you enable metrics, /metrics is served over plain HTTP with no authentication. It reports counts, durations and capacity readings for the node and carries no document content, and it keeps no series per bucket. It is still information about your system. It listens on the loopback interface by default, so a scraper on the same machine needs nothing more; to let a monitoring host reach it, bind to the address of your monitoring network (not 0.0.0.0 on a host that faces the internet):

HOARDDB_METRICS_ENABLED=true HOARDDB_METRICS_LISTEN=192.0.2.5:9090 hoarddb-server

With metrics off, the node’s capacity warnings still go to its log and to hoardDB status; see Is this node full?. The profiling endpoint is also unauthenticated and plain; leave it off on a server.

What is not protected

A short list, so that you do not assume otherwise:

• A connection’s roles are set when it logs in. A role change reaches an open connection only at its next login; a drop, a disable or a new password ends it.
• A node cut off from the cluster master cannot learn of a drop, a disable or a new password. It refuses that user’s logins once it is more than auth_config_max_staleness_seconds behind, but a connection already open on it lasts until the change reaches it.
• There is no way to revoke one login token without affecting that user’s others. Revocation is per user.
• Anyone who holds the cluster token can act as a cluster member and can forge logins (above).
• Anyone who can read the data directory can read a plaintext node’s data and its credentials: the cluster token, the signing key and the TLS private key are plaintext files there. The only file sealed on a node without encryption at rest is users.json, and its key sits in the same directory. If a copy of such a directory is lost, rotate the root password and the cluster token and give the node a new TLS identity.
• Plaintext credentials sealed on an upgrade leave their old disk blocks behind. The node replaces each file with its sealed form; the freed blocks are not overwritten. Only disk encryption covers freed blocks.
• A dump file, and the key file you export for escrow, are not encrypted.
• The CLI’s -insecure flag removes the server-identity check for that session.

Reporting a vulnerability

Please report a security problem privately, not in a public issue or a discussion.

Send your report by email to admin@hoarddb.com. Include:

• what you found and which version or commit it affects (hoarddb-server -v);
• how to reproduce it, as short as you can make it;
• what an attacker gains, if you know.

We acknowledge a report, agree a disclosure date with you, and fix the problem before it is described in public. A security fix is announced in the changelog after it ships, and the people who reported it are credited if they want to be. Please do not test against systems you do not own.

Source: docs/user/security.md in the repository.