Pre-1.0 hoardDB is pre-1.0. Expect breaking changes.

Limits

Every size and count that hoardDB enforces, with its default, how to change it and what you see when you reach it. This page is the numbers; Limitations is the list of things hoardDB does not do, and explains the reasoning behind several of these.

A limit that an operator can change is a setting of one node. A write that reaches a replica is not checked against the replica’s own value, so keep these settings the same on every node of a cluster.

How big can one thing be

Why a filter has limits. A filter runs once for every document a scan, search, count or distinct examines, so a filter that looks small can cost a lot of work in a large bucket. The limits keep that work in proportion to the size of the filter. A filter that is too big on its own (a pattern, a list or a nesting past its limit) is refused before any document is read. A $regex is also compiled once per query, not once per document, and what it costs a document is its steps times the bytes of the field it scans: a long field can make an ordinary pattern too costly, so that refusal comes at the first document whose field is too long, not before the query starts. A case-sensitive pattern that is only a literal anchored at the start (^foo, or ^foo$) is a prefix comparison and costs one step per byte, so it runs over fields of up to about 1.4 MB. A literal that is not anchored at the start (foo, foo$) is charged like any other pattern: Go checks each place it might occur, which on repetitive text costs about its length times the field’s. An email shape or a list of words in (?i) form is a few dozen steps and runs over fields of tens of thousands of bytes, (?i)foo is charged as a program and not as a literal, a pattern of 100 steps runs over up to 15,000 bytes, and one of 512 steps, up to 2,900. Bytes are counted, not characters. Each refusal names the limit and what to do instead, for example a $or of 200 filters on one field becomes an $in, or two queries; for a long field, use a shorter pattern, or narrow the query with an indexed field first.

Why a name has a limit. A node keeps one record per bucket, under a key made of the database name and the bucket name, and its store refuses a key over 65,000 bytes. Replication keeps further records under the same names. 255 bytes each, counted in UTF-8 bytes (so fewer characters if a name uses non-Latin letters), is the limit you already know from file names, and it leaves those records plenty of room. If you want to keep a long identifier, store it as a field of a document and name the bucket by a short id.

Why the document and blob limits are not 32 MiB. Everything a client sends, and the copy a node sends to a replica, travels in one frame of at most 32 MiB, and the frame also carries the key, the bucket name and the request’s envelope. The ceilings leave room for that: a document tops out at 30 MiB, and a blob at 32 MiB less 64 KiB. A value outside the range you can set stops the server from starting, with a message naming the setting and the range; it is never quietly lowered.

What the refusals say. They state the size you sent, the limit this server is running with, the setting that changes it and what to do instead:

Blob "videos/intro.mp4" is 5.0 GiB; this server's limit is 16 MiB
(storage.max_blob_bytes, maximum 31.9 MiB). For larger files, keep the file on
your own storage and store a pointer document — see "Large file stores" in the
docs.

<label> is <size>; this server's limit is <limit> per document
(storage.max_document_bytes, maximum 30 MiB).

The CLI refuses a blob larger than the 31.9 MiB ceiling before it sends any of it, since no server would accept it.

What to do instead of raising a limit.

• A list that keeps growing belongs in a nested store field, whose elements are stored separately and do not count towards the parent document.
• A file that is large belongs on storage you run, with a pointer document in hoardDB. Large file stores is the pattern.
• A bulk load that is refused for its size is a bulk load that should be split: hoarddb ingest halves a refused batch and retries, so a file of any size loads. See Load a file.

How many buckets

A bucket is a key range in the one store a node keeps for all its buckets. It costs no file, no file descriptor, no memory mapping and no goroutine of its own, and it adds one small entry to the node’s in-memory catalog. Adding a bucket is adding keys, not opening anything.

The figure to plan around is thousands of buckets per node. hoardDB keeps data with what it describes, so a node can hold many small buckets, and it does not publish a larger number until it has measured one on this layout. Higher counts may work on your hardware; they are not a promise. What a bucket costs, and how to see how many a node serves, is in Is this node full?.

hoardDB keeps no metric per bucket. A series per bucket is not viable at these counts, so the metrics are totals for the node, and per-bucket detail is a command you run (describe bucket, stats).

The memtable

The memtable is the write buffer in front of the node’s one store. It is a per-node setting: every bucket shares it, so it does not grow with the number of buckets.

A new node picks its size once, from the memory it is allowed (its container or cgroup limit, or the machine’s RAM), and keeps it on every restart:

The automatic choice never goes above 128 MiB. The last column is a prediction for one client loading in batches, about 200 MB plus 8.2 MB for every MiB of memtable; many concurrent writers need more.

To choose it yourself, set storage.memtable_mb (HOARDDB_MEMTABLE_MB) to a value from 32 to 1024 MiB. Sizes of 256 MiB and above are only ever set this way. A value outside the range stops the server at start:

storage.memtable_mb is 8, below the minimum of 32. The store's largest atomic
batch is 15% of the memtable, and under 32 MiB it drops below 5 MB, which the
ingest batch budget and batch splitting assume. Set storage.memtable_mb (or
HOARDDB_MEMTABLE_MB) to a value from 32 to 1024, or remove it to run at the size
this node's store chose at its first start

A bigger memtable stalls writes less and holds up to three of them in memory, each needing about 1.3 times its size, so 1024 MiB is already about 3.9 GB. It is fixed at start. A node that crashed at a larger size cannot be opened at a smaller one until it has been started once at the old size and stopped cleanly. Configuration shows how to read the size a node chose, and why.

Connections and requests

The client budget (HOARDDB_MAX_CONNS) and the node-to-node budget are separate, so a flood of clients cannot starve replication. A connection over a cap is answered with OVERLOADED, naming the cap and its setting, and closed. While 64 such refusals are already in progress on a port, a further refused connection is closed without the message: retry after a short wait. A connection carries one request at a time unless the client reads ahead within the window above; a client that wants concurrency opens another connection. A query past the concurrency limit waits; it is not refused.

A login still being checked holds one of the HOARDDB_TCP_AUTH_MAX_FAILS places for its address until its password is checked, because it may yet fail. A login that finds every place held this way waits for one to come free and is then let in, so a burst of correct logins larger than the limit (an application starting a big connection pool, a load generator) succeeds, a little slower. A login that waits more than 5 seconds, or arrives while 80 others from its address already wait, is answered RATE_LIMITED with “too many logins from this address are being checked at once”, and its connection is closed. Its password was not checked and it does not count as a failed login: reconnect and retry, or raise HOARDDB_TCP_AUTH_MAX_FAILS. Raising it also lets more wrong passwords from one address be tried before the address is refused.

A value outside a range with a stated clamp is brought to the nearest end of it when the server loads its configuration. The sizes in the first table are the other kind: outside their range, the server refuses to start.

Checking a limit against your data

Before you rely on a limit, try it at its edge on your own data. The sizes and counts above are what the build enforces, and they were read from its configuration, not measured on your workload: your hardware, your kernel’s file-descriptor and memory-mapping limits, and the shape of your documents decide where a node becomes slow before any of them is reached. Is this node full? lists the resources that bind first and what to do about each.

Source: docs/user/limits.md in the repository.