Data isolation with buckets

Buckets let you isolate ledgers within the same database server. This is useful for separating data by client, application, or environment.

Understanding PostgreSQL Schemas#

At a technical level, a bucket in Formance is directly mapped to a PostgreSQL schema. In PostgreSQL, a schema is a namespace that groups together database objects (tables, views, indexes, functions):

• Without schemas: All tables would live in one big global namespace, which could quickly become messy
• With schemas: You can organize and separate data logically, avoid name collisions, and apply different permissions

Two tables with the same name can exist in different schemas, as they live under different namespaces.

How Formance Uses Schemas#

When you create a ledger, Formance stores its data inside a PostgreSQL schema (the bucket). This has several implications:

System Schema#

Formance uses a special internal schema called _system to track metadata about your ledgers and buckets.

Practical Considerations#

• Data Isolation: Buckets are a good fit if you need strong separation (e.g., per-tenant ledgers in a multi-tenant system)
• Performance Management: For high-volume workloads, spreading ledgers across multiple buckets can reduce contention and keep queries fast
• Simplicity: If isolation isn't critical, sticking with the _default bucket keeps things straightforward

Creating a bucket#

Buckets are automatically created when you create a new ledger. By default, if the bucket is not specified, the ledger is created in the _default bucket.

Create a ledger in a specific bucket:

fctl ledger create testing

Response:

{
  "data": {
    "bucket": "bucket0",
    "metadata": {},
    "features": {
      "ACCOUNT_METADATA_HISTORY": "SYNC",
      "HASH_LOGS": "SYNC",
      "MOVES_HISTORY": "ON",
      "MOVES_HISTORY_POST_COMMIT_EFFECTIVE_VOLUMES": "SYNC",
      "TRANSACTION_METADATA_HISTORY": "SYNC"
    },
    "id": 2,
    "name": "testing",
    "addedAt": "2024-10-03T08:27:11.540373Z"
  }
}

Features#

Each ledger can be configured with a set of features at creation time. Features let you tune ledger behavior for different use cases (high write throughput, full audit trail, and so on).

When you create a ledger without specifying features, all features are enabled with their default values. You only need to specify features you want to override; unspecified features receive their defaults.

Features summary#

MOVES_HISTORY#

Values: ON | OFF
Default: ON

When enabled, the ledger tracks every individual fund movement for each account/asset pair in the moves table. Each move record includes account address, asset, amount, source/destination flag, insertion date, effective date, and post-commit volumes.

When enabled (ON): Full balance history at any point in time, historical balance queries, point-in-time support. Required for effective volumes calculation.

When disabled (OFF): Only current balances are available; better write performance and lower storage.

MOVES_HISTORY_POST_COMMIT_EFFECTIVE_VOLUMES#

Values: SYNC | DISABLED
Default: SYNC

Maintains the post_commit_effective_volumes column in moves, which tracks volumes ordered by effective date (not insertion date). When you create backdated transactions, effective volumes for later moves are updated automatically.

When enabled (SYNC): Transaction responses include postCommitEffectiveVolumes. Accurate historical balance queries and correct handling of backdated transactions.

When disabled (DISABLED): The postCommitEffectiveVolumes property is not present on transaction responses. Better write performance when transactions are always inserted in chronological order.

HASH_LOGS#

Values: SYNC | ASYNC | DISABLED
Default: SYNC

Provides cryptographic integrity verification for the ledger log chain. Each log entry is hashed (SHA-256) and chained to the previous log's hash.

• SYNC: Hashes are computed synchronously during each log insert. Hash is immediately available; only one log can be inserted at a time per ledger (advisory lock). Best for strong integrity guarantees with lower throughput.
• ASYNC: Logs are inserted without a hash; a background worker computes hashes in batches (blocks). No locking on insert, so higher write throughput. Requires the ledger worker process and configuration (e.g. --worker-async-block-hasher-max-block-size, --worker-async-block-hasher-schedule).
• DISABLED: No hashing. Maximum write performance; no integrity verification.

ACCOUNT_METADATA_HISTORY#

Values: SYNC | DISABLED
Default: SYNC

When enabled, every change to account metadata is historized with revision tracking. You can query account metadata at any point in time using the pit parameter. When disabled, only current metadata is stored and PIT queries return current values.

TRANSACTION_METADATA_HISTORY#

Values: SYNC | DISABLED
Default: SYNC

When enabled, every change to transaction metadata is historized. You can query transaction metadata at any point in time using the pit parameter. When disabled, only current metadata is stored and PIT queries return current values.

Setting features at ledger creation#

Use POST /v2/{ledger} and pass features in the request body. Only specify features you want to override.

Request body:

{
  "bucket": "optional-bucket-name",
  "metadata": {},
  "features": {
    "FEATURE_NAME": "VALUE"
  }
}

Create ledger with all defaults:

curl -X POST $FORMANCE_API_URL/api/ledger/v2/my-ledger

Create high-throughput ledger (minimal features):

curl -X POST $FORMANCE_API_URL/api/ledger/v2/high-throughput \
  -H "Content-Type: application/json" \
  -d '{
    "features": {
      "MOVES_HISTORY": "OFF",
      "MOVES_HISTORY_POST_COMMIT_EFFECTIVE_VOLUMES": "DISABLED",
      "HASH_LOGS": "DISABLED",
      "ACCOUNT_METADATA_HISTORY": "DISABLED",
      "TRANSACTION_METADATA_HISTORY": "DISABLED"
    }
  }'

Create ledger with async hashing:

curl -X POST $FORMANCE_API_URL/api/ledger/v2/async-hashing \
  -H "Content-Type: application/json" \
  -d '{
    "features": {
      "HASH_LOGS": "ASYNC"
    }
  }'

Override specific features only (others get defaults):

curl -X POST $FORMANCE_API_URL/api/ledger/v2/custom \
  -H "Content-Type: application/json" \
  -d '{
    "features": {
      "HASH_LOGS": "DISABLED",
      "MOVES_HISTORY_POST_COMMIT_EFFECTIVE_VOLUMES": "DISABLED"
    }
  }'

Feature sets#

Default (full audit): All features enabled. Use when you need full audit capabilities.

Minimal (performance): All features disabled or minimized. Use for high-throughput scenarios where an audit trail is not required.

Async hashing (balanced): HASH_LOGS: "ASYNC" with other features at defaults. Use for higher throughput with eventual integrity verification (requires the ledger worker).

Feature impact summary#

Deleting and restoring buckets#

You can delete buckets and restore them within a retention period if needed.

Deleting a bucket#

curl -X DELETE $FORMANCE_API_URL/api/ledger/v2/_/buckets/my-bucket

When you delete a bucket:

• All ledgers in the bucket become hidden from normal queries
• The data is preserved for 30 days (configurable)
• You can restore the bucket during this period

Restoring a bucket#

If you deleted a bucket by mistake, restore it before the retention period ends:

curl -X POST $FORMANCE_API_URL/api/ledger/v2/_/buckets/my-bucket/restore

All ledgers in the bucket become visible again.

Listing deleted ledgers#

To see deleted ledgers, add includeDeleted=true to your query:

curl -X GET $FORMANCE_API_URL/api/ledger/v2?includeDeleted=true

Deleted ledgers include a deletedAt timestamp in the response:

{
  "cursor": {
    "data": [
      {
        "name": "my-ledger",
        "bucket": "my-bucket",
        "addedAt": "2025-01-10T10:00:00Z",
        "deletedAt": "2025-01-14T15:30:00Z"
      }
    ]
  }
}

Configuring retention#

By default, deleted buckets are permanently removed after 30 days. To change this, configure the worker:

ledger worker \
  --worker-bucket-cleanup-retention-period=2160h \
  --worker-bucket-cleanup-schedule="0 0 */6 * * *"