config: isolated mode for an instance #4632

Totktonada · 2024-11-08T14:28:21Z

Related dev. issue(s): tarantool/tarantool#10796

Product: Tarantool
Since: 3.3.0
Audience/target: application developers, administrators
Root document:

https://www.tarantool.io/en/doc/latest/reference/configuration/configuration_reference/
(possibly somewhere in) https://www.tarantool.io/en/doc/latest/admin/replication/
SME: @ Totktonada

Details

The option is to temporary isolate an instance to perform replicaset repairing activities: debug a problem on the isolated instance (without affecting the non-isolated part) or extract some data from the isolated instance and apply them on the non-isolated part of the replicaset.

groups:
  g:
    replicasets:
      r:
        instances:
          i-001: {}
          i-002: {}
          i-003: {}
          i-004:
            isolated: true

In this mode the instance doesn't perform processing of iproto requests, doesn't perform background data modifications and refuses to accept updates from other replicaset members. Also, other replicaset members don't replicate data from the isolated instance.

Technically speaking, when the option is enabled, the following actions are performed:

iproto stops listening for new connections.
iproto drops all the current connections.
The instance goes to read-only mode.
The instance is disconnected from all the replication upstreams.
Other replicaset members exclude the isolated instance from the replication upstreams.

One new startup constraint is added in the isolated mode:

The instance can't be bootstrapped (a local snapshot is required to start).

Definition of done

Describe the new option in the configuration reference
Verify whether anything else should be updated in the documentation

This commit just adds the option into the cluster configuration schema. Next commits will add the relevant logic. Part of tarantool#10796 NO_DOC=tarantool/doc#4632

In the isolated mode an instance refuses to fetch any data from other replicaset members. An attempt to bootstrap a database on such an instance would produce its own database (for example, it assigns its own replicaset UUID). Unlikely a user wants to place two instances into the same replicaset in the configuration and get two different databases. Part of tarantool#10776 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

The isolated instance shouldn't produce new transactions, including ones from a background fiber started by an application or a role. So, the isolated instance forcefully goes to the read-only mode disregarding any other configuration options. Part of tarantool#10776 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

An isolated instance shouldn't process user traffic. This commit adds a logic to the isolated mode that stop listening for new connections and drop existing connections in iproto. Part of tarantool#10776 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

The isolated mode can be enabled to stop data modifications on the given instance, including ones from the replication. It may help to debug a problem or extract some needed data. Part of tarantool#10776 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

The isolated instance goes to RO and drops iproto connections -- it should effectively stop replication from it. There is no sense to retry an attempt to connect to it and flood logs with errors. Let's just remove the isolated instance from the upstreams list. Part of tarantool#10776 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

This way it is harder to overlook that the instance is isolated. It is likely important information for a person who performs administration tasks. Closes tarantool#10776 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

This commit just adds the option into the cluster configuration schema. Next commits will add the relevant logic. Part of tarantool#10796 NO_DOC=tarantool/doc#4632

This commit just adds the option into the cluster configuration schema. Next commits will add the relevant logic. Part of #10796 NO_DOC=tarantool/doc#4632

In the isolated mode an instance refuses to fetch any data from other replicaset members. An attempt to bootstrap a database on such an instance would produce its own database (for example, it assigns its own replicaset UUID). Unlikely a user wants to place two instances into the same replicaset in the configuration and get two different databases. Part of tarantool#10776 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

In the isolated mode an instance refuses to fetch any data from other replicaset members. An attempt to bootstrap a database on such an instance would produce its own database (for example, it assigns its own replicaset UUID). Unlikely a user wants to place two instances into the same replicaset in the configuration and get two different databases. Part of tarantool#10796 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

In the isolated mode an instance refuses to fetch any data from other replicaset members. An attempt to bootstrap a database on such an instance would produce its own database (for example, it assigns its own replicaset UUID). Unlikely a user wants to place two instances into the same replicaset in the configuration and get two different databases. Part of #10796 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

The isolated instance shouldn't produce new transactions, including ones from a background fiber started by an application or a role. So, the isolated instance forcefully goes to the read-only mode disregarding any other configuration options. Part of tarantool#10796 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

The isolated mode can be enabled to stop data modifications on the given instance, including ones from the replication. It may help to debug a problem or extract some needed data. Part of tarantool#10796 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

The isolated instance goes to RO and drops iproto connections; thus, the replication from it is effectively stopped. There is no sense to retry an attempt to connect to it and flood logs with errors. Let's just remove the isolated instance from the upstreams list. Part of tarantool#10796 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

An isolated instance shouldn't process user traffic. This commit adds a logic to the isolated mode that stop listening for new connections and drop existing connections in iproto. Part of tarantool#10796 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

The isolated instance shouldn't produce new transactions, including ones from a background fiber started by an application or a role. So, the isolated instance forcefully goes to the read-only mode disregarding any other configuration options. Part of tarantool#10796 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

The isolated instance shouldn't produce new transactions, including ones from a background fiber started by an application or a role. So, the isolated instance forcefully goes to the read-only mode disregarding any other configuration options. Part of #10796 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

An isolated instance shouldn't process user traffic. This commit adds a logic to the isolated mode that stop listening for new connections and drop existing connections in iproto. Part of tarantool#10796 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

An isolated instance shouldn't process user traffic. This commit adds a logic to the isolated mode that stop listening for new connections and drop existing connections in iproto. Part of #10796 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

The isolated mode can be enabled to stop data modifications on the given instance, including ones from the replication. It may help to debug a problem or extract some needed data. Part of tarantool#10796 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

The isolated instance goes to RO and drops iproto connections; thus, the replication from it is effectively stopped. There is no sense to retry an attempt to connect to it and flood logs with errors. Let's just remove the isolated instance from the upstreams list. Part of tarantool#10796 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

The isolated mode can be enabled to stop data modifications on the given instance, including ones from the replication. It may help to debug a problem or extract some needed data. Part of #10796 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

The isolated instance goes to RO and drops iproto connections; thus, the replication from it is effectively stopped. There is no sense to retry an attempt to connect to it and flood logs with errors. Let's just remove the isolated instance from the upstreams list. Part of #10796 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

This way it is harder to overlook that the instance is isolated. It is likely important information for a person who performs administration tasks. Closes tarantool#10796 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

This way it is harder to overlook that the instance is isolated. It is likely important information for a person who performs administration tasks. Closes #10796 NO_DOC=tarantool/doc#4632 NO_CHANGELOG=added together with the configuration option

The ```isolated:true``` option moves the instance to RO mode In this mode, various debugging & maintanance operations can be performed Fixes #4632

Totktonada added the server [area] Task relates to Tarantool's server (core) functionality label Nov 8, 2024

Totktonada mentioned this issue Nov 8, 2024

config: add isolated mode for an instance tarantool/tarantool#10796

Closed

7 tasks

Totktonada changed the title ~~[WIP] config: isolated mode for an instance~~ config: isolated mode for an instance Nov 24, 2024

p7nov added 3.3 config labels Dec 3, 2024

p7nov self-assigned this Dec 3, 2024

p7nov removed their assignment Jan 27, 2025

AArdeev self-assigned this May 26, 2025

AArdeev added a commit that referenced this issue May 27, 2025

Adds isolated instance mode description

ea27552

The ```isolated:true``` option moves the instance to RO mode In this mode, various debugging & maintanance operations can be performed Fixes #4632

AArdeev linked a pull request May 27, 2025 that will close this issue

Adds isolated instance mode description #5136

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

config: isolated mode for an instance #4632

config: isolated mode for an instance #4632

Totktonada commented Nov 8, 2024 •

edited by xuniq

Loading

config: isolated mode for an instance #4632

config: isolated mode for an instance #4632

Comments

Totktonada commented Nov 8, 2024 • edited by xuniq Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details

Definition of done

Totktonada commented Nov 8, 2024 •

edited by xuniq

Loading