Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[doc][2.25.1] Add flags for Advisory locks #25659

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
2 changes: 1 addition & 1 deletion docs/content/preview/architecture/transactions/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ menu:
type: indexpage
---

Transactions and strong consistency are a fundamental requirement for any RDBMS. YugabyteDB's distributed transaction architecture supports fully distributed [ACID](../key-concepts#acid) transactions across rows, multiple tablets, and multiple nodes at any scale and is inspired by [Google Spanner](https://research.google.com/archive/spanner-osdi2012.pdf). Transactions can span across tables in DocDB.
Transactions and strong consistency are a fundamental requirement for any RDBMS. YugabyteDB's distributed transaction architecture supports fully distributed [ACID](../key-concepts#acid) transactions across rows, multiple tablets, and multiple nodes at any scale, and is inspired by [Google Spanner](https://research.google.com/archive/spanner-osdi2012.pdf). Transactions can span across tables in DocDB.

## Fundamentals

Expand Down
92 changes: 84 additions & 8 deletions docs/content/preview/explore/transactions/explicit-locking.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Explicit locking
headerTitle: Explicit locking
linkTitle: Explicit locking
description: Explicit locking in YugabyteDB.
headcontent: Row locking in YugabyteDB
headcontent: Row locking and advisory locks in YugabyteDB
menu:
preview:
identifier: explore-transactions-explicit-locking-1-ysql
Expand All @@ -23,16 +23,18 @@ type: docs

</ul>

## Row-level locks

YugabyteDB's YSQL supports explicit row-level locking, similar to PostgreSQL. Explicit row-locks ensure that two transactions can never hold conflicting locks on the same row. When two transactions try to acquire conflicting lock modes, the semantics are dictated by YugabyteDB's [concurrency control](../../../architecture/transactions/concurrency-control/) policies.

The following types of row locks are supported:

* `FOR UPDATE`
* `FOR NO KEY UPDATE`
* `FOR SHARE`
* `FOR KEY SHARE`
- FOR UPDATE
- FOR NO KEY UPDATE
- FOR SHARE
- FOR KEY SHARE

The following example uses the `FOR UPDATE` row lock with the [fail-on-conflict](../../../architecture/transactions/concurrency-control/#fail-on-conflict) concurrency control policy. First, a row is selected for update, thereby locking it, and subsequently updated. A concurrent transaction should not be able to abort this transaction by updating the value of that row after the row is locked.
The following example uses the FOR UPDATE row lock with the [fail-on-conflict](../../../architecture/transactions/concurrency-control/#fail-on-conflict) concurrency control policy. First, a row is selected for update, thereby locking it, and subsequently updated. A concurrent transaction should not be able to abort this transaction by updating the value of that row after the row is locked.

{{% explore-setup-single %}}

Expand All @@ -45,7 +47,7 @@ yugabyte=# INSERT INTO t VALUES ('k1', 'v1');

Next, connect to the universe using two independent ysqlsh instances. You can connect both session ysqlsh instances to the same server or to different servers.

Begin a transaction in the first session and perform a `SELECT FOR UPDATE` on the row in the table `t`. This locks the row for an update as a part of a transaction that has a very high priority (that is, in the `high priority bucket`, as explained in [Transaction priorities](../../../architecture/transactions/transaction-priorities/)):
Begin a transaction in the first session and perform a SELECT FOR UPDATE on the row in the table `t`. This locks the row for an update as a part of a transaction that has a very high priority (that is, in the `high priority bucket`, as explained in [Transaction priorities](../../../architecture/transactions/transaction-priorities/)):

```sql
yugabyte=# BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
Expand Down Expand Up @@ -76,7 +78,7 @@ yugabyte=# UPDATE t SET v='v1.1' WHERE k='k1';
ERROR: All transparent retries exhausted. Operation failed. Try again: bb3aace4-5de2-41f9-981e-d9ca06671419 Conflicts with higher priority transaction: d4dadbf8-ca81-4bbd-b68c-067023f8ee6b
```

This operation fails because it conflicts with the row-level lock and as per `Fail-on-Conflict` concurrency control policy, the transaction aborts itself because it has a lower priority.
This operation fails because it conflicts with the row-level lock and as per Fail-on-Conflict concurrency control policy, the transaction aborts itself because it has a lower priority.

Note that the error message appears after all [best-effort statement retries](../../../architecture/transactions/concurrency-control/#best-effort-internal-retries-for-first-statement-in-a-transaction) have been exhausted.

Expand All @@ -99,3 +101,77 @@ COMMIT
```

This should succeed.

## Advisory locks

{{<tags/feature/tp>}}Advisory locks are available in v2.25.1 and later.

YSQL also supports advisory locks, where the application manages concurrent access to resources through a cooperative locking mechanism. Advisory locks can be less resource-intensive than table or row locks for certain use cases because they don't involve scanning tables or indexes for lock conflicts. They are session-specific and managed by the client application.

In PostgreSQL, if an advisory lock is taken on one session, all sessions should be able to see the advisory locks acquired by any other session. Similarly, in YugabyteDB, if an advisory lock is acquired on one session, all the sessions should be able to see the advisory locks regardless of the node the session is connected to. This is achieved via the pg_advisory_locks system table, which is dedicated to hosting advisory locks. All advisory lock requests are stored in this system table.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there changes to pg_locks view? eg, locktype can now also be advisory?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pg_locks changes are planned, but its not yet in the master builds.

To enable the use of advisory locks in a cluster, you must set the [Advisory lock flags](../../../reference/configuration/yb-tserver/#advisory-locks-flags).

### Using advisory locks

Advisory locks in YugabyteDB are semantically identical to PostgreSQL, and are managed using the same the functions. Refer to [Advisory lock functions](https://www.postgresql.org/docs/15/functions-admin.html#FUNCTIONS-ADVISORY-LOCKS) in the PostgreSQL documentation.

You can acquire an advisory lock in the following ways:

- Session level

Once acquired at session level, the advisory lock is held until it is explicitly released or the session ends.

Unlike standard lock requests, session-level advisory lock requests do not honor transaction semantics: a lock acquired during a transaction that is later rolled back will still be held following the rollback, and likewise an unlock is effective even if the calling transaction fails later. A lock can be acquired multiple times by its owning process; for each completed lock request there must be a corresponding unlock request before the lock is actually released.

```sql
SELECT pg_advisory_lock(10);
```

- Transaction level

Transaction-level lock requests, on the other hand, behave more like regular row-level lock requests: they are automatically released at the end of the transaction, and there is no explicit unlock operation. This behavior is often more convenient than the session-level behavior for short-term usage of an advisory lock.

```sql
SELECT pg_advisory_xact_lock(10);
```

Advisory locks can also be exclusive or shared:

- Exclusive Lock

Only one session/transaction can hold the lock at a time. Other sessions/transactions can't acquire the lock until the lock is released.

```sql
select pg_advisory_lock(10);
select pg_advisory_xact_lock(10);
```

- Shared Lock

Multiple sessions/transactions can hold the lock simultaneously. However, no session/transaction can acquire an exclusive lock while shared locks are held.

```sql
select pg_advisory_lock_shared(10);
select pg_advisory_xact_lock_shared(10);
```

Finally, advisory locks can be blocking or non-blocking:

- Blocking lock

The process trying to acquire the lock waits until the lock is acquired.

```sql
select pg_advisory_lock(10);
select pg_advisory_xact_lock(10);
```

- Non-blocking lock

The process immediately returns a boolean value stating if the lock is acquired or not.

```sql
select pg_try_advisory_lock(10);
select pg_try_advisory_xact_lock(10);
```
Original file line number Diff line number Diff line change
Expand Up @@ -210,9 +210,9 @@ ERROR: partman is not a valid partitioning type for pg_partman

The pg_partman `create_parent()` function requires an access exclusive lock on the parent table to create new child partitions. Currently, access exclusive locks are not supported in YugabyteDB, and are disabled in this function.

### ADVISORY LOCKS
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if you enable pg_partman but don't set the advisory lock flags for the cluster

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then the advisory locks calls will fail with the message to enable the preview feature.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like there is dev work to get pg_partman to use advisory_locks. Will undo this change for now.

### Advisory locks

Advisory locks, used in some pg_partman functions to create, drop/delete partitioned tables, are not supported in YugabyteDB. Attempts to acquire these locks are disabled.
Advisory locks, used in some pg_partman functions to create, drop/delete partitioned tables, are not yet supported in YugabyteDB. Attempts to acquire these locks are disabled.

### Background worker process

Expand Down
14 changes: 14 additions & 0 deletions docs/content/preview/reference/configuration/yb-master.md
Original file line number Diff line number Diff line change
Expand Up @@ -1033,6 +1033,20 @@ expensive when the number of yb-tservers, or the number of databases goes up.

{{< /note >}}

## Advisory lock flags

Support for advisory locks is {{<tags/feature/tp>}}.

To learn about advisory locks, see [Advisory locks](../../../explore/transactions/explicit-locking/#advisory-locks).

##### --ysql_yb_enable_advisory_locks

Enables advisory locking.

This value must match on all yb-master and yb-tserver configurations of a YugabyteDB cluster.

Default: false

## Advanced flags

##### --allowed_preview_flags_csv
Expand Down
20 changes: 20 additions & 0 deletions docs/content/preview/reference/configuration/yb-tserver.md
Original file line number Diff line number Diff line change
Expand Up @@ -1638,6 +1638,26 @@ After a DDL statement that includes updating DocDB system catalog completes, YB-
When the flag `ysql_ddl_transaction_wait_for_ddl_verification` is enabled, YSQL waits for any YB-Master background operations to finish before returning control to the user.
{{< /note >}}

## Advisory lock flags

Support for advisory locks is {{<tags/feature/tp>}}.

To learn about advisory locks, see [Advisory locks](../../../explore/transactions/explicit-locking/#advisory-locks).

##### --ysql_yb_enable_advisory_locks

Enables advisory locking.

This value must match on all yb-master and yb-tserver configurations of a YugabyteDB cluster.

Default: false

##### --num_advisory_locks_tablets

Number of tablets used for the advisory locks table. It must be set before ysql_yb_enable_advisory_locks is set to true on the cluster.

Default: 1

## Advanced flags

##### --allowed_preview_flags_csv
Expand Down
20 changes: 0 additions & 20 deletions docs/content/preview/yugabyte-voyager/known-issues/postgresql.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@ Review limitations and implement suggested workarounds to successfully migrate d
- [XID functions is not supported](#xid-functions-is-not-supported)
- [REFERENCING clause for triggers](#referencing-clause-for-triggers)
- [BEFORE ROW triggers on partitioned tables](#before-row-triggers-on-partitioned-tables)
- [Advisory locks is not yet implemented](#advisory-locks-is-not-yet-implemented)
- [System columns is not yet supported](#system-columns-is-not-yet-supported)
- [XML functions is not yet supported](#xml-functions-is-not-yet-supported)
- [Large Objects and its functions are currently not supported](#large-objects-and-its-functions-are-currently-not-supported)
Expand Down Expand Up @@ -1240,25 +1239,6 @@ EXECUTE FUNCTION check_and_modify_val();

---

### Advisory locks is not yet implemented
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has Voyager been updated to know that advisory locks are supported? And it is only so in 2.25.1, what if they migrate to a diffferent version?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ddhodge , Voyager changes are tracked by https://yugabyte.atlassian.net/browse/DB-13780. This update to the documentation is independant. Do you want the document changes to be done only after the Voyager changes land on master, so as to be consistent?

Copy link
Contributor

@priyanshi-yb priyanshi-yb Jan 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Voyager uses the link to this section in assessment reports and we have a different workflow to mark a feature as fixed in certain versions, we have to do this separately once Voyager also supports the new YB version where Advisory lock is implemented. We generally don't remove the section in this docs, we just mention the YB version where the feature is fixed.


**GitHub**: [Issue #3642](https://github.com/yugabyte/yugabyte-db/issues/3642)

**Description**: YugabyteDB does not support PostgreSQL advisory locks (for example, pg_advisory_lock, pg_try_advisory_lock). Any attempt to use advisory locks will result in a "function-not-implemented" error as per the following example:

```sql
yugabyte=# SELECT pg_advisory_lock(100), COUNT(*) FROM cars;
```

```output
ERROR: advisory locks are not yet implemented
HINT: If the app doesn't need strict functionality, this error can be silenced by using the GFlag yb_silence_advisory_locks_not_supported_error. See https://github.com/yugabyte/yugabyte-db/issues/3642 for details
```

**Workaround**: Implement a custom locking mechanism in the application to coordinate actions without relying on database-level advisory locks.

---

### System columns is not yet supported

**GitHub**: [Issue #24843](https://github.com/yugabyte/yugabyte-db/issues/24843)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ Voyager then generates a report that includes:
- **Recommended data distribution:** Suggests effective sharding strategies for tables and indexes.
- **Performance metrics:** Analyzes workload characteristics to recommend optimizations in YugabyteDB.
- **Migration time estimate:** Provides an estimated time for data import into YugabyteDB based on experimental data.
- **Unsupported query constructs:** Identifies SQL features and constructs not supported by YugabyteDB, such as advisory locks, system columns, and XML functions, and provides a list of queries containing these constructs.
- **Unsupported PL/pgSQL objects:** Identifies SQL features and constructs that are not supported by YugabyteDB, such as advisory locks, system columns, and XML functions, within PL/pgSQL objects in the source schema. It reports the individual queries within these objects that are not supported, such as queries in the PL/pgSQL block for functions and procedures, or the select statements in views and materialized views that contain unsupported constructs.
- **Unsupported query constructs:** Identifies SQL features and constructs not supported by YugabyteDB, such as system columns, and XML functions, and provides a list of queries containing these constructs.
- **Unsupported PL/pgSQL objects:** Identifies SQL features and constructs that are not supported by YugabyteDB, such as system columns, and XML functions, within PL/pgSQL objects in the source schema. It reports the individual queries within these objects that are not supported, such as queries in the PL/pgSQL block for functions and procedures, or the select statements in views and materialized views that contain unsupported constructs.

When running migration assessment, keep in mind the following:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,6 @@ The following are examples of payloads that are collected during some migration
}
],
"unsupported_query_constructs": {
"Advisory Locks": 2,
"System Columns": 0,
"XML Function": 1
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -210,10 +210,6 @@ ERROR: partman is not a valid partitioning type for pg_partman

The pg_partman `create_parent()` function requires an access exclusive lock on the parent table to create new child partitions. Currently, access exclusive locks are not supported in YugabyteDB, and are disabled in this function.

### ADVISORY LOCKS

Advisory locks, used in some pg_partman functions to create, drop/delete partitioned tables, are not supported in YugabyteDB. Attempts to acquire these locks are disabled.

### Background worker process

The pg_partman background worker process, which automates partition maintenance without the need of an external scheduler, is disabled in favor of using pg_cron to manage such tasks externally.
Expand Down