Skip to content

Release Notes

Latest Changes🔗

  • feat: reject multi-statement SQL before it reaches the driver. PR #591 by @zschumacher.
  • v1: reject multi-statement SQL before it reaches the driver. A pydapper command now executes exactly one SQL statement: SQL containing more than one raises the new MultipleStatementsError before a cursor is acquired, so nothing reaches the DBAPI. Previously the outcome was whatever the driver happened to do and no adapter raised — psycopg2 returned the rows of the last statement, psycopg the rows of the first, mysql-connector-python executed every statement including appended DDL, and sqlite3 raised a driver error whose type varies by Python version (on 3.10, a sqlite3.Warning, which is not a sqlite3.Error). Detection is a lexical scan sharing the placeholder scanner's skip set, not a SQL parser: a ; inside single-quoted text, double-quoted text, a -- line comment, or a /* ... */ block comment is ordinary text, and a single trailing ; followed only by whitespace and comments is still one statement. The guard runs in BaseSqlParamHandler.__init__, so it covers every sync and async command entry point on every first-party and third-party adapter, and it is certified by four new mandatory cases in the core-sync and core-async conformance profiles (62 cases per mode, up from 58). MySQL additionally denies the capability on the wire: connections pydapper opens now clear CLIENT_MULTI_STATEMENTS at connect time. Scripts and PL/SQL blocks belong on the DBAPI connection directly — see One statement per call.
  • fix: close block-exit gaps in the transactions conformance profile. PR #582 by @zschumacher.
  • fix: close three block-exit gaps in the transactions conformance profile found by an adversarial sweep of the milestone. Each was confirmed by mutating the runtime and watching the suite stay green. The profile now asserts that a block whose rollback failed does not then commit — losing the rollback must not silently become durability — and the mutant table pins that the block's exit routes through the adapter's own commit()/rollback() rather than straight to the connection, so an adapter override is honored there too. No runtime behavior changed, and every first-party adapter still passes both modes. The nesting RuntimeError now reads "cannot be nested or entered concurrently", because the per-instance guard also rejects a second asyncio task sharing the instance — it is not a synchronization primitive, and Transactions now says so rather than implying thread safety. The sync transaction() docstring gained the abandoned-block hazard its async twin already documented (a driver that commits on connection-context-manager exit makes the abandoned work durable before generator finalization can roll it back), and the async twin's "unlike the sync twin" now scopes only to the closed-event-loop clause it actually describes. Docs fixes: the #465 release-notes paragraph was truncated mid-sentence by a merge and is restored, the execute / execute_async pages now warn that neither commits and link the per-driver table (closing the funnel behind #68), and the aiopg section states its connection-context-manager exit behavior like every other driver page.
  • docs: document and test per-driver transaction behavior. PR #581 by @zschumacher.
  • v1: every database-support page now documents its driver's transaction behavior in a dedicated Transactions section — the autocommit default, exactly what exiting with pydapper.connect(...) does for that driver, and the server-side surprises. The driver context managers fall into three families: commit on clean exit (sqlite3, psycopg2 — neither closes; psycopg commits then closes), close only (mysql-connector-python — uncommitted DML is discarded; aiopg — everything was already durable), and close with an implicit rollback (pymssql, oracledb); BigQuery is its own case — its DBAPI connection has no context manager at all, so exit performs no driver call. The pymssql section resolves the long-standing explicit-commit confusion (#68): a non-autocommit pymssql connection holds an always-open BEGIN TRAN and close() rolls it back, so an insert inside a plain with connect(...) block reports success and then silently disappears — commit explicitly or use transaction(). MySQL and Oracle implicitly committing DDL statements, sqlite3's legacy isolation_level autocommitting DDL issued outside an open implicit transaction, and aiopg's autocommit-only model are likewise documented. The exit-behavior and implicit-commit claims are pinned by new live tests under each backend's marker, the Transactions page gained a per-driver summary table, the conformance matrix driver-limits cells carry the same facts, and the sqlite, mysql, pymssql, and oracledb live conformance runs gained the transactions-profile inclusion assert the postgres runs already had — all six declaring adapters now prove the profile actually ran.
  • test: make the transactions conformance profile defend itself. PR #580 by @zschumacher.
  • feat: add async commit, rollback, and transaction APIs. PR #579 by @zschumacher.
  • v1: CommandsAsync now exposes the async transaction APIs — commit(), rollback(), and a transaction() async context manager — with semantics identical to the sync APIs: commit on clean exit; rollback then re-raise on any exception (including KeyboardInterrupt) with the block's error winning over an ordinary rollback failure (a BaseException that is not an Exception — such as a task cancellation — raised during the rollback is never swallowed and propagates in its place); a failed exit commit propagates unchanged with no rollback attempt; no SQL on enter; and RuntimeError on nested blocks on the same instance. The methods are deliberately unsuffixed — the class is async-only, so an _async suffix would be redundant — and all three are gated behind AdapterCapability.TRANSACTIONS, raising UnsupportedFeatureError for adapters that do not declare it. Psycopg3CommandsAsync declares the capability; AiopgCommands does not, because aiopg always runs in autocommit mode and its connection-level commit()/rollback() raise. The transactions conformance profile now ships nine async cases mirroring the sync inventory case-for-case, the runners exercise every declaring async adapter automatically, and the repository's fake async driver gained snapshot transactionality so the async profile has service-free live coverage. See Transactions.
  • fix: pin every assertion in the transactions conformance profile, and cover two behaviors it missed. The shipped profile is the third-party-facing definition of "implements transactions correctly", but none of its own assertions was under test: deleting all of them left the suite green, and deliberately non-conformant adapters still earned a clean certificate. A table of non-conformant adapters now pins every assertion — in both modes — to the one case and the one failure message that must catch it, the profile's inventory is asserted as a literal list of case ids (an expectation derived from capability_profiles() can never notice the profile shrinking), and the documented case table is checked against the shipped one. The profile grows from nine cases per mode to eleven, sync and async alike: transactions.context-base-exception-rollback (a BaseException that is not an Exception — a KeyboardInterrupt, say — still rolls back, never commits, and propagates as the same object, and an interrupt raised by the rollback itself wins over the block's ordinary error, which survives as its __context__) and transactions.context-not-reentrant (nested blocks on one command instance raise RuntimeError and roll the outer block back, and the guard clears so the next sequential block still commits). No runtime behavior changed; every first-party adapter already passes both new cases in both modes.
  • fix: close five gaps in the owned-resource and adapter-registration contracts. PR #576 by @zschumacher.
  • fix: identify connections by type, not repr, in selection errors. PR #575 by @zschumacher.
  • fix: verify preparation-hook ordering on the conformance query paths. PR #573 by @zschumacher.
  • docs: document command-owned cursor exits and the #541 behavior change. PR #574 by @zschumacher.
  • feat: add sync commit, rollback, and transaction APIs. PR #572 by @zschumacher.
  • v1: synchronous commands now expose explicit transaction APIs — commit(), rollback(), and a transaction() context manager that commits on clean exit and rolls back (then re-raises) on any exception, including KeyboardInterrupt. The block's error always wins over a rollback failure, a failed exit commit propagates unchanged with no rollback attempt, entering a block emits no SQL (the DBAPI's implicit transaction start is the contract), and nested blocks on the same Commands instance raise RuntimeError. All three methods are gated behind AdapterCapability.TRANSACTIONS: an adapter that does not declare it raises UnsupportedFeatureError before the connection is touched. Every first-party sync adapter declares the capability except BigQuery, whose DBAPI has no connection-level transactions (commit() is a no-op and there is no rollback()); async command classes stay undeclared until the async transaction APIs land. This is also the first shipped conformance capability profile: capability_profiles() now carries transactions (nine sync cases — durability, rollback, context-manager lifecycle, delegation, and error precedence), the runners automatically exercise it for every declaring adapter, and the previously provisional capability surface (ConformanceProfile, SyncCase, AsyncCase, capability_profiles()) is now stable. See Transactions.
  • fix: keep connection representations out of automatic adapter-selection errors. The zero-match and ambiguous-match using() / using_async() failures now identify the connection by the module-qualified name of its type — for example 'sqlite3.Connection' — instead of interpolating repr(connection), so a driver representation that embeds the DSN the connection was opened with can no longer reach a pydapper error message. Both messages still name the requested mode, the matching adapters in the ambiguous case, and the adapter= escape hatch.
  • fix: harden command-owned cursor cleanup and complete the async drain seam. Four user-observable behaviors changed. First, cleaning up a command-owned cursor no longer discards a BaseException that is not an Exception: a KeyboardInterrupt, SystemExit, asyncio.CancelledError, or GeneratorExit raised by a cursor's __exit__/__aexit__ or close() while a command error is already propagating now propagates in place of that command error, which is preserved as its __context__. Previously a Ctrl-C or a cancellation raised during cleanup was silently swallowed. Ordinary Exception cleanup failures are still discarded so the active command error is re-raised unchanged, but they are now recorded at DEBUG on the pydapper._context logger with their traceback instead of vanishing. Second, a cursor whose __enter__/__aenter__ raises is now closed best-effort instead of leaking; pydapper created and solely owns that cursor, __exit__/__aexit__ is still not called for a cursor that never entered, and a failing close() never replaces the entry error. Third, CommandsAsync gained _on_query_single_more_than_one_result_async, the previously missing async twin of the sync query_single drain seam, invoked at the same point of query_single_async, so sync and async exception timing match for an adapter that must drain unread rows before its cursor can be closed. The default is a no-op and no first-party adapter overrides it. Documenting the _on_* drain seams alongside the _prepare_* hooks is deliberately deferred to the v1 export and typing audit (#484); they remain private hooks outside the documented compatibility surface. Fourth, cancelling a task that is awaiting an async cursor no longer leaks that cursor when the cancellation lands after acquisition has already completed. Acquisition runs as an independent future, so await and async with could raise CancelledError while that future held a fully created cursor with no owner: the raising await binds nothing and a failed __aenter__ is never exited, so nothing was left to close it. The stranded cursor is now closed best-effort through the same discard policy, the CancelledError is never suppressed, and a cursor another task is concurrently awaiting is never closed. A wrapper that closed its resource this way — on a cancelled acquisition or on a failed __aenter__, which share one policy — is spent, and a later await or async with on it raises RuntimeError instead of handing the closed cursor out again. This does not fix the same race for connect_async(), which still strands its connection: connect_async() resolves to a CommandsAsync, and CommandsAsync exposes no close(), so pydapper has nothing it can close there. A wrapper that closed nothing is deliberately never marked spent, so awaiting a cancelled connect_async() wrapper again still returns the live CommandsAsync — that is the only remaining way to reach the open connection and close it.
  • feat: add reusable adapter conformance suite and capability profiles. PR #565 by @zschumacher.
  • v1: a reusable adapter conformance suite now ships in the installed distribution at pydapper.testing.adapter_conformance. Adapter authors implement a typed SyncAdapterHarness / AsyncAdapterHarness and run the mandatory core-sync and core-async profiles against their real concrete command classes — no pytest, no optional database drivers, and no copied repository tests required. The two mode profiles are independent (a dual-mode adapter must pass both), results and failures are structured and deterministic (profile id, case id, original cause, and missing harness field), and framework-owned recording/fault-injection connections prove lifecycle behavior — cursor cleanup on success and failure, active-error precedence over cleanup failures, truthy-exit non-suppression, bounded single-row fetches, and fail-before-driver-work validation — through the real adapter classes rather than mocks of them. Harness configuration fields may be declared on the subclass or assigned per instance, so a DSN that only exists once a test container has started needs no type: ignore. A failure inside the harness's own create_commands() is attributed to the harness through CaseResult.harness_setup_failed / ConformanceReport.harness_setup_failed and never reported as adapter behavior, so one broken seeding call cannot masquerade as dozens of behavioral failures. Both runners take an optional case_ids filter for shortening the debug loop against a slow backend: unknown ids and empty selections raise CaseSelectionError before anything runs, and a filtered report carries covers_full_inventory=False, so only report.passed and report.covers_full_inventory is a conformance result. Optional behaviors remain independent AdapterCapability profiles: the catalog starts empty, zero-case profiles are rejected, and a declared capability without a populated profile fails conformance, so an unimplemented capability can never appear covered — that capability surface is provisional until the first profile ships. All eight first-party adapters (nine command classes) adopt the suite in the repository's backend test suites. See Adapter conformance.
  • feat: declare first-party adapters as installed entry points and remove eager bootstrap. PR #564 by @zschumacher.
  • v1: adapters are now discovered through the standard pydapper.adapters entry-point group and load lazily. First-party and third-party adapters use one provider contract: an installed distribution declares an entry point per adapter name whose synchronous zero-argument callback registers exactly that name through the public register_adapter(). The eight stable first-party names (sqlite3, psycopg2, psycopg, aiopg, mysql, pymssql, oracledb, google) are unchanged and are now declared by the pydapper distribution itself; the private eager first-party bootstrap was removed, so a plain import pydapper no longer registers adapters, imports adapter command modules, or imports optional database drivers. DSN-based connect() / connect_async() and explicit using(..., adapter=name) / using_async(..., adapter=name) selection load only the requested provider (explicit selection still bypasses connection predicates), while automatic using() / using_async() selection loads every installed provider before evaluating predicates, so an unrelated broken provider can fail automatic selection but never exact-name selection. Precedence is deterministic: a direct runtime register_adapter() call wins for the process, the first-party provider wins over an external provider using the same name, and duplicate external providers fail before either loads with every conflicting distribution named — metadata enumeration order never chooses a winner. Provider failures identify the adapter and provider distribution, preserve the original exception as the cause, and roll back their registry mutations; successful callbacks run at most once per process and the installed catalog is cached per process. Runtime registration via register_adapter() remains fully supported, no public API is added or deprecated, and no new runtime dependency is introduced. See Adapter registration for the packaging contract for adapter authors.
  • feat: extract first-party adapter provider callbacks. PR #563 by @zschumacher.
  • feat: load installed adapter providers before automatic selection. PR #562 by @zschumacher.
  • feat: add private deterministic load-all-providers pass. PR #561 by @zschumacher.
  • feat: resolve installed adapter providers from name-based public paths. PR #560 by @zschumacher.
  • feat: add private exact-name adapter resolution and provider precedence. PR #559 by @zschumacher.
  • feat: add private transactional loader for adapter provider entry points. PR #558 by @zschumacher.
  • feat: replace dsnparse with an owned URL DSN parser. PR #557 by @zschumacher.
  • v1: pydapper now owns its narrow URL-style DSN parser using the standard library's urllib.parse. A direct implementation is smaller and lower risk than vendoring a generic parser API that pydapper does not consume. This removes the mandatory dsnparse dependency, so it is no longer installed with pydapper, while preserving default and exact explicit adapter routing, including third-party adapters. Parse-result fields now have accurate public types, and representations and parser-generated errors no longer expose credential-bearing DSNs or passwords. Decoded network hosts are revalidated so encoded controls and Unicode delimiter lookalikes cannot bypass authority parsing.
  • feat: add private lazy entry-point discovery catalog for pydapper.adapters. PR #556 by @zschumacher.
  • feat: validate capability declarations and add command preparation hooks. PR #555 by @zschumacher.
  • feat: validate adapter capability declarations and add command preparation hooks. register_adapter() now validates each supplied command class's capabilities declaration (a frozenset of AdapterCapability members) for both modes before the registry is touched, so an invalid declaration fails atomically. Every first-party command class explicitly declares its current — empty — capability set, and commands.supports(AdapterCapability.X) reports declared support. New documented, compatibility-sensitive adapter-author preparation seams run inside the command-owned cursor lifecycle for every sync and async command family: _prepare_cursor*() once per acquired cursor and _prepare_command*() once per executed handler, both receiving a normalized CommandOptions instance. No optional capability (transactions, timeouts, stored procedures, readonly, max rows, etc.) is implemented by this change. The *_or_default helpers now forward the normalized options to query_first / query_single; see Breaking Changes below for the subclass-override impact.
  • feat: add AdapterCapability vocabulary and command capability checks. PR #554 by @zschumacher.
  • test: cover query_multiple runtime failures inside the command-owned cursor lifecycle. PR #553 by @zschumacher.
  • v1: a driver cursor can no longer suppress an internal command failure. Every sync and async command method does all of its result work — execution, fetching, cardinality checks, duplicate-column validation, scalar extraction, and row projection — inside the one cursor the command owns, and that cursor's disposal can no longer change the outcome of the call: a native cursor __exit__ / __aexit__ that returns a truthy value is ignored while a command error is active, and a cleanup failure never replaces an active command error, which still propagates as the same exception object after cleanup runs exactly once. Previously a driver cursor exit could change that outcome in two distinct ways. A truthy exit swallowed the real failure and let the command fall through instead of raising, so a caller could receive a partial or invalid value in place of the error — a query_multiple() tuple shorter than the batch it was given, None from a query() that promises a list of rows, or an unrelated UnboundLocalError from work the swallowed failure never completed. An exit that raised replaced the real command error with its own cleanup exception, so the caller was told the cursor failed to close rather than why the command failed. This rule is scoped to cursors pydapper owns internally; the user-visible connection context managers (with pydapper.connect(...) as commands: and async with pydapper.connect_async(...) as commands:) still proxy to the dbapi connection and keep ordinary Python suppression semantics. See Command-owned cursor lifecycle.
  • fix: project query_single rows inside the command-owned cursor lifecycle. PR #551 by @zschumacher.
  • fix: run mysql query_first inside the command-owned cursor lifecycle. PR #552 by @zschumacher.
  • fix: project query_single rows inside the command-owned cursor lifecycle. PR #550 by @zschumacher.
  • fix: validate and extract scalar results inside the command-owned cursor lifecycle. PR #549 by @zschumacher.
  • fix: project query_first rows inside the command-owned cursor lifecycle. PR #548 by @zschumacher.
  • fix: async command-owned cursor exception precedence. PR #547 by @zschumacher.
  • fix: sync command-owned cursor exception precedence. PR #546 by @zschumacher.
  • fix: validate complete query batches before DBAPI work. PR #545 by @zschumacher.
  • fix: validate complete query batches before DBAPI work. query_multiple and query_multiple_async now construct and validate every parameter handler before acquiring a cursor, so a missing or invalid parameter in any query of the tuple fails before any query executes or fetches. This is client-side prevalidation, not transactional execution; a validated batch can still fail partway through on a runtime database, fetch, column, or mapping error.
  • chore(ai): Add CLAUDE.md symlink to AGENTS.md. PR #544 by @zschumacher.
  • fix: harden async context lifecycle. PR #540 by @zschumacher.
  • fix: harden async context lifecycle. Async resources now resolve exactly once and context-manager exception suppression propagates correctly; both await connect_async() and async with connect_async() usage remain unchanged.
  • feat: stabilize adapter registration. PR #539 by @zschumacher.

Breaking Changes🔗

  • Multi-statement SQL now raises MultipleStatementsError on every adapter instead of being handed to the driver. SQL that previously "worked" by accident behaves differently everywhere: on psycopg2 and aiopg db.query("select 1 as a; select 2 as b") returned [{'b': 2}], on psycopg it returned [{'a': 1}], on mysql-connector-python every appended statement executed, and on sqlite3 it raised a driver error whose type depends on the Python version — on 3.10 a sqlite3.Warning, which except sqlite3.Error cannot catch. All of them now raise before a cursor is acquired. Split the statements into separate calls, or run the script against the DBAPI connection directly (commands.connection.cursor()).
  • mysql-connector-python connections opened by pydapper no longer negotiate CLIENT_MULTI_STATEMENTS. pydapper.connect("mysql://...") now passes client_flags clearing that bit, and clears it even when you supply your own: a list or tuple keeps your entries and has the denial appended, and a positive int is passed through with only that bit masked off. Code that relied on sending a statement batch through a pydapper-opened MySQL connection — including through a raw cursor taken off commands.connection — now gets the server's own syntax error. Open your own connection and pass it to using() to keep the flag. Two kinds of input now raise ValueError instead of connecting: client_flags equal to exactly CLIENT_MULTI_STATEMENTS, and falsy values that are neither an integer, None, nor an empty list or tuple ('', set(), {}). Both are cases where the driver resolves client_flags or ClientFlag.get_default() and would silently restore a default that has the flag set, so pydapper refuses rather than negotiating a capability it just denied. None, 0, False, and an empty list or tuple are all read as "use the default" and resolve to the denial. One consequence: because pydapper always passes client_flags, a client_flags entry in a my.cnf loaded via option_files is now ignored — the driver only applies option-file values for keys absent from the connect arguments. Pass those flags to connect() instead and pydapper will preserve them.
  • The guard's notion of "literal" and "comment" is one fixed ANSI-flavored set, so SQL using dialect-specific forms can now be refused even though it is a single statement. Confirmed cases: a ; inside a mysql backtick identifier (select `a;b` from t) or a pymssql bracket identifier (select [a;b] from t), and a mysql # comment following a trailing semicolon (select 1 as a; # done). Rewrite the identifier or comment, or use a -- comment. Making the literal forms per-adapter is tracked in #590.
  • sql must now be a str. bytes and bytearray previously reached the driver — and, because the scanner is a str lexer, bypassed the multi-statement guard entirely — and now raise TypeError before any driver work. Decode the value before passing it.
  • A top-level ; inside a PostgreSQL dollar-quoted body (psycopg2, psycopg) or an Oracle anonymous PL/SQL block (oracledb) is now rejected. The guard is lexical and has no dialect awareness, so DO $$ ... ; ... $$ and BEGIN ... ; ... END; trip it even though they are one statement to the server. This is deliberate: the alternative — exempting SQL by leading keyword — would let BEGIN; select 1; commit; through on the default PostgreSQL adapter, which is the exact shape the guard exists to stop. Run these blocks on the DBAPI connection directly: commands.connection.cursor().execute(plsql_block).
  • DSNs now follow the focused <database>[+<adapter>]://... v1 grammar. Incidental inherited conveniences such as name=value strings, constructor default injection, dict-like mutation, tuple-style iteration or indexing, environment helpers, and multiple hosts are intentionally not reproduced. Multi-plus input such as db+adapter+extra://... previously invented the adapter name adapter_extra; it is now rejected. Query values are no longer implicitly coerced: for example, ?code=001&enabled=true previously produced 1 and True but now preserves "001" and "true"; a bare query key such as ?flag now maps to {"flag": ""} instead of raising. Corrected DSN examples place ports after the host; literal and percent-encoded colons in passwords remain valid. SQLite slash counts are now explicit: sqlite:///relative/path.db targets relative/path.db; use sqlite:////absolute/path.db for /absolute/path.db (the three-slash form previously targeted /relative/path.db).
  • Command delegation: query_first_or_default, query_single_or_default, and their async equivalents now forward the normalized options= keyword to query_first / query_single (and the async equivalents) instead of validating and discarding it. Subclass overrides of those four target methods must accept an options= keyword; overrides without it now raise TypeError when called through the *_or_default helpers. Add *, options=None to the override signature to migrate.
  • Adapter registration: register and register_async were removed. Use register_adapter and explicit adapter= selection where needed.

Other Changes🔗

0.13.1🔗

Internal🔗

0.13.0🔗

Internal🔗

0.12.0🔗

Features🔗

Bug Fixes🔗

  • 🔧 fix sqlite not opening in subdirectories. PR #344 by @arieroos.

Internal🔗

0.11.1🔗

Features🔗

  • ✨ Add callable that returns a model as a supported model type. PR #333 by @zschumacher.

0.11.0🔗

Breaking changes🔗

  • Python 3.8 support deprecated
  • cx_oracle support deprecated

Internal🔗

0.10.0🔗

Features🔗

Internal🔗

Docs🔗

0.9.0🔗

Bug fixes🔗

Internal🔗

0.8.0🔗

Features🔗

Internal🔗

Docs🔗

0.7.0🔗

Features🔗

Internal🔗

0.6.0🔗

Features🔗

0.5.3🔗

Internal🔗

  • 🔧 Add variable length tuple typing annotation for query_multiple. PR #67 by @enewnham.
  • 🔧 Dependabot updates 2022-10-28. PR #85 by @zschumacher.
  • 🔧 Better developer support for arm chips. PR #52 by @zschumacher.

0.5.2🔗

Docs🔗

  • 🔧 Add example for serializing one-to-many relationships to docs. PR #44 by @zschumacher.

Internal🔗

0.5.1🔗

Internal🔗

0.5.0🔗

Features🔗

Internal🔗

0.4.0🔗

Features🔗

0.3.0🔗

Features🔗

  • ✨ support PYDAPPER_DSN environment variable for connections. PR #21 by @zschumacher.

Internal🔗

  • 🔧 Cache oracle-instantclient download in test workflow. PR #20 by @zschumacher.

0.2.0🔗

Features🔗

0.1.2🔗

🚀 First stable release of pydapper!